Electronic device, control method of electronic device, and computer readable medium

Abstract

The present invention provides a display control device, a display control method and a computer-readable medium. The display control device includes: a display controller configured to display a graphic image together with an HDR image on a display unit, wherein the display controller is further configured to display the graphic image at a second brightness when the brightness of the HDR image is a first brightness, and to display the graphic image at a fourth brightness higher than the second brightness when the brightness of the HDR image is a third brightness higher than the first brightness.

Description

Technical Field

[0001] The present invention relates to a display control device, a display control method and a computer readable medium. Background Art

[0002] In recent years, advances in LED elements and the like have led to wider use of displays supporting HDR (High Dynamic Range), which enables the upper limit brightness (maximum brightness; peak brightness) to be enhanced by adjusting the brightness of black to be darker. Displays supporting HDR are capable of displaying image data with a high dynamic range (HDR) without compressing the image data. In HDR, scenes with reduced contrast in a conventional dynamic range (hereinafter referred to as SDR: Standard Dynamic Range), such as clouds in a blue sky and neon lights in a night scene, can be reproduced with higher realism (with improved grayscale). Although this HDR is mainly used in moving images to be recorded on Blu-ray discs and the like, its future application is expected to expand to still images.

[0003] On the rear LCD display of a camera device, when the captured image is played back and displayed, shooting information (speed, aperture value, ISO sensitivity and image name) and an OSD (graphic image) of status information during playback are often superimposed on the captured image.

[0004] Japanese Patent Laid-Open No. 2019-057824 discloses receiving peak brightness or gamma from a display device and adjusting the OSD to be brighter in advance when it is determined that the brightness of the OSD will become darker, and also discloses determining which of an image and an OSD is important, thereby adjusting the OSD to a brightness suitable for one of them. WO 2016 / 038950 discloses that when an OSD with SDR (100 cd / m 2 ) is converted to HDR data, the brightness grayscale value of the OSD is set to diffuse white (which corresponds to the dominant white in the image (300 to 500 cd / m 2 )).

[0005] However, a problem with the conventional technology disclosed in Japanese Patent Laid-Open No. 2019-057824 and WO 2016 / 038950 is that the visibility of the OSD is poor when the OSD is displayed as bright or dark relative to the rear image. Summary of the invention

[0006] In view of this, an object of the present invention is to display an OSD with appropriate brightness.

[0007] In its first aspect, the present invention provides a display control device, comprising: a display controller, which is configured to display a graphic image together with an HDR image on a display unit, wherein the display controller is also configured to (1) display the graphic image at a second brightness when the brightness of the HDR image is a first brightness, and (2) display the graphic image at a fourth brightness higher than the second brightness when the brightness of the HDR image is a third brightness higher than the first brightness.

[0008] The present invention provides a display control method, comprising: displaying a graphic image together with an HDR image on a display unit; displaying the graphic image at a second brightness when the brightness of the HDR image is a first brightness; and displaying the graphic image at a fourth brightness higher than the second brightness when the brightness of the HDR image is a third brightness higher than the first brightness.

[0009] The present invention provides a computer-readable medium storing a program, wherein the program enables a computer to execute the above-mentioned display control method.

[0010] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1A and 1B is an external view of a digital camera according to a first embodiment;

[0012] Figure 2 is a block diagram showing the structure of a digital camera according to a first embodiment;

[0013] Figure 3 is a diagram showing connection of an external device according to a first embodiment;

[0014] Figure 4A is a flowchart showing the LV shooting mode processing according to the first embodiment;

[0015] Figure 4B is a flowchart showing a quick review display process according to the first embodiment;

[0016] Figures 5A to 5C is a sequence diagram of connection processing according to the first embodiment;

[0017] Fig. 6A is a flowchart showing a shooting menu process according to the first embodiment;

[0018] Figure 6B is a flowchart showing a shooting menu process according to the first embodiment;

[0019] Fig. 7Ais a flowchart showing an HDR shooting process according to the first embodiment;

[0020] Figure 7B is a flowchart showing an HDR shooting process according to the first embodiment;

[0021] Figure 7C is a diagram showing a CxCy plane according to the first embodiment;

[0022] Fig.7D is a flowchart showing an HDR shooting process according to the first embodiment;

[0023] Fig. 7E is a diagram showing an example of a display image according to the first embodiment;

[0024] Figure 7F is a diagram showing an example of a display image according to the first embodiment;

[0025] Figure 7G is a diagram showing an example of a gray scale correction amount according to the first embodiment;

[0026] Figure 7H is a diagram showing an example of a gray scale correction amount according to the first embodiment;

[0027] Figures 8A to 8E is a diagram showing a configuration example of a file according to the first embodiment;

[0028] Fig. 9A is a flowchart showing playback mode processing according to the first embodiment;

[0029] Fig. 9B is a flowchart showing playback mode processing according to the first embodiment;

[0030] Fig. 9C is a flowchart showing playback mode processing according to the first embodiment;

[0031] Fig.9D is a flowchart showing playback mode processing according to the first embodiment;

[0032] Fig.9E is a flowchart showing playback mode processing according to the first embodiment;

[0033] Fig.9F is a flowchart showing playback mode processing according to the first embodiment;

[0034] Figure 9G is a flowchart showing playback mode processing according to the first embodiment;

[0035] Figure 9H is a flowchart showing playback mode processing according to the first embodiment;

[0036] Fig. 10A is a flowchart showing playback mode processing according to the first embodiment;

[0037] Fig. 10B is a flowchart showing playback mode processing according to the first embodiment;

[0038] Fig.11A is a flowchart showing playback menu processing according to the first embodiment;

[0039] Fig. 11B is a functional block diagram related to RAW development processing according to the first embodiment;

[0040] Fig. 12A is a flowchart showing an OSD brightness setting process according to the first embodiment;

[0041] Fig. 12B is a flowchart showing an OSD brightness setting process according to the first embodiment;

[0042] Fig. 12C is a flowchart showing an OSD brightness setting process according to the first embodiment;

[0043] Figures 13A to 13E is a diagram showing an example of OSD information display;

[0044] Fig.14 is a diagram showing OSD information display according to the first embodiment;

[0045] Fig.15 : is a diagram showing the correspondence relationship between the photographing conditions and the luminance grayscale values ​​according to the second embodiment.

[0046] Fig.16 is a flow chart showing brightness changes according to the second embodiment;

[0047] Figures 17A to 17C is a diagram showing an example of a display image according to the third embodiment;

[0048] Fig.18 is a flowchart showing an OSD brightness setting process according to a third embodiment;

[0049] Fig.19A and 19B is a diagram showing an example of a display image according to the fourth embodiment; and

[0050] Fig. 20 is a flowchart showing an OSD brightness setting process according to the fourth embodiment. DETAILED DESCRIPTION

[0051] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be understood that the embodiments described below represent only examples of methods for implementing the present invention, and can be appropriately corrected or modified according to the structure of the device to which the present invention is applied and various conditions. In addition, the various embodiments can also be appropriately combined.

[0052] (First embodiment)

[0053] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. Figure 1A and 1B An external appearance of a digital camera 100 is shown as an example of a device to which the present invention can be applied. Figure 1A is a front perspective view of the digital camera 100, and Figure 1B is a rear perspective view of the digital camera 100 .

[0054] The display unit 28 is a display provided on the back of the digital camera 100 for displaying images and various types of information. The viewfinder-out display unit 43 is a display provided on the upper surface of the digital camera 100 for displaying various setting values ​​of the digital camera 100 such as shutter speed and aperture. The terminal cover 40 is a cover for protecting a connector (not shown) of a connection cable or the like for connecting the digital camera 100 to an external device. The quick return mirror 12 is raised and lowered by an actuator (not shown) under instructions from a system control unit 50 (to be described later). The communication terminal 10 is a communication terminal used by the digital camera 100 to communicate with a lens unit 150 (to be described later; attachable and detachable). The eyepiece viewfinder 16 is an endoscopic viewfinder for checking the focus and composition of an optical image of a subject obtained through the lens unit 150 by observing a focusing screen 13 (to be described later). The cover 202 is a cover of a slot for storing a recording medium 200 (to be described later). The grip portion 91 is a holding portion configured in a shape that can be easily gripped by the user's right hand when holding the digital camera 100 .

[0055] In addition, the digital camera 100 has a mode switching switch 60, a shutter button 61, a main electronic dial 71, a power switch 72, a sub-electronic dial 73, a four-way key 74, a set button 75, and an LV button 76. The digital camera 100 also has a zoom-in button 77, a zoom-out button 78, and a playback button 79. The digital camera 100 may have other operating members. The various operating members will be described later.

[0056] Figure 2 1 is a block diagram showing a configuration example of the digital camera 100 .

[0057] The lens unit 150 is a lens unit to which an interchangeable photographic lens is mounted. Although the lens 103 is generally composed of a plurality of lenses, Figure 2, the lens 103 is shown as only one lens in a simplified manner. The communication terminal 6 is a communication terminal used by the lens unit 150 to communicate with the digital camera 100, and the communication terminal 10 is a communication terminal used by the digital camera 100 to communicate with the lens unit 150. The lens unit 150 communicates with the system control unit 50 via the communication terminals 6 and 10. In addition, the lens unit 150 controls the aperture 1 via the aperture drive circuit 2 using the lens system control circuit 4 inside. Furthermore, the lens unit 150 performs focusing by shifting the position of the lens 103 via the AF drive circuit 3 using the lens system control circuit 4.

[0058] The AE (Auto Exposure) sensor 17 measures the brightness of a subject (subject light) passing through the lens unit 150 .

[0059] The focus detection unit 11 outputs information about the defocus amount to the system control unit 50. The system control unit 50 controls the lens unit 150 based on the defocus amount information to perform phase difference AF.

[0060] During exposure, live view shooting, moving image shooting, etc., the quick return mirror 12 (hereinafter, mirror 12) is raised and lowered by an actuator (not shown) under instructions from the system control unit 50. The mirror 12 is a mirror for switching the light beam incident from the lens 103 between the viewfinder 16 side and the image pickup unit 22 side. Although the mirror 12 is usually arranged to guide (reflect) the light beam to the viewfinder 16 (mirror down), when shooting and live view display are performed, the mirror 12 is flipped up to guide the light beam to the image pickup unit 22 and retreat from the inside of the light beam (mirror up). In addition, the mirror 12 is configured as a semi-transparent and semi-reflective mirror so that light can partially pass through its central portion, and the mirror 12 allows a part of the light beam to pass through and be incident on the focus detection unit 11 for focus detection.

[0061] By observing the focusing screen 13 via the pentaprism 14 and the viewfinder 16 , the photographer can check the focus and composition of the optical image of the subject obtained through the lens unit 150 .

[0062] The shutter 101 is a focal plane shutter capable of freely controlling the exposure time of the imaging unit 22 under the control of the system control unit 50 .

[0063] The imaging unit 22 is an imaging element constituted by a device that converts an optical image into an electric signal, such as a CCD, a CMOS, etc. The A / D converter 23 converts an analog signal output from the imaging unit 22 into a digital signal.

[0064] The image processing unit 24 performs prescribed processing (pixel interpolation, resizing processing such as reduction, and color conversion processing, etc.) on the data from the A / D converter 23 or the data from the memory control unit 15. In addition, the image processing unit 24 performs prescribed arithmetic processing using the image data of the captured image, and the system control unit 50 performs exposure control and distance measurement control based on the calculation results obtained by the image processing unit 24. Therefore, processing such as AF (auto focus) processing, AE processing, and EF (pre-flash pre-emission) processing in the TTL (through the lens) method is performed. The image processing unit 24 also performs prescribed arithmetic processing using the image data of the captured image, and performs AWB (auto white balance) processing in the TTL method based on the obtained calculation results.

[0065] The output data from the A / D converter 23 is written to the memory 32 via the image processing unit 24 and the memory control unit 15, or is directly written to the memory 32 via the memory control unit 15. The memory 32 stores the image data obtained by the imaging unit 22 and converted into digital data by the A / D converter 23, and the image data to be displayed on the display unit 28. The memory 32 has a sufficient storage capacity for storing a prescribed number of still images and a prescribed time of moving images and audio.

[0066] In addition, the memory 32 also serves as a memory (video memory) for image display. The D / A converter 19 converts the data for image display stored in the memory 32 into an analog signal, and supplies the analog signal to the display unit 28. In this way, the image data for display written in the memory 32 is displayed by the display unit 28 via the D / A converter 19. The display unit 28 displays on a display such as an LCD according to the analog signal from the D / A converter 19. By subjecting the digital signal that has undergone A / D conversion by the A / D converter 23 to D / A conversion by the D / A converter 19 and storing it in the memory 32, and sequentially sending the D / A converted signal to the display unit 28 and displaying the signal, an electronic viewfinder function can be realized and real-time image display (live view display) can be performed. Hereinafter, the image displayed by the live view display will be referred to as an "LV image".

[0067] The in-viewfinder display unit 41 displays a frame (AF frame) indicating a distance measurement point currently being autofocused, an icon indicating a setting state of the camera, and the like via the in-viewfinder display unit drive circuit 42 .

[0068] The out-of-viewfinder display unit 43 displays various setting values ​​including the shutter speed and the aperture of the digital camera 100 via the out-of-viewfinder display unit drive circuit 44 .

[0069] The digital output I / F 90 supplies the data for image display stored in the memory 32 to the external device 300 without modifying the form of the digital signal. In this way, the image data for display that has been written in the memory 32 is displayed by the external device 300.

[0070] The nonvolatile memory 56 is an electrically erasable and recordable memory, and uses, for example, an EEPROM. Constants and programs used for the operation of the system control unit 50 are stored in the nonvolatile memory 56. In this case, the program refers to a program for executing various flowcharts described later in this embodiment.

[0071] The system control unit 50 is a control unit composed of at least one processor or one circuit, and controls the entire digital camera 100. The system control unit 50 realizes the respective processing steps of the present embodiment (to be described later) by executing the program recorded in the above-mentioned nonvolatile memory 56. The system memory 52 is, for example, a RAM, and the system control unit 50 develops constants and variables used for the operation of the system control unit 50, and the program read from the nonvolatile memory 56, etc. on the system memory 52. ​​In addition, the system control unit 50 also performs display control by controlling the memory 32, the D / A converter 19, the display unit 28, and the like.

[0072] The system timer 53 is a time measuring unit for measuring time used in various controls and measuring time corresponding to an internal clock.

[0073] The power control unit 80 is composed of a battery detection circuit, a DC-DC converter, and a switching circuit for switching between blocks to be powered, etc., and detects whether a battery is installed, the type of battery, and the remaining battery power, etc. In addition, the power control unit 80 controls the DC-DC converter based on the detection result and the instruction from the system control unit 50, and supplies necessary voltages to the respective units including the recording medium 200 for a necessary period of time. The power unit 30 is composed of a primary battery such as an alkaline battery, a lithium battery, etc., a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, or an AC adapter, etc.

[0074] The recording medium I / F 18 is an interface with a recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a recording medium such as a memory card for recording captured images, and is composed of a semiconductor memory, a magnetic disk, or the like.

[0075] The communication unit 54 transmits and receives video signals and audio signals to and from an external device connected wirelessly or by a wired cable. The communication unit 54 can also be connected to a wireless LAN (local area network) or the Internet. In addition, the communication unit 54 can also communicate with an external device via Bluetooth (registered trademark) or Bluetooth low energy. The communication unit 54 can transmit images (including LV images) captured by the camera unit 22 and images recorded on the recording medium 200, and receive image data and various other types of information from an external device.

[0076] The posture detection unit 55 detects the posture of the digital camera 100 with respect to the gravity direction. Based on the posture detected by the posture detection unit 55, it can be determined whether the image captured by the imaging unit 22 is an image captured while the digital camera 100 is held horizontally or an image captured while the digital camera 100 is held vertically. The system control unit 50 can add posture information corresponding to the posture detected by the posture detection unit 55 to an image file of an image (captured image) that has been captured by the imaging unit 22, and record a rotated version of the image, etc. An acceleration sensor, a gyro sensor, or the like can be used as the posture detection unit 55. It is also possible to detect the motion (panning, tilting, lifting, whether it is still, etc.) of the digital camera 100 using the acceleration sensor or the gyro sensor constituting the posture detection unit 55.

[0077] The operation unit 70 is used to input various operation instructions to the system control unit 50. The operation unit 70 includes various operation members as an input unit for accepting operations from the user (user operation). For example, the operation unit 70 includes a push button, a rotary dial, a touch sensor, etc. Specifically, the operation unit 70 includes a mode switching switch 60, a shutter button 61, a main electronic dial 71, a power switch 72, a sub-electronic dial 73, and a four-way key 74. In addition, the operation unit 70 includes a setting button 75, an LV button 76, a zoom-in button 77, a zoom-out button 78, and a playback button 79. In addition, the operation unit 70 includes an AF on button 70b, a quick setting button 70c, an active frame switching button 70d, a menu button 70e, a function button 70f, and an information button 70g. By selecting and operating various function icons displayed on the display unit 28 or the external device 300, each operation member of the operation unit 70 is appropriately assigned a function for each scene and used as one of the various function buttons. Examples of the function button include an end button, a return button, an image feed button, a jump button, a zoom-out button, and an attribute change button. For example, when the menu button 70e is pressed, a menu screen that enables configuration of various settings is displayed on the display unit 28 or the external device 300. The user can intuitively configure various settings using the menu screen displayed on the display unit 28 or the external device 300, as well as the four-way key 74 and the setting button 75.

[0078] The mode switching switch 60 is an operating member for switching between various modes. The mode switching switch 60 switches the operating mode of the system control unit 50 to any one of a still image recording mode, a moving image shooting mode, and a playback mode. The modes included in the still image recording mode are an automatic shooting mode, an automatic scene judgment mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode), and a program AE mode. Other available modes include various scene modes and a custom mode that constitute shooting settings for different shooting scenes. Using the mode switching switch 60, the user can directly switch to any one of these modes. Optionally, after temporarily switching to the shooting mode list screen using the mode switching switch 60, another operating member can be used to selectively switch to any one of a plurality of displayed modes. In the same way, the moving image shooting mode can also include a plurality of modes.

[0079] The shutter button 61 is an operation member for issuing a shooting instruction. The shutter button 61 includes a first shutter switch 62 and a second shutter switch 64. The first shutter switch 62 is turned on halfway by the operation of the shutter button 61 by so-called half-pressing (shooting preparation instruction), and a first shutter switch signal SW1 is generated. According to the first shutter switch signal SW1, the system control unit 50 starts the operation of AF processing, AE processing, AWB processing, and EF processing, etc. The second shutter switch 64 is turned on when the operation of the shutter button 61 is completed by so-called full pressing (shooting instruction), and a second shutter switch signal SW2 is generated. According to the second shutter switch signal SW2, the system control unit 50 starts a series of imaging processing operations from reading a signal from the imaging unit 22 to writing image data to the recording medium 200.

[0080] The main electronic dial 71 is a rotational operation member, and by turning the main electronic dial 71, setting values ​​such as shutter speed and aperture can be changed. The power switch 72 is an operation member for turning on and off the power of the digital camera 100. The sub-electronic dial 73 is a rotational operation member, and by turning the sub-electronic dial 73, operations such as moving a selection frame and feeding an image can be performed. The four-way key 74 is configured so that the upper part, the lower part, the left part, and the right part are respectively depressible. The four-way key 74 enables processing corresponding to the pressed part of the four-way key 74. The setting button 75 is a button mainly used to determine the selected item.

[0081] The LV button 76 is a button for turning on and off the live view (hereinafter, LV) in the still image shooting mode. In the moving image shooting mode, the LV button 76 is used to issue an instruction to start and stop the moving image shooting (recording). The zoom button 77 is an operation button for turning on and off the zoom mode during the live view display in the shooting mode and changing the magnification in the zoom mode. In the playback mode, the zoom button 77 is used as a zoom button for enlarging the playback image or increasing the magnification of the playback image. The zoom-out button 78 is a button for reducing the magnification of the enlarged playback image and reducing the displayed image. The playback button 79 is an operation button for switching between the shooting mode and the playback mode. By pressing the playback button 79 in the shooting mode, it is possible to switch to the playback mode, and the latest image among the images recorded in the recording medium 200 can be displayed on the display unit 28.

[0082] The AF ON button 70 b is a button for instructing execution of AF. The pressing direction of the AF ON button 70 b is parallel to the direction (optical axis) of the subject light incident on the image pickup unit 22 from the lens 103 .

[0083] The quick setting button 70c (hereinafter, the Q button 70c) is a button for displaying a quick setting menu, which is a list of setting items configurable in each operation mode. For example, when pressed while waiting for shooting in live view shooting, a list of setting items such as electronic front curtain shutter, brightness of the monitor, WB of the LV screen, split screen display zoom, and silent shooting is displayed in a single line superimposed on the LV. By selecting any option in the displayed quick setting menu using the four-way key 74 and pressing the setting button 75, the user can change the setting for the selected setting item or make a transition to the operation mode.

[0084] The active frame switching button 70d is a button for switching between active zoom positions (frames) of two zoom positions in the split screen display zoom process. In addition, different functions are assigned according to the operation mode, and when pressed in the playback mode, a protection attribute can be given to the image being displayed.

[0085] The menu button 70 e is a button for causing the display unit 28 or the external device 300 to display a menu screen that enables configuration of various settings.

[0086] The function buttons 70f are buttons that enable different functions to be assigned to the respective buttons. Each function button 70f is arranged at a position where the button can be operated by a finger (middle finger, ring finger, or little finger) of the right hand holding the grip 91, and the pressing direction is parallel to the direction (optical axis) of the subject light incident from the lens 103 to the imaging unit 22.

[0087] The information button 70g is a button for switching between various information displays and the like.

[0088] The touch panel 70a detects contact with respect to the touch panel 70a. The touch panel 70a and the display unit 28 may be integrally constructed. For example, the touch panel 70a is configured so that the transmittance of light does not hinder the display of the display unit 28, and is mounted to the upper layer of the display surface of the display unit 28. Subsequently, the input coordinates on the touch panel 70a and the display coordinates on the display unit 28 are associated with each other. Therefore, a GUI (Graphical User Interface) can be provided that enables the user to feel as if the screen displayed on the display unit 28 can be directly manipulated. The system control unit 50 can detect the following operations for the touch panel 70a or the following states of the touch panel 70a.

[0089] A state in which a finger or pen that was not previously in contact with the touch panel 70a newly touches the touch panel 70a, or in other words, the start of touch (hereinafter referred to as touch-down)

[0090] A state where a finger or a pen is touching the touch panel 70a (hereinafter referred to as touch-on)

[0091] A state in which a finger or a pen is moving while touching the touch panel 70a (hereinafter referred to as touch-move)

[0092] A state in which the finger or pen previously in contact with the touch panel 70a is separated from the touch panel 70a, in other words, the end of the touch (hereinafter referred to as touch-up).

[0093] A state where nothing is touching the touch panel 70a (hereinafter referred to as touch-off)

[0094] When a touch is detected, touch continuation is detected at the same time. Generally, after a touch, unless a touch stop is detected, touch continuation is continuously detected. When a touch movement is detected, touch continuation is also detected at the same time. Even if a touch continuation is detected, a touch movement is not detected unless the touch position moves. When a touch stop is detected for all previously touching fingers or pens, an untouch is detected.

[0095] The above-mentioned operation and state and the position coordinates of the finger or pen touching the touch panel 70a are notified to the system control unit 50 through the internal bus. In addition, based on the notified information, the system control unit 50 determines which operation (touch operation) is performed on the touch panel 70a. Regarding the touch movement, the moving direction of the finger or pen on the touch panel 70a can be determined for each of the vertical component and the horizontal component on the touch panel 70a based on the change in the position coordinates. When a touch movement of at least a specified distance is detected, it is determined that a sliding operation has been performed. An operation involving moving a finger quickly on the touch panel 70a for a certain distance while keeping the finger in contact with the touch panel 70a and then releasing the finger is called a flick. In other words, a flick refers to an operation in which the finger quickly tracks the touch panel 70a as if flicking on the touch panel 70a. When a touch movement of at least a specified distance at at least a specified speed is detected, and then the touch is detected to stop, it can be determined that a flick has been performed (it can be determined that a flick has occurred after the sliding operation). In addition, a touch operation involving touching multiple locations (for example, two points) at the same time and bringing the touch locations closer to each other is called pinch-in, and a touch operation involving moving the touch locations away from each other is called pinch-out. Pinch-out and pinch-in are collectively referred to as a pinch operation (or simply pinch-out). As the touch panel 70a, a touch panel using any of various methods including a resistive film method, a capacitive method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, an image recognition method, and an optical sensor method can be used. Any of a method in which a touch is detected when contact is made with the touch panel and a method in which a touch is detected when a finger or a pen approaches the touch panel can be used.

[0096] In addition, in addition to the camera body, the present invention can also be applied to a control device (display control device) that remotely controls the camera (including a network camera) by communicating with the camera (including a network camera) via wired or wireless communication. Examples of devices that remotely control the camera include smart phones, tablet PCs, and desktop PCs. The camera can be remotely controlled by notifying the camera of commands for performing various operations from the control device and configuring settings based on the operations and processes performed by the control device. In addition, the control device can be configured to receive a live view image captured by the camera via wired or wireless communication and display the live view image.

[0097] In addition, although the example of applying the present invention to a digital camera is described in the above embodiment, the present invention is not limited to this example. Specifically, the present invention can be applied to any device including a display unit such as a PDA, a mobile phone terminal, a mobile image viewer, a printer device equipped with a display, a digital photo frame, a music player, a game device, and an e-book reader.

[0098] Figure 3 This is a diagram showing the digital camera 100 and the external device 300. When the digital camera 100 and the external device 300 are connected via the connection cable 302, the display unit 28 of the digital camera 100 turns off (does not display), and the screen displayed on the display unit 28 is displayed on the display 301 of the external device 300.

[0099] <LV Shooting Mode Processing>

[0100] Figure 4A This is a flowchart showing the LV shooting mode processing performed by the digital camera 100. This processing is implemented when the system control unit 50 expands the program recorded in the non-volatile memory 56 onto the system memory 52 and executes the program.

[0101] In S401, the system control unit 50 determines whether the HDR shooting mode is set. When the HDR shooting mode is set, the system control unit 50 proceeds to S402; otherwise (when the SDR shooting mode is set), the system control unit 50 proceeds to S422.

[0102] In S402, the system control unit 50 determines whether the digital camera 100 is connected to the external device 300. When connected, the system control unit 50 proceeds to S403; otherwise, the system control unit 50 proceeds to S404.

[0103] In S403, the system control unit 50 performs the connection process between the digital camera 100 and the external device 300. Details of the connection process will be described later with reference to Figures 5A to 5C In this embodiment, it is assumed that an HDR connection will be formed when the external device supports HDR connection, and an SDR connection will be formed when the external device does not support HDR connection.

[0104] In S404, the system control unit 50 (image processing unit 24) performs image development on the image captured by the imaging unit 22 and converted into a digital signal by the A / D converter 23 in HDR (with HDR image quality).

[0105] In S405, the system control unit 50 determines whether the display unit 28 for displaying the LV image or the external device 300 supports HDR (supports HDR display). When it supports HDR, the system control unit 50 proceeds to S409; otherwise, the system control unit 50 proceeds to S406.

[0106] In S406, the system control unit 50 determines whether the setting of the HDR auxiliary display is "Auxiliary 1" or "Auxiliary 2". When the setting of the HDR auxiliary display is "Auxiliary 1", the system control unit 50 enters S407, but when the setting of the HDR auxiliary display is "Auxiliary 2", the system control unit 50 enters S408. In this case, the HDR auxiliary display refers to a function that enables the user to confirm the grayscale of the HDR image data by converting the HDR image into the SDR image while giving priority to a specific range of the HDR image when converting the HDR image into the SDR image. In the present embodiment, the setting of the HDR auxiliary display can be switched (changed) between the two settings of "Auxiliary 1" and "Auxiliary 2". "Auxiliary 1" is a high brightness priority setting, while "Auxiliary 2" is a low brightness priority setting. It should be noted that the setting of the HDR auxiliary display can be modified by user operation. In addition, an HDR image refers to an image having a dynamic range wider than a prescribed range. In addition, an SDR image refers to an image having a dynamic range narrower than a prescribed range.

[0107] In S407, the system control unit 50 performs SDR conversion processing (processing of converting from HDR to SDR) on the HDR LV image developed in S404 according to the setting of “Assist 1”, and displays the LV image in SDR (with SDR image quality) on the display unit 28 or the external device 300. It is assumed that the system control unit 50 will also perform resizing processing to a size suitable for the display unit 28 or the external device 300.

[0108] In S408, the system control unit 50 performs SDR conversion processing (processing of converting from HDR to SDR) on the HDR LV image developed in S404 according to the setting of “Assist 2”, and displays the LV image in SDR on the display unit 28 or the external device 300. It is assumed that the system control unit 50 also performs resizing processing to a size suitable for the display unit 28 or the external device 300 in the same manner as S407.

[0109] In S409, the system control unit 50 displays the HDR LV image developed in S404 on the display unit 28 or the external device 300 without modifying the HDR. It is assumed that the system control unit 50 performs resizing processing to a size suitable for the display unit 28 or the external device 300 in the same manner as S407.

[0110] In S410, the system control unit 50 determines whether the menu button 70e is pressed. When the menu button 70e is pressed, the system control unit 50 proceeds to S411, and otherwise, the system control unit 50 proceeds to S412.

[0111] In S411, the system control unit 50 performs shooting menu processing and proceeds to S412. Fig. 6A and 6B Describes the details of shooting menu processing.

[0112] In S412, the system control unit 50 determines whether the information button 70g is pressed. When the information button 70g is pressed, the system control unit 50 proceeds to S413, and otherwise, the system control unit 50 proceeds to S414.

[0113] In S413, the system control unit 50 switches the display of the shooting information. Examples of the shooting information include a histogram and a highlight warning.

[0114] In S414, the system control unit 50 determines whether the shutter button 61 is in the half-pressed state (SW1). When the shutter button 61 is in the half-pressed state, the system control unit 50 proceeds to S415, and otherwise, the system control unit 50 proceeds to S420.

[0115] In S415 , the system control unit 50 performs the AF processing / AE processing described previously.

[0116] In S416, the system control unit 50 determines whether the shutter button 61 is in the fully pressed state (SW2). When the shutter button 61 is in the fully pressed state, the system control unit 50 proceeds to step S418, otherwise, the system control unit 50 proceeds to step S417.

[0117] In S417, the system control unit 50 determines whether the half-pressed (SW1) state of the shutter button 61 is maintained. When maintained, the system control unit 50 proceeds to S415, and otherwise, the system control unit 50 proceeds to S420.

[0118] In S418, the system control unit 50 performs HDR shooting processing. Figures 7A to 7H Describes the details of HDR capture processing.

[0119] In S419, the system control unit 50 performs a quick review display process. Figure 4B Describes the details of the quick review display process.

[0120] In S420, the system control unit 50 determines whether the LV button 76 (moving image button) is pressed. When the LV button 76 is pressed, the system control unit 50 proceeds to S421, and otherwise, the system control unit 50 proceeds to S438.

[0121] In S421 , the system control unit 50 subjects the image developed in HDR in S404 to HEVC compression, and records the image as an HDR moving image file.

[0122] In S422, the system control unit 50 determines whether the digital camera 100 is connected to the external device 300. When connected, the system control unit 50 proceeds to S423, and otherwise, the system control unit 50 proceeds to S424.

[0123] In S423, the system control unit 50 performs connection processing between the digital camera 100 and the external device 300. Figures 5A to 5C Details of the connection process are described. In this case, since the shooting mode is the SDR shooting mode, the digital camera 100 and the external device 300 will be connected through the SDR connection.

[0124] In S424 , the system control unit 50 (image processing unit 24 ) develops the image captured by the imaging unit 22 and converted into a digital signal by the A / D converter 23 in SDR (with SDR image quality).

[0125] In S425, the system control unit 50 displays the SDR LV image developed in S424 on the display unit 28 or the external device 300 without modifying the SDR. It is assumed that the system control unit 50 performs resizing processing to a size suitable for the display unit 28 or the external device 300 in the same manner as S407.

[0126] Since the processing steps of S426 to S433 are the same as the processing steps of S410 to S417 described previously, the description thereof will be omitted.

[0127] In S434, the system control unit 50 performs SDR shooting processing. Figures 7A to 7H The SDR shooting process is performed in the same manner as the HDR shooting process described above. However, in the case of the SDR shooting process, in S710, the system control unit 50 compresses the SDR image that has not undergone HDR development in S707 but has undergone SDR development as the main image.

[0128] In S435, the system control unit 50 performs a quick review display process. Figure 4B Describes the details of the quick review display process.

[0129] In S436, the system control unit 50 determines whether the LV button 76 is pressed. When the LV button 76 is pressed, the system control unit 50 proceeds to S437, and otherwise, the system control unit 50 proceeds to S438.

[0130] In S437 , the system control unit 50 subjects the image developed in SDR in S424 to H264 compression, and records the image as an SDR moving image file.

[0131] In S438, the system control unit 50 determines whether the playback button 79 is pressed. When the playback button 79 is pressed, the system control unit 50 proceeds to S439, and otherwise, the system control unit 50 proceeds to S440.

[0132] In S439, the system control unit 50 performs playback mode processing. Figures 9A to 9H as well as Fig. 10A and 10B Describes the details of playback mode processing.

[0133] In S440, it is determined whether the LV shooting mode process is to be terminated. If the LV shooting mode process is to be terminated, the system control unit 50 terminates the current processing flow, otherwise, the system control unit 50 proceeds to S401.

[0134] 《Quick playback display processing (S419, S435)》

[0135] Figure 4B 4 is a flowchart showing the quick review display process (S419, S435) according to the present embodiment. This process is realized when the system control unit 50 expands the program recorded in the nonvolatile memory 56 on the system memory 52 and executes the program.

[0136] In S451, the system control unit 50 determines whether the quick review display is set to “available”. When the quick review display is set to “available”, the system control unit 50 proceeds to S452, otherwise (when the quick review display is set to “unavailable”), the system control unit 50 ends the current processing flow.

[0137] In S452, the system control unit 50 determines whether shooting is performed in the HDR shooting mode. When shooting is performed in the HDR shooting mode, the system control unit 50 proceeds to S453, otherwise (when shooting is performed in the SDR shooting mode), the system control unit 50 proceeds to S460.

[0138] In S453, the system control unit 50 determines whether the display unit 28 or the external device 300 used for quick review display supports HDR. When HDR is supported, the system control unit 50 proceeds to S457, otherwise, the system control unit 50 proceeds to S454.

[0139] In S454, the system control unit 50 determines whether shooting is performed by RAW still image shooting. In the case of RAW still image shooting, the system control unit 50 proceeds to S455, otherwise (in the case of HEIF still image shooting), the system control unit 50 proceeds to S456.

[0140] In S455, the system control unit 50 converts the HDR image for display in the RAW image file into an SDR image (converts from HDR to SDR) and displays the converted image in SDR on the display unit 28 or the external device 300. The system control unit 50 may perform a resizing process to a size suitable for the display unit 28 or the external device 300.

[0141] In S456, the system control unit 50 converts the HDR image for display in the HEIF image file into an SDR image (converts from HDR to SDR), and displays the converted image in SDR on the display unit 28 or the external device 300. The system control unit 50 may perform resizing processing to a size suitable for the display unit 28 or the external device 300 in the same manner as S455.

[0142] In S457, the system control unit 50 determines whether shooting is performed by RAW still image shooting in the same manner as S454. In the case of RAW still image shooting, the system control unit 50 proceeds to S458, otherwise (in the case of HEIF still image shooting), the system control unit 50 proceeds to S459.

[0143] In S458, the system control unit 50 displays the HDR image for display in the RAW image file in HDR on the display unit 28 or the external device 300. The system control unit 50 may perform resizing processing to a size suitable for the display unit 28 or the external device 300 in the same manner as S455.

[0144] In S459, the system control unit 50 displays the HDR image for display in the HEIF image file in HDR on the display unit 28 or the external device 300. The system control unit 50 may perform resizing processing to a size suitable for the display unit 28 or the external device 300 in the same manner as S455.

[0145] In S460, the system control unit 50 determines whether shooting is performed by RAW still image shooting. In the case of RAW still image shooting, the system control unit 50 proceeds to S461, otherwise (in the case of JPEG still image shooting), the system control unit 50 proceeds to S462.

[0146] In S461, the system control unit 50 displays the SDR image for display in the RAW image file in SDR on the display unit 28 or the external device 300. The system control unit 50 may perform resizing processing to a size suitable for the display unit 28 or the external device 300 in the same manner as S455.

[0147] In S462, the system control unit 50 displays the SDR image for display in the JPEG image file in SDR on the display unit 28 or the external device 300. The system control unit 50 may perform resizing processing to a size suitable for the display unit 28 or the external device 300 in the same manner as S455.

[0148] In S463, the system control unit 50 determines whether the shutter button 61 is pressed (half-pressed (SW1)). When the shutter button 61 is pressed, the system control unit 50 ends the current processing flow, and otherwise, the system control unit 50 proceeds to S464.

[0149] In S464, the system control unit 50 determines whether the time of the quick review display (displayed by the quick review display process) has reached or exceeded at least the prescribed set time. When the prescribed set time or longer has been reached or exceeded, the system control unit 50 ends the current processing flow, otherwise, the system control unit 50 proceeds to S463. It should be noted that the set time can be configured and changed by the user.

[0150] <<Connection processing (S403, S423)>>

[0151] Figure 5A 403 is a sequence diagram showing connection processing (S403, S423) when the digital camera 100 and the external device 300 are connected to each other. The present embodiment will be described on the assumption that the digital camera 100 and the external device 300 are connected to each other.

[0152] In S501, the system control unit 50 controls the digital output I / F 90 and starts to transmit a +5V signal to the external device 300. The transmitted +5V signal is transmitted to the external device 300 through a +5V signal line (not shown) of the connection cable 302. The external device 300 receives the +5V signal of the connection cable 302 and proceeds to S502.

[0153] In S502 , the external apparatus 300 detects that the digital camera 100 is connected to the external apparatus 300 via the connection cable 302 , and proceeds to S503 .

[0154] In S503, the external device 300 starts to transmit an HPD signal (hot plug detection signal) to the digital camera 100 via the HPD signal line (not shown) of the connection cable 302. The digital output I / F 90 of the digital camera 100 receives the transmitted HPD signal via the connection cable 302. Upon receiving the HPD signal, the digital output I / F 90 notifies the system control unit 50 of the reception of the HPD signal, and proceeds to S504.

[0155] In S504 , the system control unit 50 detects the connection of the external device 300 based on the notification of the reception of the HPD signal, and proceeds to S505 .

[0156] In S505, the system control unit 50 controls the digital output I / F 90 and transmits an EDID (Extended Display Identification Data) request signal from the connection cable 302. The transmitted EDID request signal is transmitted to the external device 300 through an EDID signal line (not shown) of the connection cable 302. The external device 300 receives the EDID request signal, and proceeds to S506.

[0157] In S506, the external device 300 transmits EDID from an EDID signal line (not shown) of the connection cable 302. The digital output I / F 90 of the digital camera 100 receives the transmitted EDID via the connection cable 302. Upon receiving the EDID, the digital output I / F 90 notifies the system control unit 50 of the reception of the EDID, and proceeds to S507.

[0158] In S507, the system control unit 50 issues an instruction to the digital output I / F 90 to copy the EDID received in S506 to the memory 32. After the copy is completed, the system control unit 50 analyzes the EDID expanded on the memory, determines the video signal that can be accepted by the external device 300 (the capability of the external device 300), and proceeds to S508.

[0159] In S508, the system control unit 50 determines whether to output an HDR signal or an SDR signal. In the present embodiment, when the setting of the digital camera 100 is to enable HDR and the external device 300 supports HDR, the system control unit 50 outputs an HDR signal to the external device 300, and otherwise, outputs an SDR signal. Examples of the case where the external device 300 supports HDR include the case where an HDR signal is included in the video signal that can be accepted by the external device 300 as determined in S507.

[0160] In S509, the system control unit 50 issues an instruction to start transmitting the signal determined in S508 to the digital output I / F 90. The digital output I / F 90 having received the instruction to start transmitting starts transmitting the video signal (SDR signal or HDR signal) via the connection cable 302, and proceeds to S510.

[0161] In S510, the system control unit 50 outputs the video signal to the external device 300 via a TMDS (Transition Minimized Differential Signaling) signal line (not shown) of the connection cable 302. The external device 300 receives the video signal via the TMDS signal line (not shown) of the connection cable 302, and proceeds to S511.

[0162] In S511 , the external device 300 analyzes the video signal received in S510 , switches the driving of the display 301 to a setting that enables display of the video signal, and proceeds to S512 .

[0163] In S512 , the external device 300 displays the video signal received in S510 on the display 301 of the external device 300 .

[0164] Figure 5B and 5C 1 is a diagram showing a process when the video output is switched from an SDR image to an HDR image (or when the video output is switched from an HDR image to an SDR image) in a state where the digital camera 100 and the external device 300 are connected to each other and a video is displayed on the external device 300.

[0165] Figure 5B 1 is a sequence diagram showing a process for switching the video output from the digital camera 100 to the external device 300 from an SDR image to an HDR image. In this case, it is assumed that the connection between the digital camera 100 and the external device 300 has been completed.

[0166] In S521 , the system control unit 50 issues an instruction to the digital output I / F 90 to transmit an SDR video signal (SDR signal), and proceeds to S522 .

[0167] In S522, the system control unit 50 outputs the SDR video signal to the external device 300 via the TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the SDR video signal via the TMDS signal line (not shown) of the connection cable 302, and proceeds to S523.

[0168] In S523, the external device 300 displays the SDR video signal received in S522 on the display 301 of the external device 300. When the digital camera 100 is outputting the SDR video signal, by repeating the processing steps of S521 to S523, the SDR image is displayed on the display 301 of the external device 300. When the digital camera 100 switches the video output to the external device 300 from the SDR image to the HDR image, the processing steps S524 and thereafter are performed.

[0169] In S524 , the system control unit 50 controls the digital output I / F 90 and issues an instruction to suspend the output of the SDR video signal, and proceeds to S525 .

[0170] In S525, the system control unit 50 issues a notification to the effect that the transmission (output) of the video signal is to be suspended via the TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the notification to the effect that the transmission of the SDR video signal is to be suspended via the TMDS signal line (not shown) of the connection cable 302, and proceeds to S526.

[0171] In S526 , since the reception of the video from the digital camera 100 is suspended, the external device 300 suspends displaying the video on the display 301 of the external device 300 , and proceeds to S527 .

[0172] In S527 , the system control unit 50 issues an instruction to transmit the HDR video signal to the digital output I / F 90 , and proceeds to S528 .

[0173] In S528, the system control unit 50 outputs the HDR video signal via the TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the HDR video signal via the TMDS signal line (not shown) of the connection cable 302, and proceeds to S529.

[0174] In S529 , the external device 300 analyzes the video signal received in S528 , switches the driving of the display 301 to a setting for displaying an HDR video signal, and proceeds to S530 .

[0175] In S530, the external device 300 displays the HDR video signal received in S528 on the display 301 of the external device 300. At this time, since the processing time of S529 to S530 varies according to the performance of the external device 300, it may take about 1 to 5 seconds until the video is displayed.

[0176] Figure 5C 1 is a sequence diagram showing a process for switching the video output from the digital camera 100 to the external device 300 from an HDR image to an SDR image. In this case, it is assumed that the connection between the digital camera 100 and the external device 300 has been completed.

[0177] In S541 , the system control unit 50 issues an instruction to transmit an HDR video signal (HDR signal) to the digital output I / F 90 , and proceeds to S542 .

[0178] In S542, the system control unit 50 outputs the HDR video signal to the external device 300 via the TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the HDR video signal via the TMDS signal line (not shown) of the connection cable 302, and proceeds to S543.

[0179] In S543, the external device 300 displays the HDR video signal received in S542 on the display 301 of the external device 300. When the digital camera 100 is outputting the HDR video signal, by repeating the processing steps S541 to S543, the HDR image is displayed on the display 301 of the external device 300. When the digital camera 100 switches the video output to the external device 300 from the HDR image to the SDR image, the processing steps S544 and thereafter are performed.

[0180] In S544 , the system control unit 50 issues an instruction to suspend the output of the HDR video signal to the digital output I / F 90 , and proceeds to S545 .

[0181] In S545, the system control unit 50 issues a notification to the effect that the transmission (output) of the video signal is to be suspended via the TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the notification to the effect that the transmission of the HDR video signal is to be suspended via the TMDS signal line (not shown) of the connection cable 302, and proceeds to S546.

[0182] In S546 , since the reception of the video from the digital camera 100 is suspended, the external device 300 suspends displaying the video on the display 301 of the external device 300 , and proceeds to S547 .

[0183] In S547 , the system control unit 50 issues an instruction to the digital output I / F 90 to transmit the SDR video signal, and proceeds to S548 .

[0184] In S548, the digital camera 100 outputs the SDR video signal via the TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the SDR video signal via the TMDS signal line (not shown) of the connection cable 302, and proceeds to S549.

[0185] In S549 , the external device 300 analyzes the video signal received in S548 , switches the driving of the display 301 to a setting for displaying an SDR video signal, and proceeds to S550 .

[0186] In S550, the external device 300 displays the SDR video signal received in S548 on the display 301 of the external device 300. At this time, since the processing time of S549 to S550 varies according to the performance of the external device 300, it may take about 1 to 5 seconds until the video is displayed.

[0187] 《Shooting menu processing (S411, S427)》

[0188] Fig. 6A and 6B 2 is a flowchart showing details of the shooting menu process according to the present embodiment. This process is realized when the system control unit 50 develops a program recorded in the nonvolatile memory 56 onto the system memory 52 and executes the program.

[0189] In S601, the system control unit 50 determines whether the HDR shooting mode is set. When the HDR shooting mode is set, the system control unit 50 proceeds to S603, otherwise (when the SDR shooting mode is set), the system control unit 50 proceeds to S602.

[0190] In S602, the system control unit 50 displays a menu to be used during normal SDR shooting. In S603, the system control unit 50 displays a menu by changing functions not used during HDR shooting to a disabled state (grayed out).

[0191] In S604, the system control unit 50 determines whether the user has selected a setting item related to HDR photography. When the user has selected a setting item related to HDR photography, the system control unit 50 proceeds to S605, otherwise, the system control unit 50 proceeds to S610.

[0192] In S605, the system control unit 50 determines whether the user switches the setting related to whether to perform HDR shooting to “enable.” When the user switches the setting to “enable,” the system control unit 50 proceeds to S606, otherwise, the system control unit 50 proceeds to S607.

[0193] In S606 , the system control unit 50 changes the setting regarding whether HDR shooting is to be performed to “enabled”, and records the setting value in the system memory 52 .

[0194] In S607, the system control unit 50 determines whether the user has changed the setting of the HDR auxiliary display. When the user has changed the setting, the system control unit 50 proceeds to S608, otherwise, the system control unit 50 proceeds to S609. When the setting related to whether to perform HDR shooting is "disabled", the setting of the HDR auxiliary display is expected to be unchangeable.

[0195] In S608, the system control unit 50 changes the setting of the HDR assist display during shooting, and records the setting value in the system memory 52. ​​When the setting of the HDR assist display is "enable", at least two setting values ​​(options) are available as variables.

[0196] In S609, the system control unit 50 determines whether the user has issued an end instruction for HDR setting. When an end instruction has not been issued, the system control unit 50 proceeds to S604, and when an end instruction has been issued, the system control unit 50 proceeds to S610.

[0197] In S610, the system control unit 50 determines whether the user has selected a setting item related to the image quality of still image recording. When the user has selected a setting item related to the image quality of still image recording, the system control unit 50 proceeds to S611. Otherwise, the system control unit 50 proceeds to S651 ( Figure 6B ).

[0198] In S611, the system control unit 50 determines whether the HDR shooting mode is set. When the HDR shooting mode is set, the system control unit 50 proceeds to S612, otherwise (when the SDR shooting mode is set), the system control unit 50 proceeds to S614.

[0199] In S612, the system control unit 50 displays a setting screen for HDR shooting. In S613, the system control unit 50 accepts the recording image quality for HDR shooting by user selection. It is assumed that as the recording image quality for HDR shooting, any one of "RAW image", "HDR still image file" and "dual image simultaneous output of RAW+HDR still image file" can be selected by the user. In addition, it is assumed that as the image size, any one of "large" which is close to the number of pixels during sensor reading, "medium" which is slightly smaller, and "small" which is further smaller can be selected. In addition, it is assumed that as the compression rate to reduce (compress) the file size capacity, any one of "high image quality (low compression rate)", "standard (high compression rate)" and "low image quality (high compression rate)" can be selected.

[0200] In S614, the system control unit 50 displays a setting screen for SDR shooting. In S615, the system control unit 50 accepts the recording image quality for SDR shooting by user selection. It is assumed that a plurality of options for setting the recording image quality for SDR shooting and the like are provided in the same manner as described above.

[0201] In S651( Figure 6B), the system control unit 50 determines whether the user has selected a setting item related to the image quality of moving image recording. When the user has selected a setting item related to the image quality of moving image recording, the system control unit 50 proceeds to S652, otherwise, the system control unit 50 proceeds to S657.

[0202] In S652, the system control unit 50 determines whether the HDR shooting mode is set. When the HDR shooting mode is set, the system control unit 50 proceeds to S653, otherwise (when the SDR shooting mode is set), the system control unit 50 proceeds to S655.

[0203] In S653, the system control unit 50 displays a setting screen for HDR shooting. In S654, the system control unit 50 accepts the recording image quality for HDR shooting by user selection. Assume that as the setting for the recording image quality for HDR shooting, the user can select any one of "RAW motion image", "RAW motion image + proxy motion image", "HDR motion image file" and "three-image simultaneous output of RAW + proxy motion image + HDR motion image file". In addition, it is assumed that as the image size, any one of "8K", "4K", "Full HD", "HD" and "VGA" can be selected. In addition, it is assumed that as the compression rate to reduce (compress) the file size capacity, any one of "high image quality (low compression rate)" such as ALL-I, "standard" such as IPB, and "low image quality (high compression rate)" can be selected. In addition, the frame rate and broadcast system (NTSC, PAL, etc.) can be made selectable.

[0204] In S655, the system control unit 50 displays a setting screen for SDR shooting. In S656, the system control unit 50 accepts the recording image quality for SDR shooting by user selection. It is assumed that a plurality of options for setting the recording image quality for SDR shooting are provided in the same manner as HDR shooting.

[0205] In S657, the system control unit 50 determines whether the user has selected the setting item of HDR output. When the user has selected the setting item of HDR output, the system control unit 50 proceeds to S658, otherwise, the system control unit 50 proceeds to S660.

[0206] In S658, the system control unit 50 determines whether the user has switched the HDR output setting to “enabled.” When the user has switched the setting to “enabled,” the system control unit 50 proceeds to S659, otherwise, the system control unit 50 proceeds to S660.

[0207] In S659 , the system control unit 50 changes the HDR output setting to “enabled”, and records the setting value in the system memory.

[0208] In S660, the system control unit 50 determines whether the user has selected a setting item for viewing assistance during playback. When the user has selected a setting item for viewing assistance during playback, the system control unit 50 proceeds to S661, otherwise, the system control unit 50 proceeds to S663.

[0209] In S661, the system control unit 50 determines whether the user has switched the setting of viewing assistance during playback to “enabled.” When the user has switched the setting to “enabled,” the system control unit 50 proceeds to S662, and otherwise, the system control unit 50 proceeds to S663.

[0210] In S662 , the system control unit 50 changes the setting of viewing assist during playback to “enabled” and records the setting value in the system memory.

[0211] In S663, the system control unit 50 determines whether the user has selected a setting item for SDR conversion during transmission. When the user has selected a setting item for SDR conversion during transmission, the system control unit 50 proceeds to S664, otherwise, the system control unit 50 proceeds to S665.

[0212] In S664 , the system control unit 50 switches the setting of SDR conversion during transmission between “enabled” and “disabled”.

[0213] In S665, the system control unit 50 determines whether the user has selected other setting items. When the user has selected other setting items, the system control unit 50 proceeds to S666, otherwise, the system control unit 50 proceeds to S667. In S666, the system control unit 50 performs processing associated with other setting items.

[0214] In S667, the system control unit 50 determines whether an instruction to end the shooting menu process is received from the user. If an instruction to end the shooting menu process is received, the system control unit 50 ends the current processing flow, otherwise, the system control unit 50 proceeds to S601.

[0215] 《HDR shooting processing (S418)》

[0216] Fig. 7A 4 is a flowchart showing details of the HDR shooting process (S418) according to the present embodiment. An example in which the RAW data written to the memory 32 is developed in HDR by the image processing unit 24 will be described.

[0217] An imaging device such as a digital camera or a digital video camera has a white balance function for correcting the color tone of a captured image according to the light source at the time of shooting. The white balance function is a function for correcting the difference in color tone that changes according to the light source (a natural light source such as a sunny or cloudy day, or an artificial light source such as a fluorescent lamp or an incandescent lamp) so that white looks the same regardless of the light source.

[0218] In S701 , the system control unit 50 acquires RAW data via the memory control unit 15 .

[0219] In S702 , the system control unit 50 performs processing for determining white pixels (white search frame in-frame determination processing) based on the acquired RAW data.

[0220] In S703, the system control unit 50 calculates a white balance coefficient required for white balance processing based on the result of S702. Figure 7B The details of the processing of S702 and S703 (the calculation processing of the white balance coefficient) are described with reference to the flowchart shown.

[0221] In S704 to S706, the system control unit 50 calculates the grayscale correction parameters required for the grayscale correction process. Fig.7D The details of the calculation process of the grayscale correction parameters are described with reference to the flowchart shown.

[0222] In S707, the system control unit 50 uses the calculated white balance coefficient, grayscale correction parameters, and various parameters for HDR to perform development. As development parameters, a color matrix, camera OETF curve data, color adjustment parameters, noise reduction parameters, or sharpness parameters, etc. can be used. For example, as a color OETF, although the inverse characteristics of EOTF (Electro-Optical Transfer Function) of PQ (Perceptual Quantization) standardized by ST.2084 are assumed, a camera-side OOTF (Opto-Optical Transfer Function) can be combined. Alternatively, an OETF of HLG (Hybrid Log-Gamma) developed as ARIB STD-B67 can be used.

[0223] In S708 , the system control unit 50 resizes the image developed in S707 , generates an MPF ​​(Multi-Pixel Format) image as an image for comparison between two screens (image for simplified display), and compresses the MPF image.

[0224] In S709 , the system control unit 50 also resizes the MPF image generated in S708 to generate a thumbnail image to be used in index display (index playback) or the like, and compresses the thumbnail image.

[0225] In S710, the system control unit 50 compresses the HDR image developed in S707 as a main image. Although there are many conceivable compression methods, for example, in the case of 10-bit YUV422 data, compression can be performed by H.265 (ISO / IEC23008-2HEVC).

[0226] In S711, the system control unit 50 determines whether RAW recording is to be performed based on the user setting (recording image quality setting). When RAW recording is to be performed, the system control unit proceeds to S712, otherwise, the system control unit proceeds to S713.

[0227] In S712, the system control unit 50 compresses the RAW image and records the compressed RAW image in the recording medium 200 via the recording medium I / F 18. The compressed RAW image is added with a header and recorded as a file. Although there are several conceivable compression methods, lossless compression that is reversible without degradation or lossy compression that is irreversible but enables the file size to be reduced can be used. In addition, as detection metadata, the header records the processing results of S702, the histogram obtained in S704, and the face detection results obtained in S705. In addition, the header also records the development parameters for HDR when the user sets HDR shooting (such as white balance coefficients and grayscale correction amounts, etc.) and the MPF image for display generated in S708.

[0228] Regarding the development parameters, although the development parameters for SDR are to be used in the case of SDR shooting, when the processing speed allows a margin as in the single-shot mode, the development parameters for SDR can also be generated during HDR shooting, and both the development parameters for SDR and the development parameters for HDR can be recorded ( Fig. 8A and 8C ). In addition, when the processing speed allows a margin as in single-shot mode, in addition to the display image for HDR, the main image, MPF image, and thumbnail image of SDR image quality can be created using the development parameters for SDR, and the display image for HDR and the display image for SDR can be recorded in the same file ( Fig.8D ). In addition, when displaying a thumbnail, since it is sufficient to roughly know what the image looks like due to the small size of the image, only the thumbnail image created in S709 may be created and saved as an SDR developed image ( Fig. 8E). By adopting this structure, even products that do not support decoding of H.265, which is an HDR compression method, can display only thumbnail images.

[0229] In S713, the system control unit 50 compresses the developed HDR image and adds static metadata or dynamic metadata, and records the compressed HDR image as a file in the recording medium 200 via the recording medium I / F 18. The static metadata includes the x, y coordinates of the three primary colors and the white point of the display in accordance with CEA-861.3, as well as the maximum brightness value and the minimum brightness of the mastering display, etc. In addition, examples of static metadata include the maximum content light level and the maximum frame-average light level. In addition, dynamic metadata includes metadata for dynamic tone mapping of color quantity conversion as defined in SMPTE ST 2094. Since a minimum depth of 10 bits is required to represent the HDR characteristics with a PQ signal, and since the depth of the conventional JPEG format is 8 bits, a new container for still image HDR must be adopted. Therefore, in this embodiment, a container of HEIF (High Efficiency Image File Format) is used. HEIF is an image file format developed by MPEG (Moving Picture Experts Group) and defined in MPEG-H Part 12 (ISO / IEC 23008-12). HEIF is characterized in that not only the main image but also thumbnails, time-related images, and metadata such as EXIF ​​and XMP can be stored in a single file. Therefore, HEIF is convenient because 10-bit image sequences encoded with HEVC can also be stored.

[0230] <<Calculation Process of White Balance Coefficient (S702, S703)>>

[0231] Figure 7B : is a flowchart showing details of the calculation process (S702, S703) of the white balance coefficient according to the present embodiment.

[0232] In S721, the system control unit 50 performs debayering. For RAW data, each pixel includes only a signal of any one of R, G, and B. Since white search requires conversion to a color signal, debayering is performed. Although there are several debayering methods for generating signals of all channels of R, G, and B, for example, a signal is generated by linear interpolation using a low-pass filter.

[0233] In S722, the system control unit performs processing for subtracting the optical black component from the de-Bayered signal. Generally, for RAW data, due to the influence of noise, the optical black (OB) is not 0 but has a value. Therefore, the OB component must be subtracted from the de-Bayered signal.

[0234] In S723, the system control unit 50 calculates color signals Cx, Cy from the acquired RGB signals using the following equation (1), where Cx represents the color temperature, and Cy corresponds to the green direction correction amount. Yi represents the brightness value.

[0235]

[0236] Figure 7C The CxCy plane is shown. Figure 7C As shown in FIG. 1 , by using an imaging device to capture white at various color temperatures from high color temperature (such as during the day) to low color temperature (such as at sunset) in advance and plotting color evaluation values ​​Cx, Cy on coordinates, a white axis 1200 as a reference for detecting white can be determined. Since white has slight variations in actual light sources, a process of providing widths to both sides of the white axis 1200 is performed (S724). This (white axis having widths) will be referred to as a white search frame 1201.

[0237] In S725 , the system control unit 50 plots each pixel after de-Bayering on the CxCy coordinate system, and determines whether the pixel is within the white search frame 1201 .

[0238] In S726, the system control unit 50 performs a shading removal process for limiting the pixels that are the objects of integration in the brightness direction among the pixels that fall within the white search frame. This process is performed to prevent the calculation accuracy of the white balance coefficient from decreasing due to the fact that too dark colors are susceptible to noise. In the same way, since sensor saturation of any of the channels of too bright colors destroys the balance of the R / G ratio or the B / G ratio and causes separation from the correct color, this process is performed to prevent the calculation accuracy of the white balance coefficient from decreasing. In this case, for example, in SDR, brightness up to +1EV is regarded as an object on the brightness side, and in HDR, by regarding brightness up to +2EV as an object, a white balance coefficient that is more optimized for HDR can be calculated.

[0239] In S727 , the system control unit 50 calculates the integrated values ​​SumR, SumG, and SumB of the respective color evaluation values ​​based on Cx and Cy which are within the white search frame and have been subjected to the shading removal process (white node analysis process).

[0240] In S728, the system control unit 50 calculates white balance coefficients WBCo_R, WBCo_G, and WBCo_B from the calculated integral values ​​using the following equation (2). It should be noted that the values ​​of the white balance coefficients (WB coefficients) may be calculated for the shooting mode (SDR shooting or HDR shooting) set by the user, or may be calculated for both SDR and HDR.

[0241]

[0242] <<Calculation Processing of Grayscale Correction Parameters (S704 to S706)>>

[0243] Fig.7D : is a flowchart showing details of the calculation process (S704 to S706) of the gray scale correction parameters according to the present embodiment.

[0244] In S731, the system control unit 50 uses Fig. 7A The WB process is performed using the WB coefficients generated in the processing steps S701 to S703 shown in the figure.

[0245] In S732, the system control unit 50 performs histogram detection. Specifically, the system control unit 50 applies the gain value of the white balance obtained in S731 to the entire image data. In addition, the system control unit 50 generates a histogram as brightness information based on the pixel value (image) subjected to the gamma correction process. For the gamma correction process, although a method using a known lookup table can be used, it is preferable to use a gamma characteristic that is the same as the gamma characteristic used in the development. However, in order to save processing time and storage amount, a simplified gamma characteristic, such as a gamma characteristic approximated by a broken line, may be used. In general, since the end of an image is generally less important and is affected by a decrease in peripheral light intensity depending on the lens used for shooting, a histogram can be created by excluding pixels in the peripheral edge portion.

[0246] In S733, the system control unit 50 performs pre-processing for face detection. This involves applying reduction processing and gamma processing, etc. to the image data so that the face can be easily detected. Various known methods can be applied as pre-processing for face detection.

[0247] In S734, the system control unit 50 performs face detection processing on the preprocessed image data using a known method. As a result of the face detection processing, the area detected as a face (face area) and the reliability of the detection are obtained.

[0248] In S735, the system control unit 50 calculates a grayscale correction amount (grayscale correction amount (A)) for compensating the exposure correction amount (reduction amount) as a first grayscale correction amount. In this way, the grayscale correction amount of the input / output characteristic is calculated so that while the dark part of the image obtains appropriate exposure, high-brightness pixels of at least a predetermined brightness level are not corrected (at least the exposure correction amount is not fully compensated). Therefore, the bright part is further prevented from becoming a highlight overflow after the grayscale correction. The grayscale correction amount can be prepared in advance as a plurality of correction tables according to the exposure correction amount.

[0249] In S736, the system control unit 50 determines whether a face has been detected. Specifically, the system control unit 50 determines that a face has been detected when there is a face area whose reliability is higher than a preset evaluation threshold value among the face areas detected by the face detection processing in S734. When a face has been detected, the system control unit 50 proceeds to S737, and otherwise, the system control unit 50 proceeds to S741.

[0250] In S737, the system control unit 50 detects a partial area of ​​the detected face area as a face brightness acquisition area. The face brightness acquisition area is an area for acquiring the brightness of the bright part of the face, and the number and position of the face brightness acquisition area are not particularly limited.

[0251] In S738, for each face brightness acquisition area, the system control unit 50 obtains the average value of each type of R pixel, G pixel, and B pixel included in the area (calculates the face brightness). In addition, for each average value of RGB pixels, the gain value of white balance is applied in the same manner as the histogram detection, and after gamma correction is performed, the average value is converted into a brightness value Y using the following formula (3). It should be noted that the gain value of white balance to be applied in the histogram detection and face detection is preferably the same gain value that has been used in the WB processing for the same image data. Although the same brightness gamma used in the development is ideally used in the gamma correction, in order to save processing time and storage amount, a simplified gamma characteristic, such as a gamma characteristic approximated by a broken line, may be used.

[0252] Y=0.299×R+0.587×G+0.114×B···(3)

[0253] In S739, the system control unit 50 converts the brightness values ​​obtained in S738 for each face brightness acquisition area into values ​​assuming a state of proper exposure. The purpose of this processing is to correct the situation where the brightness of the face is detected as lower than when the image data is captured at the proper exposure because the image data is captured at an exposure lower than the proper exposure. When converting the brightness value, the exposure correction amount (reduction amount) determined by the exposure control may be compensated, or the grayscale correction amount calculated in S735 may be used.

[0254] In S740, the system control unit 50 calculates a representative value of the brightness of the detected face. As the representative value, for example, a statistic such as a maximum value can be obtained based on the brightness values ​​of each face brightness acquisition area of ​​the detected face area.

[0255] In S741, the system control unit 50 detects a histogram feature value, which is, for example, a level (SD) to which pixels with a cumulative frequency of 1% from the dark side of the histogram belong, or a level (HL) to which pixels with a cumulative frequency of 1% from the light side of the histogram belong.

[0256] In S742, the system control unit 50 converts the histogram feature value calculated in S741 to a value assumed to be shot with appropriate exposure. The purpose of this processing is to correct the situation where the histogram feature value is detected as lower than when the image data is shot with appropriate exposure because the image data is shot with lower exposure than the appropriate exposure. When converting the brightness value, the exposure correction amount (reduction amount) determined by the exposure control may be compensated, or the grayscale correction amount calculated in S735 may be used.

[0257] In S743, the system control unit 50 calculates the target correction amount. First, the system control unit 50 obtains the representative brightness value of the face or the target brightness level for the histogram feature value. Next, the system control unit 50 creates a lookup table (input / output characteristics) for determining the output brightness level for the input brightness level using spline interpolation or the like based on the target brightness level and the minimum and maximum values ​​of the brightness in the image data. Hereinafter, the lookup table will also be referred to as the grayscale correction amount (B) or the second grayscale correction amount.

[0258] In this case, in HDR, the target grayscale correction amount can be distinguished from SDR. Fig. 7E An example of display in SDR is shown. Figure 7F This shows an example of displaying in HDR. Although the luminance value of the subject is the same value, the background in SDR is at most 100 cd / m 2 , while the background in HDR has more than 100cd / m 2As a result, HDR can give a darker impression even if the brightness value of the subject is the same. This is a phenomenon known as brightness contrast that can be attributed to the visual characteristics of humans. In this case, by applying the same Figure 7G Compared with the grayscale correction amount calculated in the SDR shown in FIG. Figure 7H The correction values ​​shown, which make the dark areas less dark, can produce a result that is preferable in appearance.

[0259] Although the target brightness level for the representative brightness value of the face and the histogram feature value of the image data can be set to a fixed value that is considered to be empirically preferred, a different target brightness level can be set according to the value of the representative brightness value or the histogram feature value. In this case, a lookup table for determining the relationship of the target brightness level with respect to the input level can be prepared for each parameter (representative brightness value or histogram feature value) for which the target brightness level is to be set. A correction characteristic for achieving conversion to the target brightness level determined in this manner is obtained by a method such as spline interpolation, and if necessary, the correction characteristic is saved as a lookup table (or a relational expression) for applying the grayscale correction amount (B).

[0260] In S744, the system control unit 50 synthesizes the gray correction amount (A) calculated in S735 and the gray correction amount (B) calculated in S743 with each other. For example, the system control unit 50 obtains brightness values ​​generated by first applying the gray correction amount (A) to each input brightness level and then applying the gray correction amount (B) to the corrected brightness level, and creates a lookup table of output brightness levels with respect to each input brightness level.

[0261] In S745, the system control unit 50 performs processing (limiter processing) for limiting the upper limit value of the composite correction amount (composite grayscale correction amount) obtained in S744. By synthesizing the grayscale correction amount (A) and the grayscale correction amount (B) with each other, since the correction amount increases and the noise amount becomes more noticeable in the corrected image, a limit is imposed on the overall correction amount. The limiter processing can be implemented by preparing the maximum correction amount allowed for each brightness value as a table, and replacing the output level exceeding the maximum correction amount in the value of the lookup table created in S744 with the output level corresponding to the maximum correction amount. It should be noted that the value of the shooting mode (SDR shooting or HDR shooting) set by the user may be calculated as the grayscale correction amount, or the grayscale correction amounts of both SDR and HDR may be calculated.

[0262] Data Structures

[0263] Fig. 8AThe structure of the still image RAW image data to be recorded in the recording medium 200 in each step of the above-mentioned recording process is shown. The file format exemplified below is the ISO base media file format defined in ISO / IEC 14496-12. Therefore, the present file format has a tree structure and has corresponding nodes called boxes. In addition, each box can have a plurality of boxes as child elements.

[0264] The image data (image file) 801 has, at the beginning, a box ftyp (802) for describing the file type, a box moov (803) including all metadata, a box mdat (808) for the main body of the media data of the track, and other boxes (807). The box moov has, as child elements, a box uuid (804) for storing MetaData (805) and a trak box (806) for storing information referring to ImageData. In MetaData, metadata of the image is described, such as the date and time of creation of the image, the conditions at the time of shooting, information on whether shooting was performed in HDR or SDR, and other information related to shooting. The previously described box mdat has, as a child element, ImageData (809) which is data of the shot still image.

[0265] In this case, among the images recorded in ImageData (809), the RAW image captured in SDR and the RAW image captured in HDR are different.

[0266] Figure 8B The figure shows ImageData (809) recording a RAW image shot in SDR. ImageData (809) includes a THM image (821) developed with SDR image quality and JPEG compressed, an MPF ​​image (822), a main image (823), a RAW image (824), and RAW development parameters (825).

[0267] Figure 8C This is a diagram showing ImageData (809) having only an HDR image as a display image during HDR shooting. ImageData (809) has a THM image (826) developed with HDR image quality and HEVC compressed, an MPF ​​image (827), a main image (828), a RAW image (824), and RAW development parameters (825).

[0268] Fig.8DThis is a diagram showing ImageData (809) having an HDR image and an SDR image as display images during HDR shooting. ImageData (809) has a THM image (821) developed with SDR image quality and JPEG compression, an MPF ​​image (822), a main image (823), and a THM image (826) developed with HDR image quality and HEVC compression. ImageData (809) also has an MPF ​​image (827), a main image (828), a RAW image (824), and RAW development parameters (825).

[0269] Fig. 8E This is a diagram showing ImageData (809) that has only an SDR image as a THM image but has an HDR image as an MPF ​​image and a main image during HDR shooting. ImageData (809) has a THM image (821) developed with SDR image quality and JPEG compression, an MPF ​​image (827) developed with HDR image quality and HEVC compression, a main image (828), a RAW image (824), and RAW development parameters (825).

[0270] The file format shown in this embodiment is only an example, and other boxes may be included as needed. Alternatively, a structure in which a display image is held in a box within moov (803) or in another box (807) may be adopted.

[0271] 《Playback mode processing (SDR) (S439)》

[0272] Fig. 9A 4 is a flowchart showing details of the playback mode process (S439) using the display unit 28. This process is realized when the system control unit 50 develops a program recorded in the nonvolatile memory 56 on the system memory 52 and executes the program.

[0273] In S901, the system control unit 50 determines whether to perform index playback. When index playback is to be performed, the system control unit 50 proceeds to S902, otherwise (when normal playback is to be performed), the system control unit 50 proceeds to S903.

[0274] In S902, the system control unit 50 determines the number of images to be played back (displayed). In S903, the system control unit 50 determines the images to be played back (displayed). In S904, the system control unit 50 performs rendering processing of the images to be played back. Fig. 9B Describes the details of the drawing process.

[0275] In S905, the system control unit 50 determines whether the drawing of all images to be displayed has been completed. When the drawing has been completed, the system control unit 50 proceeds to S906, otherwise, the system control unit 50 proceeds to S903.

[0276] In S906 , the system control unit 50 outputs the image to the display unit 28 , and ends this processing flow.

[0277] The system control unit 50 can perform the operation acceptance processing described below (not shown). First, the system control unit 50 determines whether the user has pressed the menu button 70e. When it is determined that the menu button 70e has been pressed, the menu display processing is performed. Next, the system control unit 50 determines whether the user has pressed the shutter button 61. When it is determined that the shutter button 61 is pressed, the playback mode is ended and the shooting mode processing is performed. In addition, the system control unit 50 determines whether the user has operated the power switch 72 and whether the power has been disconnected. When it is determined that the power has been disconnected, the power disconnection processing is performed. In addition, the system control unit 50 determines whether the user has performed an image switching operation. When it is determined that an image switching operation has been performed, the system control unit 50 enters S903 and performs the playback processing of the next image.

[0278] Fig. 9B It is a flowchart showing the details of the drawing process (S904).

[0279] In S911, the system control unit 50 acquires information about the image to be played back. In S912, the system control unit 50 determines the image to be played back (selects the image to be loaded). Fig. 9C The details of the process for determining the image to be played back are described.

[0280] In S913, the system control unit 50 loads the image to be reproduced from the recording medium 200. In S914, the system control unit 50 performs expansion processing of the image to be reproduced. In S915, the system control unit 50 collects brightness data of each pixel from the image data expanded in S914. The brightness data is, for example, brightness information to be used in histogram processing or highlight warning processing.

[0281] In S916, the system control unit 50 determines whether the image to be replayed (the loaded image) is an HDR image. When the image to be replayed (the loaded image) is an HDR image, the system control unit 50 proceeds to S917, otherwise (when the image to be replayed (the loaded image) is an SDR image), the system control unit 50 proceeds to S920.

[0282] In S917, the system control unit 50 determines whether the setting of HDR auxiliary display during playback is "Assist 1". When set to "Assist 1", the system control unit 50 proceeds to S918, otherwise (when set to "Assist 2"), the system control unit 50 proceeds to S919.

[0283] In S918 , the system control unit 50 performs SDR conversion (conversion from HDR to SDR) on the image expanded in S914 according to the setting of “Assist 1 ”.

[0284] In S919 , the system control unit 50 performs SDR conversion (conversion from HDR to SDR) on the image expanded in S914 according to the setting of “Assist 2 ”.

[0285] In S920, the system control unit 50 performs enlargement and reduction processing on the image to fit the size of the display unit 28. In S921, the system control unit 50 determines the arrangement of the generated image, arranges the image, and ends the drawing process.

[0286] Figures 9C to 9H is a flowchart showing details of the process ( S912 ) for selecting an image to be loaded.

[0287] In S926, the system control unit 50 determines whether the acquired image can be replayed. When the image can be replayed, the system control unit 50 proceeds to S927, otherwise, the system control unit 50 proceeds to S936.

[0288] In S927, the system control unit 50 determines whether the image to be played back is a still image. When the image to be played back is a still image, the system control unit 50 proceeds to S928, and otherwise, the system control unit 50 proceeds to S935.

[0289] In S928, the system control unit 50 determines whether the image to be played back is a RAW image. When the image to be played back is a RAW image, the system control unit 50 proceeds to S929, otherwise the system control unit 50 proceeds to S930.

[0290] In S929, the system control unit 50 determines whether the RAW image to be played back is a RAW image shot in HDR. When the RAW image to be played back is a RAW image shot in HDR, the system control unit 50 proceeds to S931, otherwise (when the RAW image to be played back is a RAW image shot in SDR), the system control unit 50 proceeds to S932. Figures 8A to 8E The metadata described in the RAW file makes the judgment at this time.

[0291] In S930, the system control unit 50 determines whether the still image determined to be not a RAW image is an image shot in HDR. When the still image is shot in HDR, the system control unit 50 enters S933, otherwise (when the still image is shot in SDR), the system control unit 50 enters S934. In this embodiment, since the image shot in HDR is recorded as a HEIF file and the image shot in SDR is recorded as a JPEG file, it is determined whether the image is an HDR image or an SDR image based on whether the image is a HEIF file or a JPEG file. Optionally, metadata within the HEIF file can be used to determine whether the image is an HDR image or an SDR image.

[0292] In S931 , the system control unit 50 selects image data to be used for playback from the file of the RAW image captured in HDR.

[0293] In S932 , the system control unit 50 selects image data to be used for playback from the file of the RAW image captured in SDR.

[0294] In S933 , the system control unit 50 selects image data to be used for playback from the file of the still image developed with HDR.

[0295] In S934, the system control unit 50 selects image data to be used for playback from the files of still images imaged in SDR.

[0296] In S935 , the system control unit 50 selects image data (frames) to be displayed from the moving image file.

[0297] In S936, the system control unit 50 performs non-display processing of the playback image. In this case, information indicating that playback is not possible is displayed on the display unit 28 to notify the user that the image cannot be played back.

[0298] Fig.9D : is a flowchart showing the details of the process (S931) for selecting image data to be used for playback from the file of the RAW image captured in HDR.

[0299] In S941, the system control unit 50 determines whether index playback is to be performed. When index playback is to be performed, the system control unit 50 proceeds to S942, otherwise (when normal playback is to be performed), the system control unit 50 proceeds to S943.

[0300] In S942, the system control unit 50 determines whether the number of indexes to be replayed in the index playback is at least 36. When the number of indexes to be replayed is at least 36, the system control unit 50 proceeds to S945, otherwise, the system control unit 50 proceeds to S944. Although the judgment is made based on whether the number of indexes to be replayed is at least 36 in the present embodiment, the threshold value of the number of indexes to be replayed is only an example. Alternatively, the threshold value of the number of indexes to be replayed may be changed according to the size of the display unit 28.

[0301] In S943 , the system control unit 50 determines “HDR Main Image for Display (HEVC)” ( 828 ) as image data to be played back (image to be loaded).

[0302] In S944 , the system control unit 50 determines “HDR MPF image for display (HEVC)” ( 827 ) as image data to be played back.

[0303] In S945 , the system control unit 50 determines “HDR THM image for display (HEVC)” ( 826 ) as image data to be played back.

[0304] Fig.9E 1 is a flowchart showing another example of a process (S931) for selecting image data to be used for playback from a file of a RAW image captured in HDR. Specifically, the flowchart shows details of a process for selecting image data to be used for playback from a file of a RAW image captured in HDR when the RAW image file has an SDR display image.

[0305] In S951, the system control unit 50 determines whether to perform index playback. When index playback is to be performed, the system control unit 50 proceeds to S952, otherwise (when normal playback is to be performed), the system control unit 50 proceeds to S953.

[0306] In S952, the system control unit 50 determines whether the number of indexes to be played back in the index playback is at least 36. When the number of indexes to be played back is at least 36, the system control unit 50 proceeds to S955, otherwise, the system control unit 50 proceeds to S954.

[0307] In S953 to S955, the system control unit 50 determines whether the RAW image file to be played back contains an SDR image. Figures 8A to 8E Description is carried out by the metadata in the RAW file.

[0308] In S956 , the system control unit 50 determines “HDR Main Image for Display (HEVC)” ( 828 ) as image data to be used for playback.

[0309] In S957 , the system control unit 50 determines the “SDR main image for display (JPEG)” ( 823 ) as image data to be used for playback.

[0310] In S958 , the system control unit 50 determines “HDR MPF image for display (HEVC)” ( 827 ) as image data to be played back.

[0311] In S959, the system control unit 50 determines the "SDR MPF image (JPEG) for display" (822) as image data to be used for playback.

[0312] In S960 , the system control unit 50 determines the “HDR THM image for display (HEVC)” ( 826 ) as image data to be used for playback.

[0313] In S961 , the system control unit 50 determines “SDR THM image (JPEG) for display” ( 821 ) as image data to be used for playback.

[0314] Fig.9F 4 is a flowchart showing details of a process (S933) for selecting image data to be used for playback from a file of a still image developed with HDR.

[0315] In S971, the system control unit 50 determines whether to perform index playback. When index playback is to be performed, the system control unit 50 proceeds to S972, otherwise (when normal playback is to be performed), the system control unit 50 proceeds to S973.

[0316] In S972, the system control unit 50 determines whether the number of indexes to be played back in the index playback is at least 36. When the number of indexes to be played back is at least 36, the system control unit 50 proceeds to S975, and otherwise, the system control unit 50 proceeds to S974.

[0317] In S973 , the system control unit 50 determines “HDR Main Image (HEVC)” (not shown) as image data to be used for playback.

[0318] In S974 , the system control unit 50 determines “HDR MPF image (HEVC)” (not shown) as image data to be used for playback.

[0319] In S975 , the system control unit 50 determines “HDR THM image (HEVC)” (not shown) as image data to be used for playback.

[0320] Figure 9GThe following is a flowchart for selecting image data to be used for playback from a file of a RAW image shot in SDR.

[0321] In S981, the system control unit 50 determines whether index playback is to be performed. When index playback is to be performed, the system control unit 50 proceeds to S982, otherwise (when normal playback is to be performed), the system control unit 50 proceeds to S983.

[0322] In S982, the system control unit 50 determines whether the number of indexes to be played back in the index playback is at least 36. When the number of indexes to be played back is at least 36, the system control unit 50 proceeds to S985, and otherwise, the system control unit 50 proceeds to S984.

[0323] In S983, the system control unit 50 determines the "SDR main image for display (JPEG)" (823) as image data to be used for playback.

[0324] In S984, the system control unit 50 determines the "SDR MPF image (JPEG) for display" (822) as image data to be used for playback.

[0325] In S985 , the system control unit 50 determines the “SDR THM image (JPEG) for display” ( 821 ) as image data to be used for playback.

[0326] Figure 9H 4 is a flowchart showing details of a process (S934) for selecting image data to be used for playback from a file of a still image displayed in SDR.

[0327] In S991, the system control unit 50 determines whether index playback is to be performed. When index playback is to be performed, the system control unit 50 proceeds to S992, otherwise (when normal playback is to be performed), the system control unit 50 proceeds to S993.

[0328] In S992, the system control unit 50 determines whether the number of indexes to be played back in the index playback is at least 36. When the number of indexes to be played back is at least 36, the system control unit 50 proceeds to S995, and otherwise, the system control unit 50 proceeds to S994.

[0329] In S993 , the system control unit 50 determines “SDR main image (JPEG)” (not shown) as image data to be used for playback.

[0330] In S994, the system control unit 50 determines "SDR MPF image (JPEG)" (not shown) as image data to be used for playback.

[0331] In S995 , the system control unit 50 determines “SDR THM image (JPEG)” (not shown) as image data to be used for playback.

[0332] 《Playback mode processing (HDR) (S439)》

[0333] Fig. 10A 4 is a flowchart showing details of the playback mode process (S439) using the external device 300. This process is realized when the system control unit 50 expands a program recorded in the nonvolatile memory 56 on the system memory 52 and executes the program.

[0334] In S1001, the system control unit 50 determines whether the digital camera 100 is connected to the external device 300. When determining that the digital camera 100 is connected to the external device 300, the system control unit 50 proceeds to S1002, and otherwise, proceeds to S1005.

[0335] In S1002, the system control unit 50 determines whether the HDR setting during playback is enabled. Assume that as the HDR setting during playback, "Perform HDR playback", "Do not perform HDR playback", and "Link with shooting mode" can be selected and set. In addition, "Perform HDR playback" is a mode in which HDR output is performed as long as the external device 300 supports HDR regardless of whether the image to be played back is an HDR image or an SDR image. "Do not perform HDR playback" is a mode in which SDR output is performed. "Link with shooting mode" is a mode in which output during playback is linked to the shooting mode, and when "HDR shooting" is set to "Perform", HDR output is also performed during output, but when "HDR shooting" is set to "Do not perform", SDR output is also performed during output. In S1002, when the mode is "Perform HDR playback", the system control unit 50 enters S1003, and when the mode is "Do not perform HDR playback", the system control unit 50 enters S1005. In addition, when the mode is "linked with shooting mode", when the "HDR shooting" set in S606 is "perform", the system control unit 50 enters S1003, and when the "HDR shooting" set in S606 is "not perform", the system control unit 50 enters S1005.

[0336] In S1003, the system control unit 50 determines whether the external device 300 is a display supporting HDR. When the external device 300 supports HDR, the system control unit 50 proceeds to S1004, otherwise, the system control unit 50 proceeds to S1005.

[0337] In S1004 , the system control unit 50 outputs the HDR video (signal) to the external device 300 .

[0338] In S1005 , the system control unit 50 outputs the SDR video to the external device 300 .

[0339] Since S1006 to S1008 and S1010 and S1011 are already referenced Fig. 9A S901 to S903 and S905 and S906 described are the same, so their description will be omitted. Fig. 10B Describe the details of S1009.

[0340] Fig. 10B 3 is a flowchart showing details of the rendering process ( S1009 ) in the playback mode process using the external device 300 .

[0341] Since S1021 to S1025, S1028 and S1029 are similar to those already referenced Fig. 9B The described S911 to S915, S920, and S921 are the same, so the description thereof will be omitted.

[0342] In S1026, the system control unit 50 determines whether the image to be played back is an HDR image. When the image to be played back is an HDR image, the system control unit 50 proceeds to S1028, otherwise (when the image to be played back is an SDR image), the system control unit 50 proceeds to S1027.

[0343] In S1027 , the system control unit 50 performs HDR conversion (conversion from SDR to HDR).

[0344] 《Playback menu processing》

[0345] Fig.11A 2 is a flowchart showing details of the playback menu process according to the present embodiment. This process is realized when the system control unit 50 develops a program recorded in the nonvolatile memory 56 on the system memory 52 and executes the program.

[0346] In S1101, the system control unit 50 refers to the user setting in the setting items (not shown) of RAW development to determine whether to perform RAW development. When RAW development is to be performed, the system control unit 50 proceeds to S1102, but when RAW development is not to be performed, the system control unit 50 proceeds to S1103.

[0347] In S1102, the system control unit 50 performs RAW development processing on the RAW image file specified by the user. For example, the system control unit 50 performs HDR development and saves the developed image file as a HEIF file, or performs SDR development and saves the developed image file as a JPEG file. Fig. 11BDescribes the details of RAW development processing.

[0348] In S1103, the system control unit 50 refers to the setting item (not shown) of the SDR conversion of the HDR file to determine whether to perform SDR conversion (conversion from HDR to SDR). When SDR conversion is to be performed, the system control unit 50 proceeds to S1104, otherwise, the system control unit 50 proceeds to S1105.

[0349] In S1104, the system control unit 50 performs SDR conversion on the HDR image file specified by the user. Since the HDR image is an image generated in a color space where the OETF is PQ and the color gamut is BT.2020 or the like, tone mapping and gamut mapping processing for a color space such as γ2.2 or sRGB of SDR must be performed. Although known techniques can be used as a specific method, for example, tone mapping is performed so that appropriate exposure conforms to SDR, which enables a result with adjusted brightness compared to SDR.

[0350] In S1105, the system control unit 50 refers to the setting items of file transfer (not shown) to determine whether to perform file transfer. When file transfer is to be performed, the system control unit 50 proceeds to S1106, but when file transfer is not to be performed, the system control unit 50 proceeds to S1107.

[0351] In S1106, the system control unit 50 performs a transfer process on the image file specified by the user. When transferring an HDR image file, if the receiving destination is only capable of SDR display, the image file may be transferred as an SDR image file after the SDR conversion described in S1104 is performed in the camera.

[0352] In S1107, the system control unit 50 determines whether to end the playback menu processing. When the playback menu processing is to be ended, the system control unit 50 ends the current processing flow, otherwise, the system control unit 50 proceeds to S1101.

[0353] Fig. 11B1102 is a functional block diagram of each function for performing RAW development processing (S1102). The system control unit 50 loads the captured RAW image 1111 recorded in the recording medium 200 and performs RAW development processing by the image processing unit 24. Each pixel of the RAW image indicates only the intensity about a single color plane. It should be noted that the RAW image includes RAW (SDR) captured in SDR and RAW (HDR) captured in HDR. In addition, there may be a case where RAW (SDR) is developed in SDR as it is and a case where RAW (SDR) is developed in HDR. On the contrary, there is a case where RAW (HDR) is developed in HDR and a case where RAW (HDR) is developed in SDR. The white balance unit 1112 performs processing for making white whiter. When RAW (HDR) is developed in HDR, white balance processing is performed using the white balance coefficient for HDR recorded in the file. In addition, when developing in SDR, white balance processing is performed by generating a white balance coefficient for SDR stored in the file. When white balance coefficients of both HDR and SDR are recorded in a RAW image, a necessary one can be appropriately used.

[0354] The color interpolation unit 1113 performs noise reduction and interpolates the color mosaic image to generate a color image in which all pixels have color information of R, G, and B. A basic color image is generated from the generated color image via a matrix conversion unit 1114 and a gamma conversion unit 1115. Subsequently, a process for adjusting the appearance of the image is performed on the color image by a color brightness adjustment unit 1116. For example, image correction such as detecting a night view and enhancing its chromaticity is performed according to the scene. When grayscale correction is performed simultaneously, when RAW (HDR) is developed with HDR, grayscale correction is performed using the grayscale correction amount for HDR recorded in the file. When developing with SDR, grayscale correction is performed by calculating the grayscale correction amount for SDR using the face detection result and histogram recorded in the file. When grayscale correction amounts of both HDR and SDR are recorded in the RAW image, the required one can be appropriately used.

[0355] For the image to which the desired color adjustment has been made, the compression unit 1117 compresses the high-resolution image by a method such as JPEG or HEVC, and the recording unit 1118 generates a developed image to be recorded in a recording medium such as a flash memory, etc. Since the HEIF container described previously is capable of storing multiple images, an image developed in SDR can be stored together with an image developed in HDR.

[0356] <OSD brightness setting processing>

[0357] Fig. 12A1 is a flowchart showing an example of OSD brightness setting processing according to the present embodiment. This processing is realized when the system control unit 50 expands the program recorded in the nonvolatile memory 56 on the system memory 52 and executes the program. This processing is a processing for adjusting (setting) the brightness of the OSD when performing information display of the OSD on the HDR display.

[0358] In S1201, the system control unit 50 determines whether the display (in this embodiment, the external device 300) as the display destination of the OSD supports HDR display. When the display supports HDR display, the system control unit 50 proceeds to S1202, otherwise, the system control unit 50 proceeds to S1206.

[0359] In S1202, the system control unit 50 determines whether the image to be displayed is a single image. When the image to be displayed is a single image, the system control unit 50 proceeds to S1203, otherwise (in the case of multiple images), the system control unit 50 proceeds to S1204.

[0360] In S1203, the system control unit 50 determines the luminance grayscale value (signal value of white) of the OSD based on the (single) image as the display object. Fig. 12B Details of a process for determining a luminance grayscale value of an OSD based on an image are described.

[0361] In S1204, the system control unit 50 determines the brightness grayscale value of the OSD based on the image(s) to be displayed. Fig. 12B Details of a process for determining a luminance grayscale value of an OSD based on an image are described.

[0362] In S1205, the system control unit 50 obtains the displayable brightness of the display. This is done to determine whether the display is capable of displaying based on the brightness grayscale value of the OSD determined in S1203 or S1204. As a method of obtaining the displayable brightness of the display, for example, EDID information including display brightness information of the display is obtained using DDI (Display Data Channel). It should be noted that, as a prerequisite, both the graphics board and the display must support DDI. When the display brightness information of the display cannot be obtained, the display brightness selected by the user from the expected values ​​of the display brightness prepared in advance can be obtained. Examples of expected values ​​of display brightness of the display include 1000 cd / m2 that complies with the DisplayHDR standard of VESA. 2 、600cd / m 2 and 400cd / m 2 And 1000cd / m2 in line with UltraHD PremiumLOGO 2 and 500cd / m2 .

[0363] In S1206, the system control unit 50 determines the brightness grayscale value of the OSD based on the SDR conversion image (the image after SDR conversion; the SDR image). For example, the maximum brightness grayscale value is calculated among the brightness grayscale values ​​of all pixels constituting the SDR conversion image, and the maximum brightness grayscale value is determined as the brightness grayscale value of the OSD. Since SDR uses a color gamut such as Rec.709 or γ2.2, the brightness grayscale value is a relative value. However, with the popularity of HDR displays in recent years, the peak brightness when displayed in SDR on HDR displays has become higher. In this case, displaying in SDR in the traditional manner may cause the following problem: 255 (8 bits) as the brightness grayscale value of white (white brightness) is equal to 1000 cd / m 2 And it becomes too bright in the same way as HDR. When the peak brightness of the SDR display can be obtained from the EDID information, as the brightness grayscale value of white of the OSD, the code value of the SDR corresponding to the brightness grayscale value determined by the same method as that for HDR (S1202 and subsequent processing) can be set.

[0364] In S1207, the system control unit 50 adjusts the brightness gray value of the OSD. Specifically, the system control unit 50 determines the smaller brightness gray value between the brightness gray value of the OSD determined in S1203 or S1204 and the displayable brightness of the display acquired in S1205 as the final brightness gray value of the OSD.

[0365] In S1208, the system control unit 50 determines the brightness grayscale value of the gray characters in the OSD. The gray characters represent the OSD indicating that the function displayed by the gray characters is disabled. Fig. 12C A process for determining the brightness grayscale value of a gray character is described.

[0366] In S1209 , the system control unit 50 sets the determined luminance grayscale value (luminance grayscale value of OSD and luminance grayscale value of gray characters).

[0367] Fig. 12B : is a flowchart showing details of the process (S1203, S1204) for determining the luminance grayscale value (signal value of white) of the OSD according to the present embodiment.

[0368] In S1211, the system control unit 50 acquires the maximum brightness grayscale value among all pixels constituting the display image. For example, the system control unit 50 analyzes the histogram of the brightness signal of the display image and acquires the signal value of the last bin on the high brightness side (the maximum value of the brightness signal). In this case, since simply selecting the maximum value may result in acquiring a value with an extremely low frequency in the signal value of a bright spot, etc., and may result in a low correlation with the appearance, for example, the average value of the highest (high brightness side) 1% of the signal value of the histogram may be acquired instead of the maximum value. In addition, when there are multiple images as display objects (in the case of S1204), the maximum brightness grayscale value is acquired from each of the multiple images, and the maximum value, minimum value, average value, median value, mode, etc. of the acquired multiple brightness grayscale values ​​are acquired.

[0369] In S1212, when the brightness grayscale value acquired in S1211 is lower than the minimum brightness grayscale value (prescribed value) maintained in advance, the system control unit 50 determines the minimum brightness grayscale value as the brightness grayscale value of the OSD. When the OSD is displayed according to the maximum value acquired in S1211, in the case of a scene composed solely of dark pixels (low-key scene), the OSD becomes darker. Therefore, in the present embodiment, the minimum brightness value is maintained in advance as the brightness of the OSD, and when the maximum value acquired in S1211 is lower than the minimum brightness value, the minimum brightness value is set as the brightness grayscale value of the OSD. As the minimum brightness value, for example, 100 cd / m described as a practical standard in the appendix of ITU-R BT.1886 can be used. 2 .

[0370] In S1213, the system control unit 50 sets the brightness grayscale value determined by the above process as the brightness grayscale value of the OSD. In other words, when the brightness of the OSD (graphic image) is represented by L1 and the brightness of the display image (HDR image) is represented by L2, L1=f(L2) can also be regarded as an increasing function in a broad sense. It should be noted that it is only necessary that the brightness of the OSD increases as the brightness of the display image increases, and the brightness of the OSD can increase linearly in proportion to the brightness of the display image or increase in a step manner (in a step function).

[0371] Fig. 12C : is a flowchart showing details of the process (S1208) for determining the brightness grayscale value of the gray character in the OSD.

[0372] In S1221, the system control unit 50 acquires the luminance grayscale values ​​of white and gray during SDR display. For example, the system control unit 50 acquires 255 (8 bits) as the luminance grayscale value of white during SDR display, and acquires 128 (8 bits) as the luminance grayscale value of gray during SDR display.

[0373] In S1222, the system control unit 50 converts the value acquired in S1221 into an ICtCp value. Specifically, the signal values ​​of white and gray of SDR are converted from XYZ values ​​into ICtCp values ​​using the conversion formula described in the generally disclosed white paper version 7.2 (ICtCp_DolbyWhitePaper.pdf) of Dolby.

[0374] In S1223, the system control unit 50 calculates the difference ΔI (perceptual difference) between the white ICtCp value and the gray ICtCp value. Since the brightness in SDR is relative brightness, the character white is not always 100 cd / m 2 By performing linear scaling in XYZ color space halfway through the conversion formula to ICtCp, it is possible to accommodate the assumption of 200 cd / m in SDR etc. 2 This is the case where the characters are white.

[0375] In S1224 , the system control unit 50 acquires the luminance grayscale value of the OSD determined in S1207 for HDR.

[0376] In S1225, the system control unit 50 converts the value acquired in S1224 into an ICtCp value in the same manner as the SDR.

[0377] In S1226, the system control unit 50 calculates the ICtCp value of the gray character. In HDR, when the OETF of the image is PQ, since the code value and the brightness value have a one-to-one correspondence, the linear scaling described previously is not required. When the OETF of the image is HLG, since the value is relative, the desired scaling is required in the same manner as SDR. In doing so, the assumed system gamma must also be considered. In addition, in order to achieve the perceptual difference between white and gray in SDR in HDR, the ICtCp value obtained by subtracting the perceptual difference ΔI from the ICtCp value of white in HDR can be used as the ICtCp value of gray in HDR.

[0378] In S1227, the system control unit 50 calculates the brightness grayscale value (Y) from the ICtCp value of the gray character in HDR calculated via LMS, XYZ, and RGB as described above.

[0379] In S1228 , the system control unit 50 sets the luminance grayscale value of the gray characters in the OSD during display in HDR.

[0380] Figures 13A to 13Cis a diagram showing a display image according to a conventional method. Conventionally, when a captured image is played back and displayed on a rear LCD display of a camera device or the like, an OSD (graphic image) of shooting information (speed, aperture value, ISO sensitivity, and image name) and status information during playback or the like is superimposed and displayed on the captured image.

[0381] For example, in the case of SDR images, Fig.13A As shown in FIG. 1 , the maximum (upper limit) grayscale value that an SDR image can take and the white color of an OSD are usually set to the same maximum value of 255 in terms of 8-bit code. However, in the case of an HDR image, especially in the case of image data that conforms to the PQ standard, the maximum grayscale value that exists in the HDR image is usually not 1023 in terms of 10-bit code. Therefore, Fig. 13B As shown, a maximum value of 1023 is assigned to the white color of the OSD, which makes the OSD appear brighter than the image and displayed in a dazzling manner (depending on the performance of the display). On the other hand, there is a concern that the image of the background is perceived as relatively dark and difficult to show. Therefore, for example, Fig. 13C As shown, a gray value of 520 is assigned to the white color of the OSD, which eliminates glare. In this case, the value 520 is consistent with the 100 cd / m described as the de facto standard in the appendix of ITU-R BT.1886. 2 However, in the case where the image data of the background is bright, etc., there is a concern that the white color of the OSD may be perceived as relatively dark and difficult to be seen. In addition, although there is a case where the white color of the OSD is made darker so that the white color is perceived as gray, since gray often indicates a disabled function, there is a concern that the user may be misled.

[0382] Fig.13D and 13E is a diagram showing a display image according to this embodiment. Fig.13D As shown in FIG. 1 , since the maximum brightness grayscale value of the background image is the same as the brightness grayscale value of the white characters in the OSD (graphic image), the visibility of the OSD is improved. Fig.13E As shown, even when the maximum brightness of the background image is different in the image to be displayed, by matching the brightness of the white characters of the OSD with the maximum brightness of the background image for each image, improved visibility of the OSD can always be achieved. As described above, according to the first embodiment, even with an HDR display capable of displaying high brightness, the OSD superimposed on the background image can be displayed with good visibility.

[0383] In addition, due to the above processing, even after switching to display in HDR, Fig.14The perceived difference between the gray position (gray character or symbol) (1402) and the white position (1401) of the OSD (displayed in SDR) shown in FIG. 1 is also maintained, and the brightness grayscale value of white is changed. Therefore, the gray brightness can be perceived with the same feeling as in SDR.

[0384] (Second embodiment)

[0385] Next, the second embodiment of the present invention is described. In the first embodiment, the brightness grayscale value of the OSD is determined based on the maximum brightness grayscale value of the background image. The second embodiment is different from the first embodiment in that the brightness grayscale value of the OSD is determined based on the shooting conditions of the background image.

[0386] Specifically, in Fig. 12A In the process of S1203 or S1204 in the image processing, the brightness grayscale value of the OSD is determined according to the input dynamic range setting during shooting and the gamma setting during display. Although 1023 corresponds to 10000cd / m in PQ 2 , but when the input dynamic range is narrow, it is conceivable that as output, values ​​up to 1023 will not be exhausted. The maximum value of the output signal actually used depends on the gamma curve at the time of development, and when the gamma characteristics include high contrast, the maximum value of the output signal also increases. Therefore, for example, as shooting conditions, there are maximum signal values ​​corresponding to four combinations of the narrow D range setting and the wide D range setting in the shooting mode during shooting, and the high contrast gamma setting (development mode) and the low contrast gamma setting (development mode) during development. Fig.15 : is a diagram showing an example of four maximum signal values ​​and four maximum brightness values ​​determined according to shooting conditions. As a brightness grayscale value of white of OSD, a maximum signal value corresponding to the shooting conditions may be set.

[0387] As described above, according to the second embodiment, by determining the brightness gray value of white in the OSD based on the shooting conditions of the image rather than the pixel value of the image, the visibility of the OSD is improved. For example, when a plurality of images are sequentially fed and viewed during image playback, even when the maximum brightness value of the content changes between images, the brightness gray value of white of the OSD remains unchanged, and visibility can be improved by suppressing flickering.

[0388] However, there is still a problem that flicker may occur when a plurality of images with different shooting conditions are sequentially fed and viewed. Fig.16As shown, when the shooting conditions are different between the first image and the second image, the brightness of the OSD may be gradually changed (1601). When the images are fed from the second image to the third image, the brightness is gradually changed in the same manner (1602). However, when the images are fed at a high speed by an operation such as continuously pressing an image feed button (not shown), the brightness does not need to be gradually changed. In this case, for example, by displaying the OSD at a brightness corresponding to the shooting conditions of the image displayed when the pressing of the image feed button is released while maintaining the brightness of the first image, flickering can be reduced (1603).

[0389] (Third Embodiment)

[0390] Next, the third embodiment of the present invention is described. In the first and second embodiments, the brightness grayscale value of the white color of the OSD is set based on the pixel value of the display image or the shooting condition. The third embodiment is different from the first and second embodiments in that when multiple images including HDR images and SDR images are multi-played by index display, etc., the brightness grayscale value of the white color of the OSD is determined based on the multiple images. In other words, in the above-mentioned first embodiment, the processing of S1204 is different.

[0391] Figures 17A to 17C 1 is a diagram showing an example of a multi-play display according to the present embodiment. For example, assuming that among the four images arranged in 2×2, the upper two images have a brightness of 100 cd / m 2 The maximum brightness value of the SDR image, and the lower two images are 648cd / m 2 In this case, the brightness grayscale value of the white color of the OSD is made to be the same as Fig.17A The HDR image shown in FIG. 1 is consistent with the HDR image shown in FIG. 1 , which makes the brightness of the OSD excessively increased relative to the brightness of the SDR image, and the visibility of the OSD is reduced, and at the same time, the visibility of the SDR image is reduced. In addition, the brightness grayscale value of the white color of the OSD is made consistent with the HDR image shown in FIG. Fig. 17B The SDR image shown is consistent, which makes the brightness of the OSD excessively reduced relative to the brightness of the HDR image, and the visibility of the OSD is reduced. In view of this, in the third embodiment, the brightness grayscale value of white of multiple images is adjusted.

[0392] Fig.18 is a flowchart showing an example of an OSD brightness setting process according to the third embodiment.

[0393] In S1801, the system control unit 50 acquires the brightness grayscale values ​​of the white color of a plurality of images. In S1802, the system control unit 50 calculates the median value of the brightness grayscale values ​​of the white color of all acquired images. In S1803, the system control unit 50 sets the calculated median value as the brightness grayscale value of the white color of the OSD. Fig. 17C As shown, the SDR image, HDR image and OSD are displayed in a balanced manner, so visibility can be improved. It should be noted that the method of calculating the brightness grayscale value of white from multiple images (S1802) is not limited to the median value, and a statistical value such as an average value or a mode may be used.

[0394] As described above, according to the third embodiment, when a plurality of images are displayed simultaneously (side by side), the brightness grayscale value of white in the OSD is determined based on the brightness grayscale value of white of all images. Therefore, for example, even when multiple displays such as index display are performed, visibility can be improved for all of the SDR images, HDR images, and OSD.

[0395] (Fourth embodiment)

[0396] Next, a fourth embodiment of the present invention will be described. In the first to third embodiments, one brightness gray value of the white color of the OSD is set for the entire screen. In other words, in the first to third embodiments, one brightness gray value of the white color of the OSD is set based on the entire background. However, the subject has bright areas and dark areas, and the SDR image is generally darker than the HDR image. The fourth embodiment is different from the first to third embodiments in that the optimal brightness gray value of the white color in the OSD is set for each area within the plane of a single image as the background.

[0397] For example, due to Fig.19A The brightness gray value of white in the lower OSD is approximately the same as the brightness of the background image, so there is no problem in the visibility of the lower OSD. However, for the upper OSD whose brightness gray value of white is the same as that of the lower OSD, the upper OSD looks relatively dark because the background image is bright. In this case, it is difficult to preferably set only one brightness gray value of white in the OSD for one image. Therefore, considering the characteristics of the image, it is desirable to set the optimal brightness gray value of white in the OSD for each area in the image plane.

[0398] Fig. 20 is a flowchart showing an example of OSD brightness setting processing according to the fourth embodiment.

[0399] In S2001, the system control unit 50 acquires a brightness distribution image (an image indicating brightness distribution) based on the display image. As the brightness distribution image, the brightness component of the display image can be used. Since the brightness component of the middle to low band (a low frequency band whose spatial frequency is lower than the prescribed frequency) has a lower risk of image quality degradation such as tone jump in the processing described later, processing to reduce the high band (high frequency band) is performed in advance for the brightness signal of the display image. Although there are several conceivable processing steps for reducing (suppressing) the high band, for example, filtering processing of applying LPF (low pass filter) horizontally and vertically can be performed.

[0400] In S2002, using the calculated brightness distribution image, the system control unit 50 obtains the brightness grayscale value of the OSD of each area (corrects the image for information display of the OSD). In the area with high brightness in the brightness distribution image, the brightness grayscale value of the OSD may also increase. In this case, the higher the brightness of the area, the higher the brightness grayscale value of the OSD. Although there are several conceivable methods for implementing S2002, for example, it is conceivable to calculate the brightness distribution image of the display image and the image for information display of the OSD by screen processing. Screen processing is a layer synthesis method, and although known techniques can be used for screen processing, specifically, screen processing can be performed by multiplying a black / white inverted display image with the same black / white inverted image for information display of the OSD and black / white inverting the obtained result again.

[0401] In S2003, the system control unit 50 sets the luminance grayscale value of the white color of the OSD obtained in S2002 to the actual luminance grayscale value (code value). Fig.19B As shown, in the display image, a bright OSD may be superimposed and displayed on a bright region (upper end), and a dark OSD may be superimposed and displayed on a dark region (lower end).

[0402] As described above, according to the fourth embodiment, by setting the luminance grayscale value of white of OSD to be different in the area of ​​the image according to the characteristics of the image, information on OSD can be displayed with good visibility in a stable manner without depending on the subject.

[0403] In addition, the present invention can also be implemented by performing the following processing. Specifically, the present invention can also be implemented by providing software (program) that implements the functions of the above-mentioned embodiments to a system or device via a network or various storage media and causing a computer (or CPU or MPU, etc.) in the system or device to load and execute the program.

[0404] Although preferred embodiments of the present invention have been described above, it should be understood that the present invention is not limited to such embodiments and that various modifications and changes may be made without departing from the spirit and scope of the present invention.

[0405] According to the present invention, it is possible to display an OSD with appropriate brightness.

[0406] Other embodiments

[0407] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0408] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An electronic device, comprising an imaging element and a display unit, the electronic device further comprising: a display controller configured to control to display a graphic image together with the HDR image on the display unit, wherein the display controller controls to display the graphic image at a second brightness when the brightness associated with the HDR image is a first brightness, and to display the graphic image at a fourth brightness higher than the second brightness when the brightness associated with the HDR image is a third brightness higher than the first brightness, and The brightness related to the HDR image is an upper limit brightness corresponding to a dynamic range during image capture set on the electronic device, or an upper limit brightness corresponding to a gamma during image development set on the electronic device.

2. The electronic device according to claim 1, wherein: The display controller gradually changes the brightness of the graphic image in a case where switching between images to be displayed on the display unit is performed.

3. The electronic device according to claim 1, wherein: In a case where the display controller displays a plurality of images side by side on the display unit, the brightness of the graphic image is any value of a maximum value, a minimum value, an average value, a median value, or a mode of upper limit brightness of the respective images.

4. The electronic device according to claim 1, wherein: The display controller determines a brightness of the graphics image for each region within a plane of the HDR image.

5. The electronic device according to claim 1, wherein: The display controller adjusts the brightness of the graphic image based on a user-selected display brightness.

6. The electronic device according to claim 1, wherein: The display controller adjusts the brightness of the graphic image based on the displayable brightness acquired from the display unit.

7. The electronic device according to claim 1, wherein: The display controller adjusts the brightness of the graphic image based on a brightness component of a low frequency band having a spatial frequency lower than a prescribed frequency in the HDR image.

8. The electronic device according to claim 1, wherein: The display controller determines the brightness of gray indicating a disabled state in the graphics image when displayed in HDR based on a difference between an upper limit brightness of the graphics image when displayed in SDR and the brightness of gray.

9. The electronic device according to claim 1, wherein: In a case where the brightness of the graphic image determined based on the brightness related to the HDR image is lower than a prescribed value, the display controller sets the brightness of the graphic image to the prescribed value.

10. The electronic device according to claim 1, wherein: The brightness related to the HDR image is an upper limit brightness corresponding to a dynamic range during imaging set on the electronic device, and is an upper limit brightness corresponding to a gamma during imaging set on the electronic device.

11. A control method of an electronic device, the electronic device comprising an imaging element and a display unit, the control method comprising: A display control step for controlling to display the graphic image together with the HDR image on the display unit, wherein, in the display control step, control is performed to display the graphic image at a second brightness when the brightness associated with the HDR image is a first brightness, and to display the graphic image at a fourth brightness higher than the second brightness when the brightness associated with the HDR image is a third brightness higher than the first brightness, and The brightness related to the HDR image is an upper limit brightness corresponding to a dynamic range during imaging set on the electronic device, or an upper limit brightness corresponding to a gamma during imaging set on the electronic device.

12. A computer-readable medium storing a program, wherein: The program causes a computer to execute the control method according to claim 11.

13. A computer program product comprising a program, wherein: The program causes a computer to execute the control method according to claim 11.

Images