Imaging device, control method and program thereof

Abstract

To record developed image data corresponding to a selected dynamic range while including the image data in a RAW image file even when recording RAW image data in an imaging apparatus.SOLUTION: An imaging apparatus has: imaging means; encoding means that executes encoding processing on image data; and control means that, in recording a RAW image file, controls whether to record first encoded image data obtained by executing encoding processing in a first encoded form on image data with a first dynamic range generated based on RAW image data, as an image for display of the RAW image file, or to record second encoded image data obtained by executing encoding processing in a second encoded form on image data with a second dynamic range generated based on the RAW image data, as an image for display of the RAW image file.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging apparatus, and a control method and program thereof. [Background technology]

[0002] Recent imaging devices, such as digital cameras, are capable of capturing HDR images and recording them on recording media. Here, HDR stands for High Dynamic Range, a technology that generates images with a wider dynamic range than SDR (Standard Dynamic Range). Also, a RAW image refers to a raw image before development.

[0003] When recording HDR video content, the content is recorded together with identification information indicating whether or not the content is HDR video (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-7194 A Summary of the Invention [Problem to be solved by the invention]

[0005] Regular RAW image files can contain JPEG-compressed data of a developed image for display. However, JPEG images do not support HDR image quality, so images for display cannot be saved in HDR image quality. Therefore, even if a RAW image taken in HDR is displayed on an HDR display, in order to view it in HDR image quality, the RAW image must first be developed.

[0006] The present invention has been made in consideration of such problems, and aims to provide a technology that, even when recording RAW image data in an imaging device that supports multiple dynamic ranges, such as SDR and HDR, by recording developed image data corresponding to a selected dynamic range in the RAW image file, making it possible to check an image of the selected dynamic range when playing back the RAW image file. [Means for solving the problem]

[0007] In order to solve this problem, for example, an imaging device according to the present invention has the following configuration. An imaging means; An encoding means for performing an encoding process on image data; a control means for controlling, when recording a RAW image file, whether to record first encoded image data obtained by performing encoding processing in a first encoding format on image data of a first dynamic range generated based on the RAW image data as an image for display of the RAW image file, or to record second encoded image data obtained by performing encoding processing in a second encoding format on image data of a second dynamic range generated based on the RAW image data as an image for display of the RAW image file; has. Effect of the Invention

[0008] According to the present invention, even when recording RAW image data in an imaging device that supports multiple types of dynamic ranges, such as SDR and HDR, by recording developed image data corresponding to a selected dynamic range in the RAW image file, it becomes possible to check an image of the selected dynamic range, for example, when displaying a simplified file list. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is an external view of an imaging / display device according to an embodiment. [Diagram 2]FIG. 1 is a block diagram showing a configuration of an image capturing / display device according to an embodiment. [Diagram 3] Connection configuration diagram with external devices. [Figure 4A] 5 is a flowchart of LV shooting mode processing in an embodiment. [Figure 4B] Quick review flow chart. [Figure 5A] FIG. 4 is a sequence diagram of an HDMI connection process according to the embodiment. [Figure 5B] FIG. 4 is a sequence diagram of an HDMI connection process according to the embodiment. [Figure 5C] FIG. 4 is a sequence diagram of an HDMI connection process according to the embodiment. [Figure 6A] 5 is a flowchart of HDR shooting menu processing in the embodiment. [Figure 6B] 5 is a flowchart of HDR shooting menu processing in the embodiment. [Figure 7] 5 is a flowchart of HDR shooting processing in an embodiment. [Figure 8A] FIG. 4 is a diagram showing the configuration of a RAW file according to the embodiment. [Figure 8B] FIG. 4 is a diagram showing an example of an ImageData region in a RAW file. [Figure 9A] 5 is a flowchart of a playback mode process in the embodiment. [Figure 9B] 5 is a flowchart of a playback mode process in the embodiment. [Figure 9C] 5 is a flowchart of a playback mode process in the embodiment. [Figure 9D] 5 is a flowchart of a playback mode process in the embodiment. [Figure 9E] 5 is a flowchart of a playback mode process in the embodiment. [Figure 9F] 5 is a flowchart of a playback mode process in the embodiment. [Figure 9G] 5 is a flowchart of a playback mode process in the embodiment. [Figure 9H] 5 is a flowchart of a playback mode process in the embodiment. [Figure 10A]5 is a flowchart of an HDMI playback process in the embodiment. [Figure 10B] 5 is a flowchart of an HDMI playback process in the embodiment. [Figure 11] 5A and 5B are a flowchart and a diagram showing a data flow of a playback menu process according to the embodiment. [Figure 12] 4 is a flowchart of a development process according to an embodiment. [Figure 13] A diagram showing the CxCy plane. [Figure 14] 13 is a flowchart of a tone correction parameter generation process. [Figure 15] FIG. [Figure 16] A diagram showing examples of appearance in SDR and HDR. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0011] 1(a) and (b) show external views of a digital camera 100 as an example of a device to which the present embodiment is applied. FIG. 1(a) is a front perspective view of the digital camera 100, and FIG. 1(b) is a rear perspective view of the digital camera 100. In FIGS. 1(a) and 1(b), a display unit 28 is a display unit provided on the rear surface of the camera that displays images and various information. An outside-finder display unit 43 is a display unit provided on the top surface of the camera, and displays various camera settings such as shutter speed and aperture. A shutter button 61 is an operation unit for issuing shooting instructions. A mode change switch 60 is an operation unit for switching between various modes. A terminal cover 40 is a cover for protecting a connector (not shown) such as a connection cable that connects a connection cable with an external device to the digital camera 100. A main electronic dial 71 is a rotating operation member included in the operation unit 70, and a user can change settings such as shutter speed and aperture by turning this main electronic dial 71. A power switch 72 is an operation member for switching the power of the digital camera 100 between ON and OFF. The sub electronic dial 73 is a rotary operation member included in the operation unit 70, and is used for moving the selection frame, forwarding images, and the like. The cross key 74 is included in the operation unit 70, and is a cross key (four-way key) that can be pressed in the up, down, left, and right parts. Operations can be performed according to the part of the cross key 74 that is pressed. The SET button 75 is included in the operation unit 70, and is a push button that is mainly used for deciding a selection item, and the like. The LV button 76 is included in the operation unit 70, and is a button for switching live view (hereinafter, LV) ON and OFF in the still image shooting mode. In the video shooting mode, it is used to instruct the start and stop of video shooting (recording). The enlargement button 77 is included in the operation unit 70, and is an operation button for turning on and off the enlargement mode in the live view display in the shooting mode, and for changing the enlargement ratio during the enlargement mode. In the playback mode, it functions as an enlargement button for enlarging the playback image and increasing the enlargement ratio. The reduction button 78 is included in the operation unit 70, and is a button for reducing the enlargement ratio of the enlarged playback image and reducing the displayed image. The playback button 79 is included in the operation unit 70 and is an operation button for switching between a shooting mode and a playback mode.Pressing the playback button 79 during the shooting mode switches to the playback mode, and the latest image among the images recorded on the recording medium 200 can be displayed on the display unit 28 or the external device 300. The quick return mirror 12 is raised and lowered by an actuator (not shown) in response to an instruction from the system control unit 50. The communication terminal 10 is a communication terminal for the digital camera 100 to communicate with the lens side (detachable). The eyepiece finder 16 is a peer-type finder for checking the focus and composition of the optical image of the subject obtained through the lens unit 150 by observing the focusing screen 13. The lid 202 is a lid for a slot that stores the recording medium 200. The grip unit 90 is a holding unit shaped to be easily held by the right hand when the user holds the digital camera 100.

[0012] FIG. 2 is a block diagram showing an example of the configuration of a digital camera 100 according to this embodiment.

[0013] In FIG. 2, lens unit 150 is a lens unit equipped with an interchangeable photographing lens.

[0014] The lens 103 is usually composed of multiple lenses, but for simplicity, only one lens is shown here. The communication terminal 6 is a communication terminal through which the lens unit 150 communicates with the digital camera 100, and the communication terminal 10 is a communication terminal through which the digital camera 100 communicates with the lens unit 150. The lens unit 150 communicates with the system control unit 50 via the communication terminals 6 and 10, controls the aperture 1 via the aperture drive circuit 2 by the internal lens system control circuit 4, and adjusts the focus by displacing the position of the lens 103 via the AF drive circuit 3.

[0015] The AE sensor 17 measures the brightness of the subject through the lens unit 150. The focus detection unit 11 outputs defocus amount information to the system control unit 50. The system control unit 50 controls the lens unit 150 based on the information and performs phase difference AF.

[0016] The quick return mirror 12 (hereinafter, mirror 12) is instructed by the system control unit 50 during exposure, live view shooting, and video shooting, and is raised and lowered by an actuator (not shown). The mirror 12 is a mirror for switching the light beam incident from the lens 103 between the viewfinder 16 side and the imaging unit 22 side. The mirror 12 is normally arranged to reflect the light beam so as to guide it to the viewfinder 16, but when shooting or live view display is performed, it jumps up and retreats from the light beam so as to guide the light beam to the imaging unit 22 (mirror up). The mirror 12 is also a half mirror whose center portion can transmit part of the light, and transmits part of the light beam so that it is incident on the focus detection unit 11 for focus detection.

[0017] A user (photographer) can check the focus and composition of the optical image of the subject obtained through the lens unit 150 by observing the focusing screen 13 through the pentaprism 14 and the viewfinder 16 .

[0018] The shutter 101 is a focal plane shutter that can freely control the exposure time of the imaging unit 22 under the control of the system control unit 50 .

[0019] The imaging unit 22 is an imaging element composed of a CCD or CMOS element that converts an optical image into an electric signal. Filters of each color component of R, G, and B are arranged two-dimensionally and periodically on the imaging surface of the imaging unit 22. When focusing on adjacent 2×2 filters, the two diagonally opposite filters are arranged with filters of the G component, and the remaining two filters are arranged with filters of the G and B components. These 2×2 filters are arranged on the imaging surface of the imaging unit 22. Such an arrangement is generally called a Bayer arrangement. Therefore, the image represented by the signal (analog signal) output from the imaging unit 22 is also a pixel signal of the Bayer arrangement. The A / D converter 23 converts the analog signal of one pixel output from the imaging unit 22 into, for example, a 10-bit digital signal. Note that the image data at this stage is, as described above, Bayer-arranged image data with one component per pixel and 10 bits per component, and is undeveloped image data. Therefore, the image data at this stage is called RAW image data. The image data of the Bayer array after the defective pixels are repaired may be used as RAW image data. In the embodiment, the A / D converter 23 converts the analog signal into 10-bit digital data, but the number of bits may be any number greater than 8 bits, and is not particularly limited. The more bits there are, the higher the gradation expression becomes possible.

[0020] The image processing unit 24 performs predetermined pixel interpolation, resizing such as reduction, and color conversion processing on the data from the A / D converter 23 or the data from the memory control unit 15. The image processing unit 24 also performs predetermined arithmetic processing using the captured image data, and the system control unit 50 performs exposure control and distance measurement control based on the obtained arithmetic results. This allows TTL (through-the-lens) AF (autofocus) processing, AE (autoexposure) processing, and EF (flash pre-flash) processing to be performed. The image processing unit 24 also performs predetermined arithmetic processing using the captured image data, and TTL AWB (auto white balance) processing based on the obtained arithmetic results. The image processing unit 24 also performs encoding / decoding processing of image data under the control of the system control unit 50. This encoding includes JPEG and HEVC. JPEG is for encoding image data of 8 bits per color component, and HEVC is for encoding image data of more than 8 bits per color component.

[0021] The output data from the A / D converter 23 is written into the memory 32 via the image processing unit 24 and the memory control unit 15, or directly via the memory control unit 15. The memory 32 stores image data obtained by the imaging unit 22 and converted into digital data by the A / D converter 23, and image data to be displayed on the display unit 28 or the external device 300. The memory 32 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio.

[0022] The memory 32 also serves as a memory for displaying images (video memory). The D / A converter 19 converts the display image data stored in the memory 32 into an analog signal and supplies it to the display unit 28. In this way, the display image data written to the memory 32 is displayed by the display unit 28 via the D / A converter 19. The display unit 28 performs display according to the analog signal from the D / A converter 19 on a display device such as an LCD. The digital signal once A / D converted by the A / D converter 23 and stored in the memory 32 is converted to analog in the D / A converter 19 and sequentially transferred to and displayed on the display unit 28, thereby functioning as an electronic viewfinder and enabling through image display (live view display).

[0023] A frame indicating the distance measuring point for which autofocus is currently being performed (AF frame), icons indicating the camera's setting status, etc. are displayed on the in-finder liquid crystal display 41 via an in-finder display drive circuit 42. Various camera setting values ​​such as shutter speed and aperture are displayed on the outside-finder liquid crystal display 43 via an outside-finder display drive circuit 44.

[0024] The digital output I / F 90 supplies the image display data stored in the memory 32 to the external device 300 as a digital signal. For example, the digital output I / F 90 outputs video data in a stream format according to a communication protocol compliant with the HDMI (registered trademark) (High-Definition Multimedia Interface) standard. In this way, the display image data written in the memory 32 is displayed on the external device 300.

[0025] The non-volatile memory 56 is an electrically erasable and recordable memory, and may be, for example, an EEPROM. Constants, programs, and the like for operating the system control unit 50 are stored in the non-volatile memory 56. The programs referred to here are programs for executing various flowcharts described later in this embodiment.

[0026] The system control unit 50 is a control unit having at least one processor, and controls the entire digital camera 100. By executing the programs recorded in the nonvolatile memory 56 described above, each process of this embodiment described below is realized. Reference numeral 52 denotes a system memory, which is implemented by a RAM. Constants and variables for the operation of the system control unit 50, programs read from the nonvolatile memory 56, and the like are loaded into the system memory 52. ​​The system control unit 50 also performs display control by controlling the memory 32, the D / A converter 19, the digital output I / F 90, the display unit 28, and the like.

[0027] The system timer 53 is a timing unit that measures the time used for various controls and the time of a built-in clock.

[0028] The mode changeover switch 60 , the first shutter switch 62 , the second shutter switch 64 , and the operation unit 70 function as operation means for inputting various operational instructions to the system control unit 50 .

[0029] The mode changeover switch 60 changes the operation mode of the system control unit 50 to one of a still image recording mode, a video shooting mode, a playback mode, etc. Modes included in the still image recording mode include an auto shooting mode, an auto scene determination mode, a manual mode, an aperture priority mode (Av mode), and a shutter speed priority mode (Tv mode). In addition, there are various scene modes, which are shooting settings according to shooting scenes, a program AE mode, a custom mode, etc. The mode changeover switch 60 can directly change to one of these modes. Alternatively, after switching to a list screen of shooting modes with the mode changeover switch 60, one of the displayed modes may be selected and switched using other operating members. Similarly, the video shooting mode may also include a plurality of modes.

[0030] The shutter button 61 operated by the user includes a first shutter switch 62 and a second shutter switch 63. The first shutter switch 62 is turned on when the user presses the shutter button 61 halfway (instruction to prepare for shooting) and generates a first shutter switch signal SW1. When the system control unit 50 receives the first shutter switch signal SW1, it starts operations such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (pre-flash) processing. The second shutter switch 64 is turned on when the operation of the shutter button 61 is completed, that is, when the user presses it fully (instruction to shoot), and generates a second shutter switch signal SW2. The system control unit 50 starts a series of operations of shooting processing from reading out a signal from the imaging unit 22 to writing image data to the recording medium 200 by the second shutter switch signal SW2.

[0031] Each operation member of the operation unit 70 is assigned a function for each scene by selecting and operating various function icons displayed on the display unit 28 or the external device 300, and acts as various function buttons. Examples of the function buttons include an end button, a back button, an image forward button, a jump button, a filter button, and an attribute change button. For example, when the menu button 70e is pressed, a menu screen in which various settings can be made is displayed on the display unit 28 or the external device 300. The user can intuitively make various settings using the menu screen displayed on the display unit 28 or the external device 300, the four-way buttons (up, down, left, right) and the SET button.

[0032] In the embodiment, the display unit 28 has an image display function of SDR quality, that is, each of the R, G, and B color components can be displayed in 8 bits (256 gradations). When the external device 300 is connected to the digital camera 100, the external device 300 is set as the output target device for captured images and live images instead of the display unit 28. When the user operates the operation unit 70 to explicitly select either the display unit 28 or the external device 300, the selected one becomes the output target device.

[0033] The operation unit 70 is various operation members as an input unit that accepts operations from the user. The operation unit 70 includes at least the following operation units: a shutter button 61, a main electronic dial 71, a power switch 72, a sub electronic dial 73, a cross key 74, a SET button 75, an LV button 76, a zoom-in button 77, a zoom-out button 78, and a playback button 79. The cross key 74 is a directional button that allows the up, down, right, and left parts of the cross key 74 to be pressed. Although the operation unit is described as an integrated operation unit in this embodiment, the up button, the down button, the right button, and the left button may each be an independent button. Hereinafter, the up or down part is referred to as an up or down key, and the left or right part is referred to as a left or right key. The operation unit 70 also includes the following operation units.

[0034] The AF-ON button 70b is a push button switch included in the operation unit 70, and pressing it can instruct the execution of AF. The direction in which the AF-ON button 70b is pressed is parallel to the direction (optical axis) of the subject light entering the imaging unit 22 from the lens 103.

[0035] The quick setting button 70c (hereinafter, Q button 70c) is a push button switch included in the operation unit 70, and pressing it displays a quick setting menu, which is a list of setting items that can be set in each operation mode. For example, when pressed during shooting standby in live view shooting, a list of setting items such as electronic front curtain shutter, monitor brightness, WB of the LV screen, two-point magnification, silent shooting, etc. is displayed in a row superimposed on the LV. The user can change the setting of the selected setting item or switch to an operation mode by selecting any option in the displayed quick setting menu with the up and down keys and pressing the set button.

[0036] The active frame switching button 70d is a push button switch included in the operation unit 70, and by pressing it in the two-point enlargement process described later, the active enlargement position (frame) can be switched between the two enlarged positions. Also, different functions are assigned depending on the operation mode, and pressing it in the playback mode can add a protection attribute to the displayed image.

[0037] The menu button 70e is a push button switch included in the operation unit 70, and causes a menu screen on the display unit 28 or the external device 300 on which various settings can be made.

[0038] The function buttons 70f are three push button switches included in the operation unit 70, and each of the function buttons 70f is assigned a function. Each of the function buttons 70f is disposed at a position operable by a finger (middle finger, ring finger, or little finger) of the right hand holding the grip unit 90, and the pressing direction is parallel to the direction (optical axis) of the subject light entering the imaging unit 22 from the lens 103.

[0039] The info button 70g is a push button switch included in the operation unit 70, and is used to switch between various types of information displays.

[0040] The power supply control unit 80 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between blocks to which electricity is applied, and detects whether a battery is attached, the type of battery, and the remaining battery power. The power supply control unit 80 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, and supplies the necessary voltage to each unit including the recording medium 200 for the necessary period.

[0041] The power supply unit 30 is made up of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, etc. The recording medium I / F 18 is an interface with a recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a recording medium such as a memory card for recording captured images, and is made up of a semiconductor memory, a magnetic disk, etc.

[0042] The communication unit 54 is connected wirelessly or via a wired cable, and transmits and receives video signals and audio signals. The communication unit 54 can also be connected to a wireless LAN (Local Area Network) or the Internet. The communication unit 54 can transmit images (including through images) captured by the imaging unit 22 and images recorded on the recording medium 200, and can also receive image data and various other information from external devices.

[0043] The attitude detection unit 55 detects the attitude of the digital camera 100 with respect to the direction of gravity. Based on the attitude detected by the attitude detection unit 55, it is possible to determine whether the image captured by the imaging unit 22 was captured with the digital camera 100 held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the attitude detected by the attitude detection unit 55 to the image file of the image captured by the imaging unit 22, or rotate and record the image. The attitude detection unit 55 can be an acceleration sensor, a gyro sensor, or the like.

[0044] The operation unit 70 includes a touch panel 70a capable of detecting contact with the display unit 28. The touch panel 70a and the display unit 28 can be configured as one unit. For example, the touch panel 70a is configured so that the light transmittance does not interfere with the display of the display unit 28, and is attached to the upper layer of the display surface of the display unit 28. Then, input coordinates on the touch panel 70a are associated with display coordinates on the display unit 28. This makes it possible to configure a GUI (Graphical User Interface) as if the user could directly operate the screen displayed on the display unit 28. The system control unit 50 can detect the following operations or states on the touch panel 70a. A finger or pen that has not been touching the touch panel 70a touches the touch panel 70a again, that is, the start of touching (hereinafter referred to as touch-down). The touch panel 70a is in a state of being touched with a finger or a pen (hereinafter, referred to as Touch-On). Touching the touch panel 70a with a finger or a pen and moving it (hereinafter referred to as Touch-Move). The finger or pen that has been touching the touch panel 70a is released, that is, the touch ends (hereinafter, referred to as "touch-up"). A state in which nothing is being touched on the touch panel 70a (hereinafter referred to as Touch-Off).

[0045] When touch-down is detected, touch-on is also detected at the same time. After touch-down, touch-on will usually continue to be detected unless touch-up is detected. Touch-move is also detected while touch-on is detected. Even if touch-on is detected, touch-move will not be detected if the touch position has not moved. After it is detected that all fingers or pens that were touching have touched up, touch-off occurs.

[0046] These operations and states, as well as the position coordinates of the finger or pen touching the touch panel 70a, are notified to the system control unit 50 via the internal bus. The system control unit 50 determines what kind of operation (touch operation) has been performed on the touch panel 70a based on the notified information. For touch moves, the direction of movement of the finger or pen moving on the touch panel 70a can also be determined for each vertical and horizontal component on the touch panel 70a based on the change in position coordinates. If a touch move of a predetermined distance or more is detected, it is determined that a slide operation has been performed. An operation in which a finger is touched on the touch panel, quickly moved a certain distance, and then released is called a flick. In other words, a flick is an operation in which a finger is quickly traced on the touch panel 70a as if flicking it. If a touch move of a predetermined distance or more at a predetermined speed or more is detected and a touch up is detected, it can be determined that a flick has been performed (it can be determined that a flick has occurred following a slide operation). Furthermore, a touch operation in which multiple points (for example, two points) are touched simultaneously and the touch positions are brought closer to each other is called pinch in, and a touch operation in which the touch positions are moved away from each other is called pinch out. Pinch out and pinch in are collectively called pinch operation (or simply pinch). The touch panel 70a may be of any of various touch panel types, such as a resistive film type, a capacitive type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, and an optical sensor type. Depending on the type, there are types that detect a touch by contact with the touch panel, and types that detect a touch by the approach of a finger or a pen to the touch panel, and either type may be used.

[0047] The present invention is also applicable not only to the imaging device itself, but also to a control device that communicates with an imaging device (including a network camera) via wired or wireless communication and remotely controls the imaging device. Examples of devices that remotely control an imaging device include a smartphone, a tablet PC, and a desktop PC. The imaging device can be remotely controlled by notifying the control device of commands that cause the imaging device to perform various operations and settings based on operations or processes performed on the control device. Also, a live view image captured by the imaging device may be received via wired or wireless communication and displayed on the control device.

[0048] Although the above description is based on the example of application to a digital camera, the present invention is not limited to this. For example, the present invention can be applied to any device equipped with a display unit, such as a PDA, a mobile phone terminal, a portable image viewer, a printer equipped with a display, a digital photo frame, a music player, a game machine, or an electronic book reader.

[0049] 3 is a diagram showing an example of a connection between the digital camera 100 and the external device 300. When the digital camera 100 and the external device 300 are connected with a connection cable 302, the display unit 28 of the digital camera 100 is turned off, and the display of the digital camera 100 is switched to be displayed on the display 301 of the external device 300.

[0050] 4A is a flowchart showing the LV shooting mode process in the digital camera 100. This process is realized by the system control unit 50 loading a program recorded in the non-volatile memory 56 into the system memory 52 and executing it.

[0051] First, the HDR and SDR shooting modes in this embodiment will be described. The digital camera 100 of this embodiment can set the HDR or SDR shooting mode by a menu operation by the user. These modes are for setting the user's intention of whether to ultimately obtain image data of HDR quality or image data of SDR quality, and the following processing is performed in various controls depending on which mode is set. Hereinafter, shooting in the HDR and SDR shooting modes may be referred to as "HDR shooting" and "SDR shooting". However, since it is also possible to set recording in RAW format only as described later, an HDR image is not necessarily recorded when shooting in the HDR shooting mode.

[0052] In S401, the system control unit 50 determines whether or not the user's setting on the operation unit 70 is the HDR shooting mode. If the system control unit 50 determines that the HDR shooting mode is set, the process proceeds to S402. If the system control unit 50 determines that the SDR shooting mode is set, the process proceeds to S422.

[0053] In S402, the system control unit 50 determines whether or not the external device 300 is connected to the digital camera 100. If the system control unit 50 determines that the external device 300 is connected, the process proceeds to S403, and if the system control unit 50 determines that the external device 300 is not connected, the process proceeds to S404.

[0054] In S403, the system control unit 50 performs a process for connecting the digital camera 100 and the external device 300. Then, the system control unit 50 advances the process to S404. Details of this connection process will be described later with reference to Fig. 5. If the external device supports HDR connection, an HDR connection is established, and if not, an SDR connection is established.

[0055] In S404, the system control unit 50 performs development processing of HDR image quality using the image processing unit 24 on the live RAW image data obtained by capturing an image with the imaging unit 22 and converting it into a digital signal with the A / D converter 23. Hereinafter, the image obtained by the development processing of HDR image quality will be referred to as an HDR image.

[0056] In the embodiment, HDR image data is data in which one pixel is composed of three components (Luv, YCbCr, etc.), and each component is expressed in 10 bits (1024 gradations) in the embodiment. A gamma curve for HDR images (for example, PQ or HLG of ITU-R recommendation BT.2100) is applied to the HDR image data.

[0057] In S405, the system control unit 50 determines whether the device (display unit 28 or external device 300) displaying the LV image is HDR-compatible, and if it is determined that the device is not HDR-compatible, the process proceeds to S406, and if it is determined that the device is HDR-compatible, the process proceeds to S409.

[0058] In S406, the system control unit 50 checks the HDR assist display setting. If the system control unit 50 determines that Assist 1 is set, the process proceeds to S407, and if Assist 2 is set, the process proceeds to S408. Assist 1 is a setting for checking the high luminance area of ​​the HDR image, and a process is performed to allocate many gradations (code values) to the high luminance range of the HDR image. Assist 2 is a setting for checking the intermediate luminance range of the HDR image, and a process is performed to allocate many gradations to the intermediate luminance area of ​​the HDR image.

[0059] In S407, the system control unit 50 performs HDR to SDR conversion processing on the HDR image data obtained by the development processing in S404 in accordance with the Assist 1 settings, and displays the LV image data of SDR quality obtained by resizing the image data to a size suitable for the output target device (display unit 28 or external device 300), and proceeds to S410.

[0060] In S408, the system control unit 50 performs HDR to SDR conversion processing on the HDR image data obtained by the development processing in S404 in accordance with the Assist 2 settings, and displays the LV image data of SDR quality obtained by resizing the image data to a size suitable for the output target device (display unit 28 or external device 300), and proceeds to processing in S410.

[0061] Here, the SDR image data (SDR image data) in S407 and S408 refers to image data with 8 bits per component. A gamma curve for SDR images (for example, a gamma curve of the sRGB standard) is applied to the SDR image data. Note that the gamma curve of the sRGB standard is generally a curve in which the dark areas are linear and the bright areas are a power of 2.4, but for simplicity, a curve of a power of 2.2 may be used.

[0062] In S409, the system control unit 50 resizes the HDR image data obtained by the development process in S404 to a size suitable for the output target device (the display unit 28 or the external device 300), displays the resized HDR image quality image (hereinafter, HDL_LV image) live, and proceeds to processing in S410.

[0063] In S410, the system control unit 50 determines whether the menu display button 70e has been pressed, and if it is determined that the menu display button 70e has been pressed, the process proceeds to S411, and if it is determined that the menu display button 70e has not been pressed, the process proceeds to S412. In S411, the system control unit 50 performs shooting menu processing, and the process proceeds to S412. Details of this shooting menu processing will be described later with reference to FIG.

[0064] In S412, the system control unit 50 determines whether the info display button 70g has been pressed, and if it is determined that the info display button 70g has been pressed, the process proceeds to S413, and if it is determined that the info display button 70g has not been pressed, the process proceeds to S414. In S413, the system control unit 50 switches the display of shooting information, and proceeds to S414. The shooting information includes a histogram, a highlight warning table, and the like.

[0065] In S414, the system control unit 50 determines whether the shutter button 61 is half-pressed based on whether the signal SW1 has been received, and if it determines that the shutter button 61 is not half-pressed, the process proceeds to S420, and if it determines that the shutter button 61 is half-pressed, the process proceeds to S415.

[0066] In S415, the system control unit 50 performs the AF / AE process described in FIG. 2, and proceeds to S416. In S416, the system control unit 50 determines whether the shutter button 61 is fully pressed based on whether the signal SW2 is received, and if it is determined that the shutter button 61 is not fully pressed, proceeds to S417, and if it is determined that the shutter button 61 is fully pressed, proceeds to S418. In S417, the system control unit 50 determines whether the shutter button 61 is held in a half-pressed state, and if it is held in a half-pressed state, returns to S415, and if it is determined that the half-pressed state is not held, proceeds to S420. In S418, the system control unit 50 performs HDR shooting process, and records an image data file according to a recording format set in advance on the recording medium. FIG. 8A shows the data structure of the file to be recorded. Then, the system control unit 50 proceeds to S419. Details of this HDR shooting process will be described later with reference to FIG. 7. Then, in S419, the system controller 50 performs a quick review display process, and the process proceeds to S420. The quick review display process will be described in detail later with reference to FIG.

[0067] In S420, the system control unit 50 determines whether the LV button 76 has been pressed, and if it is determined that the LV button 76 has been pressed, the process proceeds to S421, and if it is determined that the LV button 76 has not been pressed, the process proceeds to S422.

[0068] In S421, the system control unit 50 compresses the image data (image data with 10 bits per component, 3 components per pixel) developed to HDR image quality in S404 using HEVC (H.265), records it as an HDR video file, and proceeds to S438.

[0069] In S422, the system control unit 50 determines whether or not the external device 300 is connected to the digital camera 100. If the system control unit 50 determines that the external device 300 is connected, the process proceeds to S423, and if the system control unit 50 determines that the external device 300 is not connected, the process proceeds to S424. In S423, the system control unit 50 performs a process of connecting the digital camera 100 and the external device 300, and proceeds to S424. Details of this connection process will be described later with reference to FIG. Note that, since the SDR shooting mode is selected, an SDR connection is established with the external device.

[0070] In S424, the system control unit 50 develops the image captured by the imaging unit 22 and converted into a digital signal by the A / D converter 23 in the image processing unit 24 to SDR image quality (3 components per pixel, 8 bits per component (256 gradations)), and proceeds to S425. Note that hereinafter, an image after development processing to SDR image quality is referred to as an SDR image.

[0071] At S425, the system control unit 50 resizes the SDR image obtained by the development process of S424 to a size suitable for the resolution of the output device (display unit 28 or external device 300), generates a live image of SDR quality (SDR_LV image), and displays the generated SDR_LV image.

[0072] In S426, the system control unit 50 determines whether the menu display button 70e has been pressed, and if it is determined that the menu display button 70e has been pressed, the process proceeds to S427, and if it is determined that the menu display button 70e has not been pressed, the process proceeds to S428. In S427, the system control unit 50 performs shooting menu processing, and the process proceeds to S428. Details of the shooting menu processing in S427 will be described later with reference to FIG.

[0073] In S428, the system control unit 50 determines whether the info display button 70g has been pressed, and if it is determined that the info display button 70g has been pressed, the process proceeds to S429, and if it is determined that the info display button 70g has not been pressed, the process proceeds to S430. In S429, the system control unit 50 switches the display of shooting information, and proceeds to S430. The shooting information includes a histogram, a highlight warning table, and the like.

[0074] In S430, the system controller 50 determines whether the shutter button 61 is half-pressed. If it is determined that the shutter button 61 is not half-pressed, the process proceeds to S436. If it is determined that the shutter button 61 is half-pressed, the process proceeds to S431.

[0075] In S431, the system control unit 50 performs the AF / AE process described in FIG. 2, and proceeds to S432. In S432, the system control unit 50 determines whether the shutter button 61 is fully pressed based on whether the system control unit 50 has received the signal SW2. If the system control unit 50 determines that the shutter button 61 is not fully pressed, the process proceeds to S433, and if the system control unit 50 determines that the shutter button 61 is fully pressed, the process proceeds to S434. In S433, the system control unit 50 determines whether the shutter button 61 is being held in a half-pressed state based on whether the system control unit 50 has received the signal SW1. If the system control unit 50 determines that the half-pressed state is being held, the process returns to S431, and if the system control unit 50 determines that the half-pressed state is not being held, the process proceeds to S436.

[0076] In S434, the system control unit 50 performs SDR shooting processing and proceeds to S435. In this SDR shooting processing, the system control unit 50 develops RAW image data obtained by SDR shooting with SDR image quality, JPEG-encodes the image of SDR image quality to generate JPEG image data, and records it on the recording medium as a JPEG file in JPEG format. If the recording setting is set to record only SDR images as JPEG files, only the JPEG files are recorded. If the recording setting is set to record JPEG files and RAW image files, the JPEG files are recorded, and the data obtained by encoding the RAW image data obtained by SDR shooting and the JPEG image data are recorded on the recording medium as RAW image files in the RAW image file format shown in FIG. 8A. In the RAW image file, ImageData809 in the data structure of FIG. 8A has the format shown in FIG. 8B(a). That is, the images of each size for display are stored in one file by integrating the encoded data obtained by JPEG encoding with 8-bit accuracy. Then, in S435, the system control unit 50 performs quick review display processing and proceeds to S436. The quick review display process will be described in detail later with reference to FIG. 4B.

[0077] In S436, the system control unit 50 determines whether the LV button 76 has been pressed, and if it is determined that the LV button 76 has been pressed, the process proceeds to S437, and if it is determined that the LV button 76 has not been pressed, the process proceeds to S438. In S437, the system control unit 50 compresses the SDR image obtained by the SDR image quality development process in S425 using H264 compression, records it as an SDR video file, and the process proceeds to S438.

[0078] In S438, the system control unit 50 determines whether the play button 79 has been pressed. If the system control unit 50 determines that the play button 79 has been pressed, the process proceeds to S439. If the system control unit 50 determines that the play button 79 has not been pressed, the process proceeds to S440. In S439, the system control unit 50 performs a play mode process, and the process proceeds to S440. Details of this play mode process will be described later with reference to FIGS. 9 and 10.

[0079] In S440, the system control unit 50 determines whether or not an instruction to end the LV mode has been received. If it is determined that an instruction to end the LV mode has not been received, the process returns to S401. If it is determined that an instruction to end the LV mode has been received, the process ends.

[0080] 4B is a flowchart showing the quick review display process of the system control unit 50. This process is realized by the system control unit 50 loading a program recorded in the non-volatile memory 56 into the system memory 52 and executing it.

[0081] In S451, the system control unit 50 determines whether the quick review display is set to ON, and if it is determined that the quick review display is set to ON, the process proceeds to S452, and if it is determined that the quick review display is not set, the process ends.

[0082] In S452, the system control unit 50 determines whether shooting was performed in HDR shooting mode, and if it determines that shooting was performed in HDR shooting mode, the process proceeds to S453, and if it determines that shooting was performed in SDR shooting mode, the process proceeds to S460.

[0083] In S453, the system control unit 50 determines whether the device (display unit 28 or external device 300) displaying the quick review is HDR-compatible, and if it is determined that it is not HDR-compatible, the process proceeds to S454, and if it is determined that it is HDR-compatible, the process proceeds to S457.

[0084] In S454, the system control unit 50 determines whether RAW still image shooting was performed, and if it is determined that RAW still image shooting was performed, the process proceeds to S455, and if it is HIEF still image shooting, the process proceeds to S456.

[0085] In S455, the system control unit 50 performs HDR to SDR conversion on the display HDR image 828 in the HDR RAW image using processing equivalent to S406 to S408, resizes the image to a size suitable for the output target device (the display unit 28 or the external device 300), displays the image in SDR image quality, and proceeds to S463.

[0086] In S456, the system control unit 50 performs HDR to SDR conversion on the display HDR image in the HEIF image using processing equivalent to S406 to S408, resizes the image to a size suitable for the output target device (display unit 28 or external device 300), displays the image in SDR image quality, and proceeds to S463.

[0087] In S457, the system control unit 50 determines whether the image was shot using RAW still image shooting, and if it is determined that the image was shot using RAW still image shooting, the process proceeds to S458, and if it is determined that the image was shot using HIEF still image shooting, the process proceeds to S459. In S458, the system control unit 50 resizes the display HDR image 828 in the HDR RAW image to a size suitable for the output target device (display unit 28 or external device 300), displays it in HDR image quality, and proceeds to S463. In S459, the system control unit 50 resizes the display HDR image in the HEIF image to a size suitable for the output target device (display unit 28 or external device 300), displays it in HDR image quality, and proceeds to S463.

[0088] In S460, the external device 300 determines whether the image was shot using RAW still image shooting, and if it is determined that the image was shot using RAW still image shooting, the process proceeds to S461, and if it is determined that the image was shot using HIEF still image shooting, the process proceeds to S462. In S461, the system control unit 50 resizes the display SDR image 823 in the SDR RAW image to a size suitable for the output target device (display unit 28 or external device 300), displays it in SDR image quality, and proceeds to S463. In S462, the system control unit 50 resizes the display SDR image in the JPEG image to a size suitable for the output target device (display unit 28 or external device 300), displays it in SDR image quality, and proceeds to S463.

[0089] In S463, the system controller 50 determines whether the shutter button 61 has been pressed. If it is determined that the shutter button 61 has not been pressed, the process proceeds to S464. If it is determined that the shutter button 61 has been pressed, the process ends.

[0090] In S464, the system control unit 50 determines whether the time set in the quick review display time has elapsed, and if it determines that the time has not elapsed, returns to S463, and if it determines that the time has elapsed, ends this process.

[0091] 5A is a sequence diagram showing a control procedure for the digital camera 100 and the external device 300 when the digital camera 100 and the external device 300 are connected. Here, the description will be given assuming that the digital camera 100 and the external device 300 are connected via HDMI.

[0092] In S501, the system control unit 50 controls the digital output I / F 90 to instruct it to start transmitting a +5V signal. As a result, the digital output I / F 90 starts transmitting a +5V signal. The transmitted +5V signal is transmitted to the external device 300 via a +5V signal line (not shown) of the connection cable 302. The external device 300 receives the +5V signal of the connection cable 302 and proceeds to S502.

[0093] In S502, the external device 300 determines that the digital camera 100 has confirmed the connection of the external device 300, and the process proceeds to S503.

[0094] In S503, the external device 300 starts transmitting an HPD signal from an HPD signal line (not shown) of the connection cable 302. The digital output I / F 90 of the digital camera 100 receives the transmitted HPD signal via the connection cable 302. Upon receiving the HPD signal, the digital output I / F 90 notifies the system control unit 50 of the HPD reception.

[0095] In S504, the system controller 50 detects a connection response from the external device 300 via the notification from the HPD, and proceeds to S505.

[0096] In S505, the system control unit 50 controls the digital output I / F 90 to transmit an EDID request signal from the connection cable 302. The transmitted EDID request signal is transmitted to the external device 300 through an EDID signal line (not shown) of the connection cable 302. The external device 300 receives this EDID request signal and proceeds to the process at S506.

[0097] In S506, the external device 300 transmits the EDID from an EDID signal line (not shown) of the connection cable 302. The digital output I / F 90 of the digital camera 100 receives this EDID via the connection cable 302. Then, upon receiving the EDID, the digital output I / F 90 notifies the system control unit 50 of the reception of the EDID.

[0098] In S507, upon receiving the notification of EDID reception, the system control unit 50 instructs the digital output I / F 90 to copy the EDID received in S506 to the memory 32. After the copy is completed, the system control unit 50 analyzes the EDID loaded in the memory 32, performs processing to determine the video signal capabilities that the external device 300 can accept, and proceeds to processing in S508.

[0099] In S508, if the main body setting is HDR enabled and the video signal capability that the external device 300 can accept as determined in S507 is compatible with HDR signals, the system control unit 50 determines to output an HDR signal to the external device 300; otherwise, it determines to output an SDR signal and proceeds to S509.

[0100] In S509, the system control unit 50 instructs the digital output I / F 90 to start transmitting the HDR or SDR video signal determined in S508. Upon receiving the video signal transmission start instruction, the digital output I / F 90 starts transmitting the video signal through the connection cable 302 and proceeds to S510.

[0101] In S510, the digital camera 100 outputs a video signal to a TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the video signal via the TMDS signal line (not shown) of the connection cable 302, and the process proceeds to S511.

[0102] In S511, the external device 300 analyzes the video signal received in S508, switches the driving of the display 301 to a setting capable of displaying the video signal, and proceeds to S512. In S512, the external device 300 displays the video signal received in S508 on the display 301 of the external device 300.

[0103] FIG. 5B is a sequence diagram showing the process of switching the video output of the digital camera 100 and the external device 300 from an SDR image to an HDR image.

[0104] In this sequence, it is assumed that the connection between the digital camera 100 and the external device 300 has been completed in the sequence described with reference to FIG. 5A.

[0105] In S521, the system control unit 50 instructs the digital output I / F 90 to transmit an SDR video signal, and the process proceeds to S522.

[0106] In S522, the digital camera 100 outputs an SDR video signal to a TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the SDR video signal via the TMDS signal line (not shown) of the connection cable 302, and the process proceeds to S523.

[0107] In S523, the external device 300 displays the SDR video received in S522 on the display 301 of the external device 300.

[0108] While the digital camera 100 is outputting an SDR signal, S521 to S523 are repeated, whereby an SDR image is displayed on the display 301 of the external device 300.

[0109] When the digital camera 100 switches the video output to the external device 300 from an SDR image to an HDR image, the processes from S524 onwards are executed.

[0110] In S524, the system control unit 50 instructs the digital output I / F 90 to stop the SDR video signal, and the process proceeds to S525.

[0111] In S525, the system controller 50 stops the video signal to the TMDS signal line (not shown) of the connection cable 302. The external device 300 stops receiving the SDR video signal via the TMDS signal line (not shown) of the connection cable 302, and the process proceeds to S526.

[0112] In S526, since the external device 300 has stopped receiving the image from the digital camera 100, the external device 300 stops displaying the image on the display 301 of the external device 300.

[0113] In S527, the system control unit 50 instructs the digital output I / F 90 to transmit an HDR video signal, and proceeds to S528.

[0114] In S528, the system control unit 50 outputs the HDR video signal to a TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the HDR video signal via the TMDS signal line (not shown) of the connection cable 302, and proceeds to the process of S529.

[0115] In S529, the external device 300 analyzes the video signal received in S528, switches the driving of the display 301 to a setting capable of displaying the HDR video signal, and proceeds to S530.

[0116] In S530, the external device 300 displays the HDR video signal received in S528 on the display 301 of the external device 300.

[0117] At this time, the processing time from S529 to S530 differs depending on the performance of the external device 300, and it takes about 1 to 5 seconds until the image is displayed.

[0118] FIG. 5C is a sequence diagram showing a process of switching the video output of the digital camera 100 and the external device 300 from an HDR image to an SDR image.

[0119] In this sequence, it is assumed that the connection between the digital camera 100 and the external device 300 has been completed in the sequence described with reference to FIG. 5A.

[0120] In S541, the system control unit 50 instructs the digital output I / F 90 to transmit an HDR video signal, and the process proceeds to S542. In S542, the system control unit 50 outputs the HDR video signal to a TMDS signal line (not shown) of the connection cable 302. In addition, the external device 300 receives the HDR video signal via the TMDS signal line (not shown) of the connection cable 302, and the process proceeds to S523.

[0121] In S543, the external device 300 displays the HDR video image received in S542 on the display 301 of the external device 300.

[0122] While the digital camera 100 is outputting the HDR signal, the HDR image is displayed on the display 301 of the external device 300 by repeating steps S541 to S543.

[0123] When the digital camera 100 switches the video output to the external device 300 from an HDR image to an SDR image, the processing from S544 onwards is executed.

[0124] In S544, the system control unit 50 instructs the digital output I / F 90 to stop the HDR video signal, and the process proceeds to S545. In S545, the system control unit 50 stops the video signal to the TMDS signal line (not shown) of the connection cable 302. The external device 300 stops receiving the HDR video signal via the TMDS signal line (not shown) of the connection cable 302, and the process proceeds to S546.

[0125] In S546, since the external device 300 has stopped receiving the image from the digital camera 100, the external device 300 stops displaying the image on the display 301 of the external device 300.

[0126] In S547, the system controller 50 instructs the digital output I / F 90 to transmit an SDR video signal, and the process proceeds to S548.

[0127] In S548, system controller 50 outputs an SDR video signal to a TMDS signal line (not shown) of connection cable 302. External device 300 receives the SDR video signal via the TMDS signal line (not shown) of connection cable 302, and proceeds to the process of S549.

[0128] In S549, the external device 300 analyzes the video signal received in S548, switches the driving of the display 301 to a setting capable of displaying the SDR video signal, and proceeds to S530. In S550, the external device 300 displays the SDR video signal received in S528 on the display 301 of the external device 300.

[0129] The processing time from S549 to S550 varies depending on the performance of the external device 300, and takes about 1 to 5 seconds until the video is displayed.

[0130] 6A and 6B are flowcharts showing details of the shooting menu process of S411 and S427 in Fig. 4A. This process is realized by the system control unit 50 loading a program recorded in the non-volatile memory 56 into the system memory 52 and executing it.

[0131] In S601, the system control unit 50 determines whether or not to perform HDR shooting based on whether the HDR shooting mode has been enabled by the user. If the system control unit 50 determines not to perform HDR shooting, the process proceeds to S602, where a menu for normal SDR shooting is displayed. If the system control unit 50 determines to perform HDR shooting, the process proceeds to S602, where a menu for HDR shooting is displayed. In S603, functions that are not used in conjunction with HDR shooting are displayed on the menu in a disabled state, such as grayed out.

[0132] In S604, the system control unit 50 determines whether or not the user has selected a setting item for whether or not to perform HDR shooting. If it is determined that the setting item has been selected, the system control unit 50 advances the process to S605, and if not, the system control unit 50 advances the process to S611. In S605, the system control unit 50 determines whether or not the setting for whether or not to perform HDR shooting has been switched to active by the user. If it is determined that the setting item has been switched to active, the system control unit 50 advances the process to S606, and if not, the system control unit 50 advances the process to S607. In S606, the system control unit 50 changes the setting for whether or not to perform HDR shooting to active, and stores the setting value in the system memory 52.

[0133] In S607, when the setting for whether or not to perform HDR shooting is enabled, the system control unit 50 determines whether or not the HDR assist display setting has been changed by the user. If it is determined that the setting has been changed, the process proceeds to S608, and if not, the process proceeds to S609. Note that, when the setting for whether or not to perform HDR shooting is disabled, it is preferable that the HDR assist display setting cannot be changed.

[0134] In S608, the system control unit 50 changes the setting to whether or not to perform the HDR assist display setting during shooting, and stores the setting value in the system memory 52. ​​There may be two or more variations in the HDR assist display setting "performed."

[0135] In this way, when the HDR shooting settings or HDR assist display settings are changed on the menu screen, the change in the display settings may be reflected in the display at the timing of transition to the live view screen. When the settings are changed on the live view screen instead of the menu screen by using a specific button on the operation unit 70, the change may be reflected in the display at the timing of the change (the timing of pressing the button).

[0136] In S609, the system control unit 50 determines whether or not the user has issued an instruction to end the HDR setting menu display process. If it is determined that an instruction to end the HDR setting menu display process has been issued, the system control unit 50 advances the process to S610.

[0137] In S610, the system control unit 50 determines whether or not the user has selected a setting item for the still image recording quality. If the system control unit 50 determines that such a selection has been made, the process proceeds to S611; otherwise, the process proceeds to S651.

[0138] In S611, the system control unit 50 determines whether or not a user has input an instruction for HDR shooting. If the system control unit 50 determines that an instruction for HDR shooting has been input, the process proceeds to S612. If the system control unit 50 determines that the instruction has not been input, the process proceeds to S614.

[0139] In S612, the system control unit 50 displays a screen for HDR shooting, and in S613, accepts user selection of the recording image quality for HDR shooting. As the set recording image quality for HDR shooting, file formats are prepared for simultaneous output of two images: RAW, HDR still image file, and RAW+HDR still image file. In addition, the image size also includes Large, which is close to the number of pixels when the sensor is read, to a slightly smaller Middle, and an even smaller Small. Furthermore, the compression rate for compressing to reduce the file size capacity includes high image quality (low compression rate), standard (high compression rate), low image quality (high compression rate), and the like.

[0140] In S614, the system control unit 50 displays a screen for SDR shooting, and accepts a user selection of the recording image quality for SDR shooting in S615. The same options as for HDR shooting are prepared for the setting recording image quality for SDR shooting.

[0141] In S651, the system control unit 50 determines whether or not the user has selected a setting item for the movie recording quality. If the system control unit 50 determines that the setting item for the movie recording quality has been selected, the process proceeds to S652; otherwise, the process proceeds to S657.

[0142] In S652, the system control unit 50 determines whether or not the user has set whether or not to perform HDR shooting. If the setting indicates that HDR shooting is to be performed, the system control unit 50 advances the process to S653, and if not, the process to S655.

[0143] In S653, the system control unit 50 displays a screen for HDR shooting, and in S654 accepts user selection of the recording image quality for HDR shooting. As the set recording image quality for HDR shooting, three simultaneous video output formats are prepared: RAW video, RAW video + proxy video, HDR video file, and RAW + proxy video + HDR video file. In addition, there are 8K, 4K, FullHD, HD, VGA, etc. as image sizes. Furthermore, there are various compression rates for reducing file size, such as high image quality (low compression rate) such as ALL-I to standard to low image quality (high compression rate) such as IPB. In addition, there are also frame rate selections and broadcasting system selections such as NTSC / PAL.

[0144] In S655, the system control unit 50 displays a screen for SDR shooting as in S653, and accepts a user selection of the recording image quality for SDR shooting in S656. The same options as for HDR shooting are prepared for the setting recording image quality for SDR shooting.

[0145] In S657, the system control unit 50 determines whether or not the user has selected the setting item for HDR output. If the system control unit 50 determines that the user has selected the setting item for HDR output, the process proceeds to S658. If not, the process proceeds to S660. In S658, the system control unit 50 determines whether or not the HDR output setting has been enabled by the user, and if it is determined that the setting has been enabled, the process proceeds to S659. If not, the process proceeds to S660. In S659, the system control unit 50 changes the HDR output setting to enabled, and stores the setting value in the system memory 52.

[0146] In S660, the system control unit 50 determines whether or not the user has selected a setting item for view assist during playback. If the system control unit 50 determines that a setting item for view assist during playback has been selected, the process proceeds to S661, and if not, the process proceeds to S663. In S661, the system control unit 50 determines whether or not the setting for view assist during playback has been switched to enabled by the user, and if it is determined that it has been switched to enabled, the process proceeds to S662, and if not, the process proceeds to S663. In S662, the system control unit 50 changes the view assist setting during playback to enabled, and stores the setting value in the system memory 52.

[0147] In S663, the system control unit 50 determines whether or not the setting item for SDR conversion during transfer has been selected by the user, and if it is determined that the setting has been selected, the process proceeds to S664, otherwise the process proceeds to S665. In S664, the system control unit 50 determines whether or not the setting for SDR conversion during transfer has been switched to enabled by the user, and if it is determined that the setting has been switched to enabled, in S664, the system control unit 50 changes the setting for SDR conversion during transfer to enabled, and proceeds to S665.

[0148] In S665, the system control unit 50 determines whether or not the user has selected other setting items related to HDR shooting, and if it is determined that they have been selected, the process proceeds to S666, otherwise, the process proceeds to S667. In S666, the system control unit 50 changes the other processes to enabled, and the process proceeds to S667.

[0149] In S667, the system controller 50 determines whether or not the user has instructed to exit the menu. If it is determined that the menu has not been exited, the system controller 50 returns the process to S660, and if it is determined that the menu has been exited, the system controller 50 ends this process.

[0150] 7 is a flowchart showing details of the HDR shooting process by the system control unit 50. This is a flow of HDR development of the RAW data written in the memory 32 by the image processing unit 24.

[0151] Imaging devices such as digital cameras and digital video cameras have a white balance function that corrects the color tone of a captured image according to the light source at the time of shooting. The white balance function corrects the difference in color tone that varies depending on the light source (natural light source such as sunny or cloudy, or artificial light source such as fluorescent light or incandescent light) so that the whiteness appears the same regardless of the light source. In S701 to S703, white balance coefficients required for white balance processing are calculated. In this embodiment, the image is shot with an exposure lower than the exposure at which the brightness of a person, etc. is appropriate, so as to prevent the gradation of high-brightness areas such as the sky from being blown out as much as possible.

[0152] First, in S701, the system control unit 50 acquires RAW image data via the memory control unit 15.

[0153] In S702, the system control unit 50 performs a white search frame determination process to determine white-like pixels in the RAW image data acquired in order to calculate the white balance coefficient.

[0154] In S703, the system control unit 50 calculates white balance coefficients based on the result of the white search frame determination.

[0155] The details of the processes in S702 and S703 will be described with reference to the flowchart in FIG.

[0156] As described above, in RAW image data, one pixel has only one of the R, G, and B component signals. In order to perform a white search, the system control unit 50 performs a de-Bayering process S1201 to convert the data into color signals, and generates signals of all channels R, G, and B for one pixel. There are several methods for de-Bayering, and for example, a signal can be generated by linear interpolation using a low-pass filter. Since RAW image data is generally affected by noise, optical black has a value other than 0. Therefore, the system control unit 50 performs a process (S1202) to subtract the OB portion from the signal after de-Bayering. Then, the system control unit 50 calculates color signals Cx and Cy from the acquired RGB signal using the following formula (S1203).

[0157]

number

[0158] Figure 13 shows the C x -C y As shown in the figure, a white axis 1200 that serves as a reference for detecting white can be determined by photographing white in advance using an imaging device from high color temperatures (e.g., daytime) to low color temperatures (e.g., dusk) and plotting the color evaluation values ​​Cx and Cy on a coordinate system. Since there is some variation in white in an actual light source, the system control unit 50 adds a certain width to both sides of the white axis 1200 (S1204). The frame that adds width to this white axis is called a white search frame 1201.

[0159] In S1205, the system control unit 50 performs C x -C yThe pixel is plotted on a coordinate system and it is judged whether it is within the white search frame. In S1206, the system control unit 50 performs a light and dark exclusion process to limit the pixels to be integrated in the luminance direction among the pixels that are within the white search frame. This is performed to prevent a decrease in the calculation accuracy of the white balance coefficient because a color that is too dark is easily affected by noise. Similarly, a color that is too bright is performed to prevent a decrease in the calculation accuracy of the white balance coefficient because the balance of the R / G or B / G ratio is lost due to sensor saturation of any of the channels, and the color moves away from the correct color. At this time, the luminance of the pixel to be subjected to the light and dark exclusion process is made different between SDR and HDR. In other words, the pixel used to calculate the white balance coefficient described later is made different between SDR and HDR. This is because HDR has a higher reproducibility in the high luminance range than SDR. In this embodiment, while SDR targets brightness up to +1EV on the brightness side, HDR targets brightness up to +2EV, making it possible to calculate a white balance coefficient that is more optimized for HDR.

[0160] In step S1207, the system control unit 50 selects the C x , C y Then, in step S1208, the system control unit 50 calculates the integral values ​​SumR, SumG, and SumB of the respective color evaluation values ​​from the calculated integral values ​​using the following equation: R , W.B.C.o. G , W.B.C.o. B Calculate.

[0161]

number

[0162] The white balance coefficient may be calculated for the shooting mode (SDR shooting or HDR shooting) set by the user, or may be calculated for both SDR and HDR.

[0163] Returning to the explanation of Fig. 7, in steps S704 to S706, the system control unit 50 calculates a tone correction table necessary for the tone correction process. Details of the tone correction will be explained using the flowchart of Fig. 14.

[0164] In S1221, the system control unit 50 performs WB processing using the WB coefficients generated in the processing of S701 to S703 in FIG. 7. In S1222, the system control unit 50 performs histogram detection. Specifically, the white balance gain value obtained in S1221 is applied to the entire image data, and a histogram is created as luminance information for pixel values ​​that have been subjected to gamma correction processing. The gamma correction processing may be performed using a known lookup table, but it is preferable to use the same gamma characteristics as those used in development. However, in order to save processing time and memory capacity, simplified gamma characteristics such as gamma characteristics approximated by broken lines may be used. Note that the edge parts of an image are generally not important, and are also affected by a decrease in peripheral light depending on the imaging lens, so a histogram may be created excluding pixels in the peripheral parts.

[0165] At S1223, the system control unit 50 performs face detection pre-processing. This is processing for making it easier to detect a face by performing reduction processing, gamma processing, etc. on the image data. At S1224, the system control unit 50 executes face detection processing on the pre-processed image data using a known method. This face detection processing obtains the position and size of an area that is thought to be a face (face area) and the reliability of detection.

[0166] At S1225, the system control unit 50 calculates a gradation correction amount (gradation correction amount (A)) for compensating for the exposure correction amount (reduction amount) as a first gradation correction amount. At this time, a gradation correction amount of input / output characteristics is calculated so that dark parts of the image are properly exposed, while high luminance pixels above a predetermined luminance level are not corrected (at least, the exposure correction amount is not completely compensated for). This makes it possible to further suppress blown-out highlights in bright parts after gradation correction. This gradation correction amount can be prepared in advance as a plurality of correction tables corresponding to the exposure correction amount. In S1226, the system control unit 50 determines that a face has been detected if any face area detected by the face detection process in S1224 has a reliability higher than a preset evaluation threshold value. If the system control unit 50 determines that a face has been detected, the process proceeds to S1227, and if the system control unit 50 determines that a face has not been detected, the process proceeds to S1231.

[0167] In S1227, the system control unit 50 calculates a part of the detected face region as a face luminance acquisition region. The face luminance acquisition region is a region for acquiring the luminance of a bright part of the face, and there is no particular restriction on the number or position thereof. In S1228, the system control unit 50 calculates the average value for each type of R pixel, G pixel, and B pixel included in each face luminance acquisition region. Furthermore, the system control unit 50 applies a white balance gain value to each average value of the RGB pixels in the same manner as in histogram detection, performs gamma correction, and then converts it into a luminance value Y by the following formula. Y=0.299×R+0.587×G+0.114×B It is preferable to use the white balance gain value used in WB processing for the same image data as the gain value applied in histogram detection and face detection. Ideally, the luminance gamma should be the same as that used in development, but simplified gamma characteristics, such as gamma characteristics approximated by broken lines, may be used to save processing time and memory.

[0168] In S1229, the system control unit 50 converts the luminance value obtained for each face luminance acquisition area in S1228 into a value assuming a proper exposure. This is a process for correcting the fact that the luminance of the face is detected as lower than that in the case of imaging with a proper exposure, because the image data is imaged with an exposure lower than the proper exposure. The conversion of the luminance value may be performed so as to compensate for the exposure correction amount (reduction amount) determined by the exposure control, or may be performed using the gradation correction amount calculated in S1225.

[0169] In S1230, the system control unit 50 calculates a representative value of the luminance of the detected face. For example, the representative value may be calculated by finding the maximum value among the luminance values ​​of the face luminance acquisition area of ​​the detected face area. The process of S1231 is a process when the system control unit 50 determines in S1226 that a face region has not been detected. In S1231, the system control unit 50 detects a histogram feature amount. The histogram feature amount may be, for example, a level (SD) to which pixels having a cumulative frequency of 1% from the dark side in the histogram belong, or a level (HL) to which pixels having a cumulative frequency of 1% from the bright side belong. In the following S1232, the system control unit 50 converts the histogram feature amount calculated in S1231 into a value assuming imaging at a proper exposure. This is a process for correcting the fact that the histogram feature amount is detected to be lower than that in the case of imaging at a proper exposure because the image data is imaged at an exposure lower than the proper exposure. The conversion of the luminance value may be performed so as to compensate for the exposure correction amount (reduction amount) determined by the exposure control, or may be performed using the gradation correction amount calculated in S1225.

[0170] In S1233, the system control unit 50 calculates a target correction amount. The system control unit 50 obtains a target luminance level for the representative luminance value of the face or a histogram feature amount. Then, the system control unit 50 creates a lookup table (input / output characteristics) that defines an output luminance level for an input luminance level as a gradation correction amount (B) by spline interpolation or the like from these target luminance levels and the minimum and maximum luminance values ​​in the image data. The gradation correction amount (B) is a second gradation correction amount.

[0171] Here, the target gradation correction amount may be different in HDR from SDR. For example, FIG. 16(a) shows an example of how it looks in SDR, and FIG. 16(b) shows an example of how it looks in HDR. The luminance value of the subject (person) is the same in both cases, but the background in SDR is at most 100 cd / m 2 whereas HDR is 100cd / m 2 16(a) and 16(b), the subject may appear darker in HDR. This is called brightness contrast, and is a phenomenon caused by human visual characteristics. For example, the subject has the same brightness in FIG. 16(a) and FIG. 16(b), but the difference between the subject's brightness and the background's brightness is larger in FIG. 16(b) than in FIG. 16(a). In such a case, the subject appears darker in FIG. 16(b) than in FIG. 16(a) by the user. In other words, since HDR can express high brightness areas such as the sky brighter, it is highly likely that the subject will appear darker than in SDR. Therefore, in this embodiment, the gradation characteristics shown in FIG. 15(a) are used in the case of SDR, whereas the gradation characteristics shown in FIG. 15(b) are used in the case of HDR, and a gradation correction amount that raises the dark areas is applied, thereby achieving a visually preferable result. Note that the gradation correction in this embodiment is an example of correction to compensate for insufficient exposure, but similar gradation correction can also be performed in brightness correction for image creation.

[0172] The target brightness level for the representative brightness value of the face or the histogram feature of the image data can be set to a fixed value that is empirically considered to be preferable, but different target brightness levels may be set according to the representative brightness value or the histogram feature value. In this case, a lookup table that defines the relationship of the target brightness level to the input level may be prepared for each parameter (representative brightness value or histogram feature value) that sets the target brightness level.

[0173] The correction characteristics for realizing the conversion to the target luminance level thus determined are obtained by a method such as spline interpolation, and are saved as a lookup table (or a relational expression) to which the gradation correction amount (B) is applied, if necessary.

[0174] In S1234, the system control unit 50 combines the amount of tone correction (A) calculated in S1225 and the amount of tone correction (B) calculated in S1233. For example, the system control unit 50 first applies the amount of tone correction (A) to each input luminance level, and then obtains a luminance value resulting from applying the amount of tone correction (B) to the luminance level after correction, and creates a lookup table of output luminance levels for each input luminance level.

[0175] In S1235, the system control unit 50 performs a process (limiter process) of limiting the upper limit of the composite correction amount (composite tone correction amount) calculated in S1234. By combining the tone correction amount (A) and the tone correction amount (B), the correction amount becomes large and the amount of noise becomes more noticeable in the corrected image, so a limit is set on the overall correction amount. The limiter process can be realized by preparing a maximum correction amount permissible for each luminance value as a table, and replacing an output level that exceeds the maximum correction amount among the values ​​of the lookup table created in S1234 with an output level corresponding to the maximum correction amount. Note that the tone correction amount may be calculated as a value for the shooting mode (SDR shooting or HDR shooting) set by the user, or may be calculated for both SDR and HDR.

[0176] Returning to the description of FIG. 7, in S707, the system control unit 50 performs development using the calculated white balance coefficient, tone correction parameters, and various HDR parameters. Other development parameters include a color matrix, camera OETF curve data, color adjustment parameters, noise reduction parameters, and sharpness parameters to generate an HDR developed image. As the camera OETF (gamma curve), for example, the inverse characteristics of the EOTF (Electro-Optical Transfer Function) of the PQ (Perceptual Quantization) of the ITU-R recommendation BT.2100 are assumed, but the flavor of the camera side may be combined as an OOTF (Opto-Optical Transfer Function). Alternatively, the OETF of the HLG (Hybrid Log-Gamma) of the ITU-R recommendation BT.2100 may be used.

[0177] In S708, the system control unit 50 resizes the image developed in S707 to generate an MPF ​​(Multi Pixel Format) image for simple display such as a two-screen comparison image, and compresses and encodes the image using HEVC.

[0178] In S709, the system control unit 50 further resizes the MPF image generated in S708 to generate and compress thumbnail images to be used for index display or the like.

[0179] In S710, the system control unit 50 compresses the HDR image developed in S707 as the main image. There are various possible compression methods, but for example, 10-bit YUV422 data may be compressed using H.265 (ISO / IEC 23008-2 HEVC).

[0180] In S711, the system control unit 50 determines the recording image quality setting by the user. If the system control unit 50 determines that the setting is to record only RAW images, the process proceeds to S712. If the system control unit 50 determines that the setting is to record only HDR images, the process proceeds to S713. If the system control unit 50 determines that the setting is to record both RAW images and HDR images, the process proceeds to S714.

[0181] In S712, the system control unit 50 compresses the RAW image, adds a header, and records the RAW image file having the structure shown in FIG. 8A on the recording medium 200 via the recording medium I / F 18. There are several possible compression methods, but lossless compression, which is lossless and lossy, which is lossy but reduces the file size, may be used. In addition, the header records the white balance white search frame determination result obtained in S1205, the histogram obtained in S704, and the face detection result obtained in S705 as detection metadata. The white search frame determination result detected here is the determination result before the light and dark exclusion process in S1206 is performed. Therefore, the same determination result is recorded in both HDR and SDR shooting. Also, when the user has set the HDR shooting mode, the development parameters for HDR, such as the white balance coefficient obtained in FIG. 12 and the gradation correction amount obtained in FIG. 14, and the MPF image for display generated by encoding the image data developed in HDR in HEVC format in S708 are also recorded as metadata, as shown in FIG. 8B(b). As described above, the contents of these data differ depending on whether the shooting is HDR or SDR. In the case of shooting in SDR, the development parameters when the above-mentioned white search frame determination and gradation characteristics for SDR are used are recorded. Even in the case of shooting in HDR, the processing of S702 to S706 may be performed for SDR, and development parameters for SDR may also be generated and both may be recorded. In addition, since generating development parameters for both HDR and SDR requires a large processing load, it may not be performed during continuous shooting, but may be performed when there is a relatively large processing load, such as during single shooting.

[0182] Furthermore, when there is room for processing load, such as in single shots, it is also possible to create an SDR image quality main image, an MPF ​​image, and a thumbnail image using SDR development parameters in addition to the HDR display image, and then record the HDR display image and the SDR display image in the same file (Figure 8B(c)).

[0183] Furthermore, when displaying thumbnails, since the images are small and it is sufficient to know what kind of image they are, it is acceptable to create and save only the thumbnail images created in S709 as SDR developed images (Fig. 8B(d)). With this configuration, it is also possible to display only thumbnail images even on display devices or PCs that do not support decoding of H.265, the HDR compression format.

[0184] In S713, the system control unit 50 compresses and encodes the developed HDR image in the HEVC format, adds static metadata or dynamic metadata, and records the image as a HEIF file in the HIEF (High Efficiency Image File Format) format on the recording medium 200 via the recording medium I / F 18. The static metadata includes the x and y coordinates of the three primary colors and white point of the display conforming to CEA-861.3, the maximum luminance value, the minimum luminance value, the content maximum luminance value (Maximum Content Light Level), the frame average luminance level (Maximum Frame-average Light Level), and the like. The dynamic metadata includes metadata of dynamic tone mapping of color volume conversion defined in SMPTE ST 2094, and the like. Note that, in order to express HDR characteristics with a PQ signal, a depth of 10 bits or more is preferable, but since the conventional JPEG format requires 8 bits, it is necessary to newly adopt a container for still image HDR. Here, we use a container for HEIF, an image file format developed by MPEG (Moving Picture Experts Group) and defined in MPEG-H Part 12 (ISO / IEC 23008-12). HEIF has the advantage that it can store not only the original image, but also thumbnails, multiple chronologically related images, and metadata such as EXIF ​​and XMP in a single file. Therefore, it can also store 10-bit image sequences encoded with HEVC, making it easy to reuse.

[0185] In S714 and S715, the processes of S712 and S713 are sequentially performed, and both the RAW image and the HDR image are recorded.

[0186] Next, the file structure of still image RAW image data recorded on the recording medium 200 in the above-mentioned recording process is shown in Fig. 8A. The file format exemplified below is the ISO base media file format defined in ISO / IEC14496-12. Therefore, this file format has a tree structure with each node called a box. Also, each box can have multiple boxes as child elements.

[0187] Image data 801 has at its beginning a box ftyp 802 for describing the file type, a box moov 803 containing all metadata, a box mdat 808 for the main body of the media data of the track, and other boxes 807. The aforementioned box moov 803 has, as child elements, a box uuid 804 for storing MetaData 805 and a trak box 806 for storing information referencing ImageData. MetaData 805 describes image metadata, and is composed of, for example, the date and time the image was created, the shooting conditions, information on whether it was shot in HDR or SDR, the aforementioned detection metadata, and other shooting information. The aforementioned box mdat 808 has, as a child element, ImageData 809, which is the shot still image data.

[0188] Note that image data recorded in ImageData809 differs between RAW images shot in SDR and RAW images shot in HDR.

[0189] Fig. 8B(a) shows ImageData 809 recorded in a RAW image captured in SDR. In this case, ImageData 809 has a THM image 821 developed in SDR quality and compressed in JPEG, an MPF ​​image 822, an original image 823, a RAW image 824, and RAW development parameters 825. Each SDR quality image has one color component that is 8 bits (256 gradations). Note that the RAW development parameters 825 in Fig. 8B(a) include at least development parameters for SDR development.

[0190] Fig. 8B(b) shows ImageData 809 recorded in a RAW image that has only the HDR image as the display image when shooting in HDR. In this case, ImageData 809 has a THM image 826 developed in HDR quality and compressed in HEVC, an MPF ​​image 827, an original image 828, a RAW image 824, and RAW development parameters 825. Each HDR quality image is an image with 10 bits (1024 gradations) per color component. The RAW development parameters 825 in Figs. 8B(b), (c), and (d) include at least development parameters for HDR development.

[0191] 8B(c) shows ImageData 809 recorded in a RAW image having both an HDR image and an SDR image as images for display when shooting in HDR. In this case, ImageData 809 has a THM image 821 developed in SDR image quality and compressed in JPEG, an MPF ​​image 822, an original image 823, a THM image 826 developed in HDR image quality and compressed in HEVC, an MPF ​​image 827, an original image 828, a RAW image 824, and RAW development parameters 825.

[0192] 8B(d) shows ImageData 809 recorded in a RAW image in which only the THM image is an SDR image when shooting in HDR, and the MPF and main image are used as images for displaying the HDR image. In this case, ImageData 809 has a THM image 821 developed in SDR quality and compressed in JPEG, an MPF ​​image 827 developed in HDR quality and compressed in HEVC, a main image 828, a RAW image 82), and RAW development parameters 825.

[0193] The file format shown in this example is one embodiment, and other boxes may be included as necessary. Also, the display image may be included in a box in the moov 803 or in another box 807.

[0194] By having the above-mentioned file format, the development parameters for SDR images are recorded in the RAW image file shot as SDR, and the development parameters for HDR images are recorded in the RAW image file shot as HDR. In this way, even when developing the RAW image later, it is possible to develop it with the development parameters reflecting the settings at the time of shooting. For example, a device that performs RAW development (which may be the digital camera 100 or other devices such as a PC) refers to MetaData 805 of the RAW image and determines whether it was shot as HDR or SDR. If it is determined that it was shot as HDR, the RAW image is developed as an HDR image using the development parameters for HDR images included in the file. Also, if it is determined that it was shot as SDR, the RAW image is developed as an SDR image using the development parameters for SDR images included in the file. In order to make it possible to perform such processing, the digital camera 100 in this embodiment records the development parameters for SDR images in the RAW image file shot as SDR, and records the development parameters for HDR images in the RAW image file shot as HDR. In addition, devices that perform RAW development can simply record developed still HDR images using the HEIF container mentioned above.

[0195] Furthermore, because the same judgment results are recorded as detection metadata for both HDR and SDR shooting, even RAW image files shot in HDR shooting mode can be developed into SDR images using the recorded detection data. Therefore, even devices that only support SDR images can properly display RAW image files shot in HDR shooting mode.

[0196] 9A is a flowchart showing details of a playback mode process by system control unit 50 using display unit 28. This process is realized by system control unit 50 expanding a program recorded in non-volatile memory 56 into system memory 52 and executing it.

[0197] In S901, the system control unit 50 determines whether index playback or normal playback is to be performed. If the system control unit 50 determines index playback in S901, the process proceeds to S902. In S902, the system control unit 50 determines the number of images to be played back.

[0198] In S903, the system control unit 50 determines an image to be played back. Then, in S904, the system control unit 50 performs a drawing process for the image to be played back.

[0199] In S905, the system control unit 50 determines whether the rendering of all images to be displayed has been completed, and if it is determined that the rendering has not been completed, the process returns to S903 and continues the rendering process. If the system control unit 50 determines that the rendering process has been completed, the process proceeds to S906. In S906, the system control unit 50 performs the image output process of S906 to the display unit 28 and ends the display process. Thereafter, the system control unit 50 performs an operation reception process.

[0200] FIG. 9B is a flowchart showing details of the drawing process of the system control unit 50 in the playback mode process using the display unit 28.

[0201] At S911, the system control unit 50 acquires information about the image to be played. At S912, the system control unit 50 determines the image to be played. At S913, the system control unit 50 reads the image to be played from the recording medium 200. At S914, the system control unit 50 performs decompression processing of the image to be played. At S915, the system control unit 50 collects data for each pixel from the image data that has been decompressed at S914. This image data is, for example, luminance information, and is used for histogram processing, highlight warning processing, and the like.

[0202] In S916, the system control unit 50 determines whether the image to be played is an HDR image or an SDR image. If the system control unit 50 determines that the image to be played is an HDR image, the process proceeds to S917. If the system control unit 50 determines that the image to be played is an SDR image, the process proceeds to S920. In S917, the system control unit 50 checks the HDR assist display setting during playback, and if the setting is assist 1, the process proceeds to S918, and if the setting is assist setting 2, the process proceeds to S919. In S918, the system control unit 50 performs HDR to SDR conversion processing on the image expanded in S914 according to the setting of assist 1. In addition, in S919, the system control unit 50 performs HDR to SDR conversion processing on the image expanded in S914 according to the setting of assist 2.

[0203] In S920, the system control unit 50 performs enlargement / reduction processing on the image expanded in S914 or the image subjected to SDR conversion processing in S918 or S910 to a size suitable for the display unit 28. Then, in S921, the system control unit 50 determines the arrangement of the generated image and ends the drawing processing.

[0204] 9C to 9H are flowcharts showing the details of the read image selection process by the system control unit 50.

[0205] In S926, the system controller 50 checks the information of the acquired image to determine whether it is possible to play the image. If it is possible to play the image, the process proceeds to S927. If it is not possible to play the image, the process proceeds to S936.

[0206] In S927, the system control unit 50 determines whether the image to be played back is a still image. If the system control unit 50 determines that the image to be played back is a still image, the process proceeds to S928. If not, the process proceeds to S935.

[0207] In S928, the system control unit 50 determines whether the image to be played back is a RAW image. If the system control unit 50 determines that the image to be played back is a RAW image, the process proceeds to S929. If not, the process proceeds to S930.

[0208] In S929, the system control unit 50 determines whether the RAW image is a RAW image captured in HDR or SDR. The system control unit 50 makes this determination using the metadata in the RAW file described in Fig. 8. If the system control unit 50 determines that the RAW image is a RAW image captured in HDR, the process proceeds to S931. If the system control unit 50 determines that the image is a SDR image, the process proceeds to S932.

[0209] In S930, the system control unit 50 determines whether the still image determined not to be a RAW image was shot in HDR or SDR. In this embodiment, images shot in HDR are recorded as HEIF, and images shot in SDR are recorded as JPEG, so whether the image is HDR or SDR is determined based on HEIF or JPEG, but metadata in the HEIF may also be used to determine whether the image is HDR or SDR.

[0210] At S931, the system control unit 50 selects image data to be used for playback from the RAW images captured in HDR. At S932, the system control unit 50 selects image data to be used for playback from the RAW images captured in SDR. At S933, the system control unit 50 selects image data to be used for playback from the still images developed in HDR. At S934, the system control unit 50 selects image data to be used for playback from the still images developed in SDR. At S935, the system control unit 50 selects image data to be displayed from the video file. At S936, the system control unit 50 performs non-display processing of the playback image. In this case, information indicating that playback is not possible is displayed to inform the user that the image cannot be played.

[0211] FIG. 9D shows a flow in which the system control unit 50 selects image data to be used for playback from a RAW image captured in HDR.

[0212] In S941, the system control unit 50 determines whether the playback is index playback or normal playback. If the system control unit 50 determines that the playback is index playback, the process proceeds to S942. If the system control unit 50 determines that the playback is normal playback, the process proceeds to S943.

[0213] In S942, the system control unit 50 determines the image data to be used based on the number of images reproduced in index reproduction. In this embodiment, the threshold value is 36, but this number is one example and may be set appropriately by the user or may be determined depending on the size of the display unit 28. If the system control unit 50 determines that the number of images to be displayed is 36 or more, the process proceeds to S945, and if the number is determined to be less than 36, the process proceeds to S944.

[0214] In S943, the system control unit 50 determines the "HDR main image for display (HEVC)" (828) as the image data to be used for playback. In S944, the system control unit 50 determines the "HDR MPF image for display (HEVC)" (827) as the image data to be used for playback. In S945, the system control unit 50 determines the "HDR THM image for display (HEVC)" (826) as the image data to be used for playback.

[0215] FIG. 9E is a flow diagram showing a process of selecting image data to be used for playback from a RAW image when a RAW image captured in HDR has an SDR image for display.

[0216] In S951, the system control unit 50 determines whether the playback is index playback or normal playback. If the system control unit 50 determines that the playback is index playback, the process proceeds to S952. If the system control unit 50 determines that the playback is normal playback, the process proceeds to S953.

[0217] In S952, the system control unit 50 determines the image data to be used based on the number of images reproduced in index reproduction. Here, the threshold value for the determination is 36. If the system control unit 50 determines that the number of images reproduced is 36 or more, the process proceeds to S955. If the system control unit 50 determines that the number of images reproduced is less than 36, the process proceeds to S954.

[0218] In S953, S954, and S955, the system control unit 50 checks whether the RAW image to be played back contains an SDR image. This determination is made using the metadata in the RAW file, as described with reference to FIG.

[0219] In S956, the system control unit 50 determines the "HDR main image for display (HEVC)" (828) as the image data to be used for playback. In S957, the system control unit 50 determines the "SDR main image for display (JPEG)" (823) as the image data to be used for playback. In S958, the system control unit 50 determines the "HDR MPF image for display (HEVC)" (827) as the image data to be used for playback. In S959, the system control unit 50 determines the "SDR MPF image for display (JPEG)" (822) as the image data to be used for playback. In S960, the system control unit 50 determines the "HDR THM image for display (HEVC)" (826) as the image data to be used for playback. In S961, the system control unit 50 determines the "SDR THM image for display (JPEG)" (821) as the image data to be used for playback.

[0220] FIG. 9F shows a flow in which the system control unit 50 selects image data to be used for playback from the HDR developed still image.

[0221] In S971, the system control unit 50 determines whether the playback is index playback or normal playback. If the system control unit 50 determines that the playback is index playback, the process proceeds to S972, and if the system control unit 50 determines that the playback is normal playback, the process proceeds to S973.

[0222] In S972, the system control unit 50 determines the image data to be used based on the number of images reproduced in index reproduction. In this embodiment, the threshold number is set to 36. If the system control unit 50 determines that the number of images reproduced is 36 or more, the process proceeds to S975. If the system control unit 50 determines that the number of images reproduced is less than 36, the process proceeds to S974.

[0223] At S973, the system control unit 50 determines "HDR main image (HEVC)" (not shown) as the image data to be used for playback. At S974, the system control unit 50 determines "HDR MPF image (HEVC)" (not shown) as the image data to be used for playback. At S975, the system control unit 50 determines "HDR THM image (HEVC)" (not shown) as the image data to be used for playback.

[0224] FIG. 9G shows a flow for selecting image data to be used for playback from RAW images captured in SDR.

[0225] In S981, the system control unit 50 determines whether index playback or normal playback is to be performed. If the system control unit 50 determines index playback, the process proceeds to S982, and if the system control unit 50 determines normal playback, the process proceeds to S983.

[0226] In S982, the system control unit 50 determines the image data to be used based on the number of images reproduced in index reproduction. In this embodiment, the threshold number is set to 36. If the system control unit 50 determines that the number of images to be displayed is 36 or more, the process proceeds to S985. If the system control unit 50 determines that the number of images to be displayed is less than 36, the process proceeds to S984.

[0227] In S983, the system control unit 50 determines the "SDR main image for display (JPEG)" (823) as the image data to be used for playback. In S984, the system control unit 50 determines the "SDR MPF image for display (JPEG)" (822) as the image data to be used for playback. In S985, the system control unit 50 determines the "SDR THM image for display (JPEG)" (821) as the image data to be used for playback.

[0228] FIG. 9H shows a flow for selecting image data to be used for playback from an SDR developed still image.

[0229] In S991, the system control unit 50 determines whether index playback or normal playback is to be performed. If the system control unit 50 determines index playback, the process proceeds to S992, and if the system control unit 50 determines normal playback, the process proceeds to S993.

[0230] In S992, the system control unit 50 determines the image data to be used based on the number of images reproduced in index reproduction. In this embodiment, the threshold number is set to 36. If the system control unit 50 determines that the number of images to be reproduced is 36 or more, the process proceeds to S995. If the system control unit 50 determines that the number of images to be reproduced is less than 36, the process proceeds to S994.

[0231] At S993, the system control unit 50 determines "SDR main image (JPEG)" (not shown) as the image data to be used for playback. At S994, the system control unit 50 determines "SDR MPF image (JPEG)" (not shown) as the image data to be used for playback. At S995, the system control unit 50 determines "SDR THM image (JPEG)" (not shown) as the image data to be used for playback.

[0232] 10A is a flowchart showing details of a playback mode process using external device 300. This process is realized by loading a program recorded in non-volatile memory 56 into system memory 52 and having system control unit 50 execute the program.

[0233] In S1001, the system control unit 50 determines whether or not the external device 300 is connected to the digital camera 100. If the system control unit 50 determines that the external device 300 is connected, the process proceeds to S1002, and if the system control unit 50 determines that the external device 300 is not connected, the process proceeds to S1005.

[0234] In S1002, the system control unit 50 determines whether the HDR setting during playback is enabled. As the playback setting, "HDR playback", "HDR playback not enabled", and "shooting mode linked" can be selected. When "HDR playback" is set, regardless of whether the image to be played is an HDR image or an SDR image, if the external device 300 supports HDR, the mode is HDR output, and "HDR playback not enabled" is SDR output. "Shooting mode linked" is a mode in which the output during playback is linked to the shooting mode. That is, in the HDR shooting mode in which "HDR shooting" is set to "enabled", HDR output is also output during playback, and in the SDR shooting mode in which "HDR shooting" is set to "not enabled", SDR output is also output during playback. Note that the default is set to "shooting mode linked", and the playback setting remains "shooting mode linked" even if the user changes the shooting mode. The link is broken only when the user changes the playback setting from "shooting mode linked" to "HDR playback" or "not HDR playback". Also, instead of "HDR playback" and "no HDR playback", file formats such as "HEIF (playback)" and "JPEG (playback)" may be options. Similarly, instead of "HDR shooting" and "no HDR shooting", file formats such as "HEIF (shooting)" and "JPEG (shooting)" may be options.

[0235] In S1002, the system control unit 50 advances the process to S1003 if "HDR playback is performed," and advances the process to S1005 if "HDR playback is not performed." Also, in the case of "shooting mode linkage," if "HDR shooting" set in S606 is "on," the system control unit 50 advances the process to S1003, and if "HDR shooting is not performed," the system control unit 50 advances the process to S1005.

[0236] In S1003, the system control unit 50 determines whether the external device 300 is a display that supports HDR. If the system control unit 50 determines that the external device 300 is a display that supports HDR, the process proceeds to S1004. If the system control unit 50 determines that the external device 300 is not compatible, the process proceeds to S1005.

[0237] In S1004, the system control unit 50 outputs an HDR signal to the external device 300. In S1005, the system control unit 50 outputs an SDR signal to the external device 300.

[0238] S1006 to S1011 are the same as S901 to S906 in FIG. 9A, and therefore the description thereof will be omitted here.

[0239] FIG. 10B is a flowchart showing details of the rendering process (S1009) when an HDR signal is output to the external device 300.

[0240] Since S1021 to S1025, S1028, and S1029 are the same as S911 to S915, S920, and S921 described with reference to FIG. 9B, the description thereof will be omitted here.

[0241] In S1026, the system control unit 50 determines whether the image to be played is an HDR image or an SDR image. If the system control unit 50 determines that the image to be played is an HDR image, the process proceeds to S1028. If the system control unit 50 determines that the image to be played is an SDR image, the process proceeds to S1027.

[0242] In S1027, the system control unit 50 performs SDR → HDR conversion processing. Subsequent steps S1028 and S1029 are the same as steps S920 and S921 in Fig. 9B. Note that details of the drawing process (S1009) when an SDR signal is output to the external device 300 are the same as those in Fig. 9B, and therefore will not be described.

[0243] 11A is a flow chart showing the details of the playback menu process. This process is realized by the system control unit 50 loading a program recorded in the non-volatile memory 56 into the system memory 52 and executing it.

[0244] In S1101, the system control unit 50 determines whether or not the user has set RAW development in the setting item (not shown) for RAW development. If the system control unit 50 determines that RAW development is not to be performed, the process proceeds to S1103, and if the system control unit 50 determines that RAW development is to be performed, the process proceeds to S1102.

[0245] In S1103, the system control unit 50 determines whether or not the setting item (not shown) for SDR conversion of the HDR file is set to HDR → SDR conversion. If the system control unit 50 determines not to perform HDR → SDR conversion, the process proceeds to S1105, and if it determines to perform HDR → SDR conversion, the process proceeds to S1104.

[0246] In S1105, the system control unit 50 determines whether or not a file transfer is set in a file transfer setting item (not shown). If the system control unit 50 determines that the file is not to be transferred, the process proceeds to S1107. If the system control unit 50 determines that the file is to be transferred, the process proceeds to S1106.

[0247] In S1107, the system control unit 50 determines whether or not to exit the menu. If the system control unit 50 determines not to exit the menu, the process returns to S1101, and if it determines to exit the menu, the system control unit 50 ends this playback menu process.

[0248] In S1106, the system control unit 50 performs a transfer process on the user-specified image file. When transferring an HDR image file, if the receiving device can only display SDR, the HDR to SDR conversion shown in S1104 may be performed in the camera before transferring the image as an SDR image file.

[0249] In S1102, the system control unit 50 performs RAW development on the RAW image file designated by the user. Details of this RAW development process will be described below with reference to the block diagram of Fig. 11(b). Note that each processing unit shown in Fig. 11(b) is included in the image processing unit 24, but may be realized by a program executed by the system control unit 50.

[0250] The system control unit 50 reads out the captured RAW image 1101 recorded on the recording medium 200, and causes the image processing unit 24 to perform RAW development processing. Since the RAW image is a set of pixels in a Bayer array, one pixel has only intensity data of a single color component. Note that there are RAW images (SDR) when SDR photography is performed and RAW (HDR) when HDR photography is performed. In addition, when developing, RAW (SDR) may be directly developed into SDR or developed into HDR. Conversely, RAW (HDR) may be developed into HDR or developed into SDR. In the white balance unit 1102, processing is performed to make white white. When RAW (HDR) is developed into HDR, white balance processing is performed using a white balance coefficient for HDR, which is HDR development meta recorded in the file. Conversely, when developing into SDR, white balance processing is performed by generating a white balance coefficient for SDR from the result of the white search frame determination, which is the detection meta stored in the file. Of course, if the RAW contains both HDR and SDR white balance coefficients, you can use the one you need.

[0251] The color interpolation unit 1103 generates a color image in which each pixel has three components (for example, R, G, and B color information) for all pixels by performing noise reduction and interpolating a color mosaic image. The generated color image passes through a matrix conversion unit 1104 and a gamma conversion unit 1105 to generate a basic color image. Then, a color brightness adjustment unit 1106 performs processing on the color image to improve the appearance of the image. For example, image correction such as detecting a sunset and emphasizing saturation is performed according to the scene. Gradation correction is also performed in the same way, but when RAW (HDR) is developed in HDR, the gradation correction is performed using the gradation correction amount for HDR, which is the HDR development meta stored in the file. Conversely, when SDR development is performed, the gradation correction amount for SDR is calculated using the face detection result and histogram, which are the detection meta recorded in the file, and gradation correction is performed. Of course, if the gradation correction amounts for both HDR and SDR are recorded in the RAW, the one required can be used as appropriate.

[0252] For an image that has been subjected to the desired color adjustment, a compression unit 1107 compresses the high-resolution image using a method such as JPEG or HEVC, and a developed image is generated to be recorded on a recording medium such as a flash memory by a recording unit 1108. Note that since the HEIF container described above can store multiple images, it is possible to store an SDR developed image in addition to an HDR developed image.

[0253] In S1104, the system control unit 50 performs SDR conversion on the user-specified HDR image file. Since the HDR image is an image generated in a color space such as PQ OETF and BT.2020 gamut, it is necessary to perform tone mapping and gamut mapping processing on the color space such as γ2.2 and sRGB of SDR. As a specific method, a known technique may be used, but for example, by performing tone mapping to align the correct exposure with SDR, it is possible to obtain a result with brightness adjusted compared to SDR.

[0254] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, in the above embodiment, HEVC (High Efficiency Video Coding) is adopted for encoding image data in which one color component exceeds 8 bits, but as long as an image in which more than 8 bits per color component can be encoded, the type of image is not particularly important. In addition, in the above embodiment, the present invention is described as being applied to a digital camera, but it may also be applied to a computer having an imaging function (such as a smartphone or a notebook PC with a camera), and is not limited to the above embodiment.

[0255] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0256] 100... digital camera, 150... lens unit, 300... external device, 50... system control unit, 70... operation unit, 200... recording medium

Claims

1. An imaging means; An encoding means for performing an encoding process on image data; a control means for controlling, when recording a RAW image file, whether to record first encoded image data obtained by performing encoding processing in a first encoding format on image data of a first dynamic range generated based on the RAW image data as an image for display of the RAW image file, or to record second encoded image data obtained by performing encoding processing in a second encoding format on image data of a second dynamic range generated based on the RAW image data as an image for display of the RAW image file; An imaging device comprising:

2. A first setting means for setting a shooting mode, When recording a RAW image file, the control means when the first setting means has set the shooting mode of the first dynamic range, control is performed so that first encoded image data obtained by performing encoding processing in the first encoding format on the image data of the first dynamic range generated based on RAW image data obtained by imaging by the imaging means in the shooting mode of the first dynamic range is recorded as a display image of a RAW image file in which the RAW image data is stored; when the second dynamic range shooting mode is set by the first setting means, control is performed so that second encoded image data obtained by performing encoding processing in the second encoding format on the image data of the second dynamic range generated based on RAW image data obtained by imaging by the imaging means in the second dynamic range shooting mode is recorded as a display image of a RAW image file in which the RAW image data is stored.

2. The imaging device according to claim 1 .

3. The control means a control unit for controlling the recording of a raw image file and the recording of image data of the first dynamic range as an image file different from the raw image file, the control unit controls the recording of a raw image file in which raw image data is stored and first encoded image data obtained by performing encoding processing in the first encoding format on image data of the first dynamic range generated based on the raw image data is stored as an image for display; When a raw image file is recorded and the image data of the second dynamic range is recorded as an image file different from the raw image file, the control is performed so as to record a raw image file in which raw image data is stored and second encoded image data obtained by performing encoding processing in the second encoding format on the image data of the second dynamic range generated based on the raw image data is stored as an image for display.

2. The imaging device according to claim 1 .

4. The imaging device as described in claim 3, characterized in that it has a second setting means for setting the recording image quality regarding whether to record only a RAW image file or to record a RAW image file and also an image file different from the RAW image file.

5. The imaging device described in Claim 4, characterized in that the second setting means sets the recording image quality to be either to record only a RAW image file, to record a RAW image file and also an image file different from the RAW image file, or to record only the different file.

6. Having a first setting means for setting a dynamic range, 5. The imaging device according to claim 4, wherein the control means controls the image file format to be recorded and the dynamic range and encoding format of the image data to be stored in the image file to be recorded, in accordance with the dynamic range set by the first setting means and the recording image quality set by the second setting means.

7. The imaging device described in claim 6, characterized in that, when a RAW image file is set to be recorded by the second setting means, if a first dynamic range is set by the first setting means, the control means controls to record first encoded image data obtained by subjecting image data of the first dynamic range generated based on RAW image data obtained by imaging by the imaging means to encoding processing in the first encoding format as a display image for the RAW image file, and, when a second dynamic range is set by the first setting means, controls to record second encoded image data obtained by subjecting image data of the second dynamic range generated based on RAW image data obtained by imaging by the imaging means to encoding processing in the second encoding format as a display image for the RAW image file.

8. The control means when a setting is made by the first setting means and the second setting means to record a RAW image file and a first image file for the first dynamic range, control is performed so that first encoded image data obtained by performing encoding processing in the first encoding format on image data of the first dynamic range generated based on RAW image data captured by the imaging means is recorded as an image for display of the RAW image file, when a setting is made by the first setting means and the second setting means to record a RAW image file and a second image file for the second dynamic range, control is performed so that second encoded image data obtained by performing encoding processing in the second encoding format on image data of the second dynamic range generated based on RAW image data captured by the imaging means is recorded as an image for display of the RAW image file.

8. The imaging device according to claim 6, wherein the first and second lenses are arranged in a first direction.

9. The control means when a setting to record a RAW image file and the first image file for the first dynamic range is set by the first setting means and the second setting means, a control is performed so that first encoded image data obtained by performing encoding processing in the first encoding format on image data of the first dynamic range generated based on RAW image data obtained by imaging by the imaging means is recorded in the RAW image file and the first image file, respectively; when a setting to record a RAW image file and a second image file for the second dynamic range is set by the first setting means and the second setting means, control is performed so that second encoded image data, which is generated based on RAW image data captured by the imaging means and subjected to encoding processing in the second encoding format, is recorded in the RAW image file and the second image file, respectively.

9. The imaging device according to claim 8.

10. The first image file is a JPEG format image file, The second image file is an image file in a High Efficiency Image File Format (HEIF) format.

10. The imaging device according to claim 8, wherein the first and second lenses are arranged in a first direction.

11. The imaging device described in Claim 1, characterized in that when recording the second encoded image data as a display image in a RAW image file, the control means controls so that third encoded image data, which is smaller in size than the second encoded image data and has been subjected to encoding processing in the first encoding format, is recorded together with the second encoded image data in the RAW image file as a display image.

12. The imaging device described in Claim 1, characterized in that the control means controls at least when recording RAW image data in a RAW image file, to record first display image data and second display image data having an image size smaller than the first display image data in the RAW image file.

13. The control means when recording the second encoded image data as the first display image data of a RAW image file, control is performed so that third encoded image data that has been subjected to encoding processing in the first encoding format is recorded as the first display image data; The imaging device described in claim 12, characterized in that when the first encoded image data is recorded as the first display image data of a RAW image file, the imaging device controls so that fourth encoded image data subjected to encoding processing in the first encoding format is recorded as the first display image data.

14. A selection means for selecting a file format of an image file when recording an image file other than the RAW image file together with the RAW image file, The control means when a first file format is selected by the selection means, control is performed so as to record a RAW image file that stores RAW image data and stores, as an image for display, first encoded image data obtained by performing encoding processing in the first encoding format on the image data of the first dynamic range generated based on the RAW image data, and an image file of the first file format that stores image data obtained by performing encoding processing in the first encoding format on the image data of the first dynamic range generated based on the RAW image data, when the second file format is selected by the selection means, control is performed so as to record a RAW image file that stores RAW image data and stores, as an image for display, second encoded image data obtained by performing encoding processing in the second encoding format on the image data of the second dynamic range generated based on the RAW image data, and an image file of the second file format that stores image data obtained by performing encoding processing in the first encoding format on the image data of the first dynamic range generated based on the RAW image data.

2. The imaging device according to claim 1 .

15. The control means when a first file format is selected by the selection means, the first encoded image data is controlled to be recorded in the RAW image file and an image file in the first file format, when the second file format is selected by the selection means, the second encoded image data is controlled to be recorded in the RAW image file and an image file in the second file format, respectively.

15. The imaging device according to claim 14.

16. The first file format is a JPEG format, and the second file format is a High Efficiency Image File Format (HEIF).

16. The imaging device according to claim 14 or 15.

17. The method of claim 1, wherein the first dynamic range image data is 8-bit image data per color component; The second dynamic range image data is image data having more than 8 bits per color component.

17. The imaging device according to claim 1,

18. An imaging device as described in any one of claims 1 to 17, characterized in that the first dynamic range is SDR and the second dynamic range is HDR.

19. An imaging device as described in any one of claims 1 to 18, characterized in that the first encoding format is a JPEG format, and the second encoding format is a HEVC format.

20. An imaging step; An encoding step of performing an encoding process on the image data; a control step of controlling, when recording a RAW image file, whether to record first encoded image data obtained by performing encoding processing in a first encoding format on image data of a first dynamic range generated based on the RAW image data as an image for display of the RAW image file, or to record second encoded image data obtained by performing encoding processing in a second encoding format on image data of a second dynamic range generated based on the RAW image data as an image for display of the RAW image file; 13. A method for controlling an imaging apparatus comprising:

21. A program for causing a computer to function as each of the means of an imaging device described in any one of claims 1 to 19.

Images