Imaging apparatus, control method thereof, and storage medium

Abstract

The present application provides an imaging apparatus, a control method thereof, and a storage medium. The imaging apparatus includes an imaging unit, a selection unit configured to select a dynamic range, a developing unit configured to perform developing processing on RAW image data obtained by the imaging unit, and a control unit configured to control so that, if the selection unit selects a first dynamic range, image data obtained by the developing unit by performing developing processing for the first dynamic range on the RAW image data is recorded as a RAW image file together with the RAW image data, and if the selection unit selects a second dynamic range, image data obtained by the developing unit by performing developing processing for the second dynamic range on the RAW image data is recorded as a RAW image file together with the RAW image data.

Description

[0001] (This application is a divisional application of the application filed on February 18, 2020, with application number 2020800168773 and title "Camera Equipment and Control Method and Program Thereof".) Technical Field

[0002] This invention relates to camera equipment and its control methods and procedures. Background Technology

[0003] Modern video recording devices, such as digital cameras, are capable of capturing HDR images and recording them on recording media. HDR stands for High Dynamic Range, and it is a technique that generates images with a wider dynamic range than SDR (Standard Dynamic Range). Furthermore, RAW images refer to the undeveloped, raw images.

[0004] When recording the content of an HDR video, the content is recorded together with identification information indicating whether the content is an HDR video (e.g., Patent Document 1).

[0005] Reference List

[0006] Patent documents

[0007] Patent document 1: Japanese Patent Application Publication No. 2018-7194 Summary of the Invention

[0008] Technical issues

[0009] A typical RAW image file can contain JPEG compressed data as the display image. However, because JPEG images do not correspond to HDR image quality, the display image cannot be saved in HDR image quality. Therefore, even when displaying a RAW image taken in HDR on an HDR monitor, the RAW image must be developed once to check for an image with HDR image quality.

[0010] The present invention was made in view of the above problems, and the present invention provides a technique by which, even when recording RAW image data in a camera device that supports multiple dynamic ranges such as SDR and HDR, the display image data corresponding to the selected dynamic range is also recorded in the RAW image file. Therefore, when playing back the RAW image file, the image with the selected dynamic range can also be examined.

[0011] Technical means to solve the problem

[0012] To address the aforementioned problems, the camera device of the present invention, for example, has the following arrangement: The camera device includes: a camera component; a selection component for selecting a dynamic range; a display component for performing display processing on RAW image data obtained by the camera component; and a control component for controlling the recording of RAW image data obtained by the camera component into a RAW image file, such that if the selection component selects a first dynamic range, image data obtained by performing display processing on the RAW image data for the first dynamic range by the display component is recorded together with the RAW image data into a RAW image file; and if the selection component selects a second dynamic range, image data obtained by performing display processing on the RAW image data for the second dynamic range by the display component is recorded together with the RAW image data into a RAW image file.

[0013] Advantages of the present invention

[0014] According to the present invention, even when recording RAW image data in a camera device that supports multiple types of dynamic range such as SDR and HDR, the display image data corresponding to the selected dynamic range is also recorded in the RAW image file. Therefore, for example, when performing a simple display (such as a file list display), the image with the selected dynamic range can be examined.

[0015] Furthermore, according to the present invention, even when recording RAW image data in a camera device that supports multiple types of dynamic range such as SDR and HDR, the display image data corresponding to the selected dynamic range is also recorded in the RAW image file. Therefore, for example, when performing a simple display (such as a file list display), an image with the selected dynamic range can be examined.

[0016] Alternatively, when recording multiple types of files in a camera device that supports multiple types of dynamic range, the present invention can prevent management complexities and maintain playback compatibility.

[0017] Other features and advantages of the invention will become clearer from the following description taken in conjunction with the accompanying drawings. Note that throughout the drawings, the same reference numerals denote the same or similar parts. Attached Figure Description

[0018] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the invention and, together with the textual description, serve to explain the principles of the invention.

[0019] Figure 1A This is an external view of the camera / display device according to an embodiment;

[0020] Figure 1B This is an external view of the camera / display device according to an embodiment;

[0021] Figure 2 This is a block diagram illustrating the configuration of a camera / display device according to an embodiment;

[0022] Figure 3 This is a diagram illustrating the connection configuration with external devices;

[0023] Figure 4A-1 This is a flowchart illustrating the LV shooting mode processing according to an embodiment;

[0024] Figure 4A-2 This is a flowchart illustrating the LV shooting mode processing according to an embodiment;

[0025] Figure 4A-3 This is a flowchart illustrating the LV shooting mode processing according to an embodiment;

[0026] Figure 4A-4 This is a flowchart illustrating the LV shooting mode processing according to an embodiment;

[0027] Figure 4B-1 It is a flowchart for quick review;

[0028] Figure 4B-2 It is a flowchart for quick review;

[0029] Figure 5A This is a sequence diagram of HDMI connection processing according to an embodiment;

[0030] Figure 5B This is a sequence diagram of HDMI connection processing according to an embodiment;

[0031] Figure 5C This is a sequence diagram of HDMI connection processing according to an embodiment;

[0032] Figure 6A-1 This is a flowchart of the HDR shooting menu processing according to an embodiment;

[0033] Figure 6A-2 This is a flowchart of the HDR shooting menu processing according to an embodiment;

[0034] Figure 6B-1 This is a flowchart of the HDR shooting menu processing according to an embodiment;

[0035] Figure 6B-2 This is a flowchart of the HDR shooting menu processing according to an embodiment;

[0036] Figure 7A This is a flowchart of HDR shooting processing according to an embodiment;

[0037] Figure 7B This is a flowchart of HDR shooting processing according to an embodiment;

[0038] Figure 8A This is a diagram illustrating the configuration of a RAW file according to an embodiment;

[0039] Figure 8B This is an example diagram showing the ImageData area in a RAW file;

[0040] Figure 8C This is a diagram showing an example of the ImageData area in a RAW file;

[0041] Figure 8D This is a diagram showing an example of the ImageData area in a RAW file;

[0042] Figure 8E This is a diagram showing an example of the ImageData area in a RAW file;

[0043] Figure 9A This is a flowchart illustrating the playback mode processing according to an embodiment;

[0044] Figure 9B This is a flowchart illustrating the playback mode processing according to an embodiment;

[0045] Figure 9C This is a flowchart illustrating the playback mode processing according to an embodiment;

[0046] Figure 9D This is a flowchart illustrating the playback mode processing according to an embodiment;

[0047] Figure 9E-1 This is a flowchart illustrating the playback mode processing according to an embodiment;

[0048] Figure 9E-2 This is a flowchart illustrating the playback mode processing according to an embodiment;

[0049] Figure 9F This is a flowchart illustrating the playback mode processing according to an embodiment;

[0050] Figure 9G This is a flowchart illustrating the playback mode processing according to an embodiment;

[0051] Figure 9H This is a flowchart illustrating the playback mode processing according to an embodiment;

[0052] Figure 10A This is a flowchart of the HDMI playback process according to an embodiment;

[0053] Figure 10B This is a flowchart of the HDMI playback process according to an embodiment;

[0054] Figure 11AThis is a flowchart of the playback menu processing according to an embodiment;

[0055] Figure 11B This is a flowchart of the playback menu processing according to an embodiment;

[0056] Figure 12 This is a flowchart of the imaging process according to an embodiment;

[0057] Figure 13 It shows C x C y A two-dimensional diagram;

[0058] Figure 14A This is a flowchart of the color correction parameter generation process;

[0059] Figure 14B This is a flowchart of the color correction parameter generation process;

[0060] Figure 15A This is a graph showing the tonal correction amount;

[0061] Figure 15B This is a graph showing the tonal correction amount;

[0062] Figure 16A This is a diagram showing an example of what an SDR would look like;

[0063] Figure 16B This is an example diagram showing what HDR looks like;

[0064] Figure 17A This is a diagram showing the configuration of the HEIF file;

[0065] Figure 17B This is a diagram showing an example of the ImageData area in a HEIF file;

[0066] Figure 17C This is a diagram showing an example of the ImageData area in a HEIF file;

[0067] Figure 17D This is a diagram showing an example of the ImageData area in a HEIF file;

[0068] Figure 18 This is a diagram showing the configuration of a JPEG file;

[0069] Figure 19A It is a flowchart of the shooting process based on the variant example; and

[0070] Figure 19B This is a flowchart of the shooting process based on the variant example. Detailed Implementation

[0071] The embodiments will be described in detail below with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the invention. Several features are described in the embodiments, but the invention is not limited to requiring all of these features; rather, these features can be appropriately combined. Furthermore, in the drawings, the same or similar configurations are given the same reference numerals, and redundant descriptions thereof are omitted.

[0072] Figure 1A and Figure 1B The appearance of a digital camera 100, which is an example of a device according to the present invention, is shown. Figure 1A This is a front stereoscopic view of the digital camera 100. Figure 1B This is a rear stereoscopic view of the digital camera 100. (See reference) Figure 1A and Figure 1BThe display unit 28 is located on the back of the camera and displays images and various information. The viewfinder external display unit 43 is located on the top surface of the camera and displays various camera settings, such as shutter speed and f-value. The shutter button 61 is an operating unit used by the user to issue shooting commands. The mode switch 60 is an operating unit used to switch between various types of modes. The terminal cover 40 is a cover used to protect connectors (not shown) (such as connecting cables for connecting external devices to the digital camera 100). The main electronic dial 71 is a rotary operating member included in the operating unit 70. For example, the user can change settings such as shutter speed or f-value by rotating the main electronic dial 71. The power switch 72 is an operating member used to turn the power of the digital camera 100 on and off. The sub-electronic dial 73 is a rotary operating member included in the operating unit 70 and used for operations such as moving the selection box and image feeding. The cross key 74 (four-way key) is included in the operating unit 70. The user can press the up, down, right, and left sides of the cross key 74. Pressing the directional pad 74 performs the operation corresponding to the pressed portion of the directional pad 74. The setting button 75 is included in the operation unit 70 and is primarily used to determine the selection item. The LV button 76 is included in the operation unit 70 and is used to turn Live View (hereinafter referred to as LV) on and off in still image shooting mode. In motion image shooting mode, this button is used to issue a command to start or stop motion image shooting (recording). The zoom-in button 77 is included in the operation unit 70 and is used to turn the zoom-in mode in the Live View display on and off in shooting mode, and to change the magnification in zoom-in mode. In playback mode, this button is used to magnify the playback image and increase the magnification. The zoom-out button 78 is included in the operation unit 70 and is used to reduce the magnification of the magnified playback image and shrink the displayed image. The playback button 79 is included in the operation unit 70 and is used to switch between shooting mode and playback mode. Pressing the playback button 79 during shooting mode will switch to playback mode, allowing the display unit 28 or external device 300 to display the latest image from the images recorded on the recording medium 200. The quick-return mirror 12 is moved up and down via an actuator (not shown) according to instructions from the system control unit 50. The communication terminal 10 allows the digital camera 100 to communicate with the lens side (detachable). The eyepiece viewfinder 16 is an observation viewfinder used to check the focus and composition of the optical image of the subject obtained through the lens unit 150 by observing the focusing screen 13. The cover 202 is a cover for the slot housing the recording medium 200. The grip 95 is a holding unit shaped to allow the user to easily grip the digital camera 100 with their right hand.

[0073] Figure 2This is a block diagram illustrating an example of the arrangement of a digital camera 100 according to this embodiment.

[0074] Reference Figure 2 The lens unit 150 includes an interchangeable lens.

[0075] Although lens 103 is usually composed of multiple lenses, for simplicity, Figure 2 Only one lens is shown, designated lens 103. Communication terminal 6 is used by lens unit 150 to communicate with digital camera 100. Communication terminal 10 is used by digital camera 100 to communicate with lens unit 150. Lens unit 150 communicates with system control unit 50 via communication terminals 6 and 10, and causes internal lens system control circuit 4 to control aperture 1 via aperture drive circuit 2, and adjust focus by moving the position of lens 103 via AF drive circuit 3.

[0076] The AE sensor 17 measures the brightness of the subject through the lens unit 150. The focus detection unit 11 outputs defocus information to the system control unit 50. Based on the input defocus information, the system control unit 50 controls the lens unit 150 to perform phase difference AF.

[0077] During exposure, live view shooting, and moving image shooting, the fast return mirror 12 (hereinafter referred to as mirror 12) is moved up and down by an actuator (not shown) according to instructions from the system control unit 50. Mirror 12 is a mirror used to switch the incident light beam from lens 103 between the eyepiece viewfinder 16 side and the imaging unit 22 side. Mirror 12 is typically used to reflect the light beam to guide it to the eyepiece viewfinder 16. During shooting or live view display, mirror 12 flips upward and retracts from the light beam (mirror rises) to guide the light beam to the imaging unit 22. Additionally, the central portion of mirror 12 is a semi-transparent, semi-reflective mirror to transmit a portion of the light. Mirror 12 transmits a portion of the light beam so that it is incident on the focus detection unit 11 used for focus detection.

[0078] The user (photographer) can check the focus and composition of the optical image of the subject obtained through the lens unit 150 by observing the focusing screen 13 through the pentaprism 14 and the eyepiece viewfinder 16.

[0079] Shutter 101 is a focal plane shutter that can freely control the exposure time of the imaging unit 22 under the control of the system control unit 50.

[0080] The imaging unit 22 is an image sensor formed by a CCD or CMOS device that converts optical images into electrical signals. Filters for the color components R, G, and B are arranged periodically in two dimensions on the imaging surface of the imaging unit 22. In adjacent 2×2 filters, the G component filters are arranged as two diagonally related filters, and the R and B component filters are arranged as the remaining two filters. These 2×2 filters are arranged on the imaging surface of the imaging unit 22. This array is generally referred to as a Bayer array. Therefore, the image represented by the signal (analog signal) output from the imaging unit 22 is also a pixel signal with a Bayer array. The A / D converter 23 converts the 1-pixel analog signal output from the imaging unit 22 into, for example, a 10-bit digital signal. Note that the image data at this stage is Bayer array image data as described above (one pixel has one component, and each component has 10 bits), and is undeveloped image data. Therefore, the image data at this stage is called RAW image data. Note that the Bayer array image data after compensating for defective pixels can also be called RAW image data. It should also be noted that in this embodiment, the A / D converter 23 converts the analog signal into 10-bit digital data, but there is no particular limitation on the number of bits, as long as it exceeds 8 bits. The more bits, the higher the hue can be expressed.

[0081] Image processing unit 24 performs resizing (such as predetermined pixel interpolation or reduction) and color conversion processing on data from A / D converter 23 or data from memory control unit 15. Additionally, image processing unit 24 performs predetermined arithmetic processing on the captured image data. System control unit 50 then performs exposure control and focus control based on the obtained arithmetic results. Through this operation, system control unit 50 performs TTL (through-lens) AF (autofocus), TTLAE (auto exposure), and TTL EF (electronic flash pre-emission) processing. Image processing unit 24 also performs predetermined arithmetic processing using the captured image data and performs TTL AWB (auto white balance) processing based on the obtained arithmetic results. Furthermore, image processing unit 24 encodes / decodes image data under the control of system control unit 50. This encoding includes JPEG and HEVC. JPEG is used to encode image data with 8 bits per color component, while HEVC is used to encode image data with more than 8 bits per color component.

[0082] Output data from A / D converter 23 is written to memory 32 via image processing unit 24 and memory control unit 15, or directly to memory 32 via memory control unit 15. Memory 32 stores image data acquired by camera unit 22 and converted into digital data by A / D converter 23, as well as image data to be displayed on display unit 28 or external device 300. Memory 32 has a sufficiently large storage capacity to store a predetermined number of still images, as well as moving images and sounds lasting a predetermined time.

[0083] The memory 32 also serves as a memory for image display (video memory). The D / A converter 19 converts the data stored in the memory 32 for image display into analog signals and provides them to the display unit 28. In this way, the image data written to the memory 32 for display is displayed via the display unit 28 through the D / A converter 19. The display unit 28 displays an image corresponding to the analog signal from the D / A converter 19 on a display device such as an LCD. The D / A converter 19 converts the digital signals stored in the memory 32 after one A / D conversion by the A / D converter 23 into analog signals, and sequentially transmits these signals to the display unit 28 to display images, thereby serving as an electronic viewfinder and performing pass-through image display (live view display).

[0084] The in-viewfinder LCD display unit 41 displays, via the in-viewfinder display unit drive circuit 42, a frame indicating the current autofocus point (AF frame) and icons indicating the camera's settings. The external viewfinder display unit 43 displays various camera settings, such as shutter speed and f-number, via the external viewfinder display unit drive circuit 44.

[0085] The digital output I / F 90 directly provides the image data for display stored in the memory 32 to the external device 300 in the form of a digital signal. For example, the digital output I / F 90 conforms to... The High Definition Multimedia Interface (HDMI) standard communication protocol outputs motion image data in streaming form. External device 300 then displays the image data written to memory 32 for display.

[0086] Non-volatile memory 56 is a memory capable of electrically erasing and recording data. For example, EEPROM and the like are used as non-volatile memory 56. Non-volatile memory 56 stores constants, programs, etc., for the operation of system control unit 50. In this case, the program is a program used to execute the various flowcharts described later in this embodiment.

[0087] The system control unit 50 is a controller with at least one processor that controls the entire digital camera 100. The system control unit 50 implements the various processes (described later) in this embodiment by executing programs recorded in the aforementioned non-volatile memory 56. The system memory 52 is RAM. Constants, variables, programs read from the non-volatile memory 56, etc., for the operation of the system control unit 50 are stored in the system memory 52. ​​The system control unit 50 also performs display control by controlling the memory 32, the D / A converter 19, the digital output I / F 90, the display unit 28, etc. Furthermore, the system control unit 50 converts the captured image data into a form to be recorded on the recording medium 200 and performs recording control to record the image data on the recording medium 200 via the recording medium I / F 18.

[0088] System timer 53 is a time measurement unit used to measure the time for various types of control and the time of the built-in timer.

[0089] The mode switch 60, the first shutter switch 62, the second shutter switch 64, and the operation unit 70 are operating components used to input various types of operation commands to the system control unit 50.

[0090] The mode switch 60 switches the operating mode of the system control unit 50 to one of, for example, still image recording mode, motion picture shooting mode, and playback mode. Still image recording modes include the following: automatic shooting mode, automatic scene detection mode, manual mode, aperture priority mode (Av mode), and shutter speed priority mode (Tv mode). Still image recording modes also include various types of scene modes with shooting settings specific to the shooting scene, program AE mode, and custom mode. The mode switch 60 directly switches the operating mode to one of these modes. Alternatively, after temporarily switching to a list screen of shooting modes using the mode switch 60, the user can select one of several display modes and switch to the selected mode using other operating components. Similarly, motion picture shooting modes can include multiple modes.

[0091] The user-operated shutter button 61 includes a first shutter switch 62 and a second shutter switch 64. When the user partially operates the shutter button, i.e., performs a so-called half-press operation (pre-shooting command), the first shutter switch 62 is activated to generate a first shutter switch signal SW1 61. Upon receiving the first shutter switch signal SW1, the system control unit 50 initiates operations such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (electronic flash pre-emission) processing. When the user fully operates the shutter button 61, i.e., performs a so-called full-press operation (shooting command), the second shutter switch 64 is activated to generate a second shutter switch signal SW2. In response to the second shutter switch signal SW2, the system control unit 50 initiates a series of shooting processes from reading signals from the imaging unit 22 to writing image data to the recording medium 200.

[0092] When a user selectively operates various function icons displayed on the display unit 28 or external device 300, each operating component of the operation unit 70 is assigned an appropriate function corresponding to the scene and used as various function buttons. Examples of function buttons are end button, back button, image send button, jump button, zoom out button, and attribute change button. For example, when the menu button 70e is pressed, the display unit 28 or external device 300 displays a menu screen where various settings can be made. The user can intuitively make various settings by using the menu screen displayed on the display unit 28 or external device 300, the four-way buttons for up, down, left, and right, and the setting button.

[0093] Note that the display unit 28 according to this embodiment has SDR quality image display function, that is, it can display each of the color components R, G, and B in 8 bits (256 hues). Also note that when the external device 300 is connected to the digital camera 100, the external device 300 is set as the output target device for captured images or live images, instead of the display unit 28. Furthermore, when the user explicitly selects either the display unit 28 or the external device 300 by operating the operation unit 70, the selected one is set as the output target device.

[0094] The operation unit 70 is an operating component used as an input unit to receive operations from the user. The operation unit 70 includes at least the following operation units: shutter button 61, main electronic dial 71, power switch 72, sub-electronic dial 73, cross key 74, setting button 75, LV button 76, zoom-in button 77, zoom-out button 78, and playback button 79. The cross key 74 is a directional button that the user can press to move up, down, right, and left. In this embodiment, the cross key 74 is described as an integrated operation unit, but the up, down, right, and left buttons can also be separate buttons. In the following description, the up or down portion will be referred to as the up / down button, and the left or right portion will be referred to as the left / right button. The operation unit 70 also includes the following operation units.

[0095] The AF-ON button 70b is a push-button switch included in the operation unit 70. The user can specify the execution of AF by pressing the AF-ON button 70b. The direction in which the AF-ON button 70b is pressed is parallel to the direction (optical axis) of the subject light incident from the lens 103 onto the imaging unit 22.

[0096] The quick settings button 70c (hereinafter referred to as the Q button 70c) is a push-button switch included in the operation unit 70. When the Q button 70c is pressed, a quick settings menu is displayed as a list of settings that can be set in various operation modes. For example, when the Q button 70c is pressed in the shooting standby state of live view shooting, settings such as electronic front curtain shutter, monitor brightness, WB of LV image, two-point magnification, and silent shooting are displayed in a list form with these settings superimposed on LV. The user selects any item in the quick settings menu by using the up / down keys and presses the settings button. The user can then change the setting corresponding to the selected setting and proceed to the operation mode corresponding to that item.

[0097] The active frame switch button 70d is a button switch included in the operation unit 70. When the active frame switch button 70d is pressed during two-point magnification processing (described later), the two magnification parts can be switched as active magnification positions (frames). Furthermore, the active frame switch button 70d is assigned different functions corresponding to the operation mode. When the active frame switch button 70d is pressed in playback mode, a protection attribute can be applied to the displayed image.

[0098] The menu button 70e is a push-button switch included in the operation unit 70, and the menu screen, which can be set in various ways, is displayed on the display unit 28 or the external device 300.

[0099] Function buttons 70f are three push-button switches included in the operation unit 70, and are assigned different functions. Each of the function buttons 70f is arranged in a position that can be operated by the fingers (middle finger, ring finger, or little finger) of the right hand holding the grip 95. The pressing direction is parallel to the direction (optical axis) of the subject light incident from the lens 103 onto the imaging unit 22.

[0100] The information button 70g is a button switch included in the operation unit 70 and used to switch between various information display operations.

[0101] The power control unit 80 includes, for example, a battery detection circuit, a DC-DC converter, and a switching circuit for switching the blocks to be supplied with power, and detects the presence / absence of the battery, the battery type, and the remaining battery power. Based on the detection results and instructions from the system control unit 50, the power control unit 80 controls the DC-DC converter and provides the required voltage to the unit including the recording medium 200 for the required time period.

[0102] The power supply unit 30 consists of a primary battery (such as an alkaline battery or a lithium battery), a secondary battery (such as a NiCd battery, a NiMH battery, or a Li battery), an AC adapter, etc. The recording medium I / F 18 is an interface with the recording medium 200, such as a memory card or a hard disk. The recording medium 200 is a recording medium (such as a memory card) used to record captured images and is formed from semiconductor memory, a hard disk, etc.

[0103] The communication unit 54 is connected wirelessly or via a wired cable to external devices and sends and receives video and audio signals. The communication unit 54 can also be connected to a wireless LAN (local area network) and the Internet. The communication unit 54 can send images captured by the camera unit 22 (including pass-through images) and images recorded on the recording medium 200, and can receive image data and other various information from external devices.

[0104] The posture detection unit 55 detects the posture of the digital camera 100 in the direction of gravity. Based on the posture detected by the posture detection unit 55, it can be determined whether the image captured by the imaging unit 22 was captured while the digital camera 100 was held in a horizontal or vertical orientation. The system control unit 50 can add orientation information corresponding to the posture detected by the posture detection unit 55 to the image file of the image captured by the imaging unit 22, or record the image while rotating it. An accelerometer, gyroscope, or similar sensor can be used as the posture detection unit 55.

[0105] Note that the operation unit 70 also includes a touch panel 70a capable of detecting touches on the display unit 28. The touch panel 70a and the display unit 28 can be integrated. For example, the touch panel 70a is configured to prevent light transmission from obstructing the display on the display unit 28 and is mounted on the upper layer of the display surface of the display unit 28. Input coordinates on the touch panel 70a are associated with display coordinates on the display unit 28. This can form a GUI (Graphical User Interface) that allows users to directly operate the screen displayed on the display unit 28. The system control unit 50 can detect the following operations or states of the touch panel 70a:

[0106] • A new touch is initiated when a finger or stylus that has not been touched on the touch panel is made on the touch panel 70a. (Hereinafter referred to as “touch”)

[0107] • The user is touching the touch panel 70a with his / her finger or pen (hereinafter referred to as "touch persistence");

[0108] • Move your finger or pen while the user touches the touch panel 70a (hereinafter referred to as “touch movement”);

[0109] • Releasing a finger or stylus from the touch panel 70a ends the touch (hereinafter referred to as "touch stop"); and

[0110] • The touch panel 70a is in a state where nothing is touching it (hereinafter referred to as "not touched").

[0111] Upon detecting a "touch," the system control unit 50 simultaneously detects "continuous touch." After a "touch," the system control unit 50 typically continues to detect "continuous touch" unless a "touch stop" is detected. While detecting "continuous touch," the system control unit 50 also detects "touch movement." Even when the system control unit 50 detects "continuous touch," it will not detect "touch movement" unless the touch location moves. Upon detecting a "touch stop" where all fingers or pens have touched the touch panel, the system control unit 50 detects "no touch."

[0112] Information indicating this operation or state, along with the coordinates of the position of the finger or stylus on the touch panel 70a, is communicated to the system control unit 50 via an internal bus. Based on the communicated information, the system control unit 50 determines the specific operation (touch operation) performed on the touch panel 70a. The system control unit 50 can determine a "touch movement," including the direction of movement of the finger or stylus on the touch panel, based on changes in position coordinates for each vertical and horizontal component on the touch panel 70a. If a "touch movement" of a predetermined distance is detected, the system control unit 50 determines that a swipe operation has been performed. The operation of quickly moving a finger a certain distance while touching the touch panel 70a and then releasing the finger is called a "flick." In other words, a "flick" is an operation where quickly drawing on the touch panel 70a appears as if the finger has flicked the touch panel 70a. When a "touch movement" exceeding a predetermined speed and distance is detected, and a "touch stop" is continuously detected, the system control unit 50 can determine that the user has performed a "flick" (it can be determined that a "flick" was performed after a swipe operation). Furthermore, a touch operation that simultaneously touches multiple parts (e.g., two points) and brings the touch locations closer together is called a "pinch," while a touch operation that moves the touch locations further apart is called a "pinch separate." "Pinch separate" and "pinch" are generally referred to as a pinch operation (or simply pinch). As the touch panel 70a, any of the following types of touch panels can be used: resistive film type, capacitive type, surface acoustic wave type, infrared type, electromagnetic induction type, image recognition type, photoelectric sensor type, etc. Some types are designed to detect touch by detecting contact with the touch panel. Other types are designed to detect touch by detecting the proximity of a finger or stylus to the touch panel. Any of these types can be used.

[0113] Note that this invention is not limited to the camera device itself, but also applies to control devices that communicate with and remotely control camera devices (including webcams) via wired or wireless communication. Examples of devices for remotely controlling camera devices include smartphones, tablet PCs, and desktop PCs. Based on operations or processing performed on the control device side, the control device can remotely control the camera device by notifying it of commands to perform various operations or settings. It can also receive live-view images captured by the camera device via wired or wireless communication and display the received images on the control device side.

[0114] Note that the embodiment has been described by way of applying the invention to a digital camera, but the invention is not limited thereto. For example, the invention is applicable to any device that includes a display unit, such as a PDA, a portable telephone terminal, a portable image viewer, a printer device including a display, a digital photo frame, a music player, a game console, or an e-book reader.

[0115] Figure 3 This diagram illustrates an example of the connection between a digital camera 100 and an external device 300. When the digital camera 100 and the external device 300 are connected via a connection cable 302, the display unit 28 of the digital camera 100 is turned off, and the display 301 of the external device 300 displays the content already displayed on the digital camera 100.

[0116] Figures 4A-1 to 4A-4 This is a flowchart illustrating the LV shooting mode processing of the digital camera 100. This processing is achieved by expanding the program recorded in the non-volatile memory 56 in the system memory 52 and executing the program by the system control unit 50.

[0117] First, the HDR shooting mode and SDR shooting mode according to this embodiment will be explained. In the digital camera 100 of this embodiment, the HDR shooting mode or SDR shooting mode can be set through menu operations or the like from the user. These modes allow the user to set whether the final image data obtained is of HDR quality or SDR quality, and various control operations are performed in the following processing according to the set mode. In the following text, shooting in HDR shooting mode and SDR shooting mode will be referred to as "HDR shooting" and "SDR shooting" in some cases. However, as will be described later, it is also possible to set it to record only in RAW format, so it is not necessary to record an HDR image when shooting in HDR shooting mode.

[0118] In S401, the system control unit 50 determines whether the settings made by the user on the operation unit 70 are for HDR shooting mode. If it is determined that HDR shooting mode is set, the system control unit 50 proceeds the processing to S402, and if it is determined that SDR shooting mode is set, it proceeds the processing to S422.

[0119] In S402, the system control unit 50 determines whether the external device 300 is connected to the digital camera 100. If it is determined that the external device 300 is connected, the system control unit 50 proceeds the process to S403, and if it is determined that the external device 300 is not connected, the process proceeds to S404.

[0120] In S403, the system control unit 50 processes the connection between the digital camera 100 and the external device 300. Then, the system control unit 50 advances the processing to S404. Details of this connection process will be discussed later. Figures 5A to 5C Please describe the process. Note that if the external device supports HDR connectivity, an HDR connection will be established; otherwise, an SDR connection will be established.

[0121] In S404, the system control unit 50 performs HDR quality image processing on the real-time RAW image data captured by the camera unit 22 and converted into digital signals by the A / D converter 23 using the image processing unit 24. The image obtained through HDR quality image processing is called an HDR image.

[0122] Note that the HDR image data in this embodiment is as follows: one pixel is formed by three components (e.g., Luv or YCbCr), and in this embodiment, each component is represented by 10 bits (1024 hues). The gamma curve of the HDR image (e.g., PQ or HLG of ITU-R Recommendation BT.2100) is applied to the HDR image data.

[0123] In S405, the system control unit 50 determines whether the device used to display the LV image (display unit 28 or external device 300) supports HDR. If it is determined that the device does not support HDR, the system control unit 50 proceeds the processing to S406, and if it is determined that the device supports HDR, it proceeds the processing to S409.

[0124] In S406, the system control unit 50 checks the HDR auxiliary display settings. If it is determined that Auxiliary 1 is set, the system control unit 50 proceeds the processing to S407, and if Auxiliary 2 is set, it proceeds the processing to S408. Auxiliary 1 is a setting used to check the high-brightness area of ​​the HDR image and performs processing to assign many hues (code values) to the high-brightness range of the HDR image. Auxiliary 2 is a setting used to check the mid-brightness range of the HDR image and performs processing to assign many hues to the mid-brightness area of ​​the HDR image.

[0125] In S407, the system control unit 50 performs HDR→SDR conversion processing on the HDR image data obtained through the display processing in S404, according to the settings of auxiliary 1. Additionally, the system control unit 50 displays SDR-quality LV image data obtained through size adjustment processing to suit the size of the output target device (display unit 28 or external device 300), and proceeds the processing to S410.

[0126] In S408, the system control unit 50 performs HDR→SDR conversion processing on the HDR image data obtained through the display processing in S404, according to the settings of auxiliary 2. Additionally, the system control unit 50 displays SDR-quality LV image data obtained through size adjustment processing to suit the size of the output target device (display unit 28 or external device 300), and advances the processing to S410.

[0127] The SDR quality image data (SDR image data) in S407 and S408 is image data with 8 bits per component. The gamma curve of the SDR image (e.g., the gamma curve of the sRGB standard) is applied to the SDR quality image data. Note that the gamma curve of the sRGB standard is generally a straight line for dark areas and a 2.4 power curve for bright areas, but it is also possible to use only a 2.2 power curve.

[0128] In S409, the system control unit 50 performs a process to adjust the size of the HDR image data obtained by the display processing in S404 to a size suitable for the output target device (display unit 28 or external device 300), displays the resized HDR quality image (hereinafter referred to as HDL_LV image) as a real-time image, and advances the processing to S410.

[0129] In S410, the system control unit 50 determines whether the menu display button 70e is pressed. If it is determined that the button is pressed, the system control unit 50 proceeds the process to S411; if it is determined that the button is not pressed, it proceeds the process to S412. In S411, the system control unit 50 performs the capture menu processing and proceeds the process to S412. Details of this capture menu processing will be referred to later. Figure 6A-1 and Figure 6A-2 as well as Figure 6B-1 and Figure 6B-2 Describe it.

[0130] In S412, the system control unit 50 determines whether the information display button 70g has been pressed. If it is determined that the button has been pressed, the system control unit 50 proceeds the process to S413; if it is determined that the button has not been pressed, it proceeds the process to S414. In S413, the system control unit 50 switches the display of shooting information and proceeds the process to S414. Examples of shooting information include a histogram and a highlight warning table.

[0131] In S414, the system control unit 50 determines whether the shutter button 61 is half-pressed based on whether signal SW1 is received. If it is determined that the button is not half-pressed, the system control unit 50 proceeds the process to S420, and if it is determined that the button is half-pressed, it proceeds the process to S415.

[0132] In S415, the system control unit 50 makes a reference. Figure 2The AE / AF processing is described, and the process proceeds to S416. In S416, the system control unit 50 determines whether the shutter button 61 is fully pressed based on whether signal SW2 is received. If it is determined that the button is not fully pressed, the system control unit 50 proceeds the process to S417, and if it is determined that the button is fully pressed, the process proceeds to S418. In S417, the system control unit 50 determines whether the half-pressed state of the shutter button 61 is maintained. If the half-pressed state is maintained, the system control unit 50 returns the process to S415, and if it is determined that the half-pressed state is not maintained, the process proceeds to S420. In S418, the system control unit 50 performs HDR shooting processing and records an image data file corresponding to a preset recording format on the recording medium. Figure 8A The data structure of the file to be recorded is shown. Then, the system control unit 50 advances the processing to S419. Note that this will be referred to later. Figure 7A and Figure 7B The details of this HDR shooting process are described below. In S419, the system control unit 50 performs fast review display processing and advances the processing to S420. Details of this fast review display processing will be discussed later. Figure 4B-1 and Figure 4B-2 Describe it.

[0133] In S420, the system control unit 50 determines whether the LV button 76 is pressed. If it is determined that the button is pressed, the system control unit 50 advances the process to S421, and if it is determined that the button is not pressed, it advances the process to S422.

[0134] In S421, the system control unit 50 compresses the image data (one pixel has three components, and each component has 10 bits of image data) that was displayed as HDR image quality in S404 using HEVC (H.265) compression, records the compressed data as an HDR motion picture file, and advances the processing to S438.

[0135] In S422, the system control unit 50 determines whether the external device 300 is connected to the digital camera 100. If it is determined that the external device 300 is connected, the system control unit 50 proceeds the process to S423; if it is determined that the external device 300 is not connected, it proceeds the process to S424. In S423, the system control unit 50 performs the process of connecting the digital camera 100 and the external device 300, and then proceeds the process to S424. Details of this connection process will be referred to later. Figures 5A to 5C Please note that because SDR shooting mode is set, external devices connect via SDR connection.

[0136] In S424, the system control unit 50 instructs the image processing unit 24 to display the image captured by the camera unit 22 and converted into a digital signal by the A / D converter 23 as an SDR image of quality (one pixel has three components, and each component has 8 bits (256 tones)), and proceeds the processing to S425. Note that in the following text, the SDR quality image will be referred to as an SDR image.

[0137] In S425, the system control unit 50 generates an SDR-quality real-time image (SDR_LV image) by performing a process that adjusts the size of the SDR image obtained by the display processing in S424 to a size suitable for the resolution of the output destination device (display unit 28 or external device 300), and displays the generated SDR_LV image.

[0138] In S426, the system control unit 50 determines whether the menu display button 70e is pressed. If it is determined that the button is pressed, the system control unit 50 proceeds the process to S427; if it is determined that the button is not pressed, it proceeds the process to S428. In S427, the system control unit 50 performs the capture menu processing and proceeds the process to S428. Details of this capture menu processing in S427 will be referred to later. Figure 6A-1 and Figure 6A-2 as well as Figure 6B-1 and Figure 6B-2 Describe it.

[0139] In S428, the system control unit 50 determines whether the information display button 70g has been pressed. If it is determined that the button has been pressed, the system control unit 50 proceeds the process to S429; if it is determined that the button has not been pressed, the process proceeds to S430. In S429, the system control unit 50 switches the display of shooting information and proceeds the process to S430. Examples of shooting information include a histogram and a highlight warning meter.

[0140] In S430, the system control unit 50 determines whether the shutter button 61 is in a half-pressed state. If it is determined that the button is not in a half-pressed state, the system control unit 50 advances the processing to S436, and if it is determined that the button is in a half-pressed state, it advances the processing to S431.

[0141] In S431, the system control unit 50 makes a reference. Figure 2The AF / AE processing is described, and the process proceeds to S432. In S432, the system control unit 50 determines whether the shutter button 61 is fully pressed based on whether signal SW2 is received. If it is determined that the button is not fully pressed, the system control unit 50 proceeds the process to S433, and if it is determined that the button is fully pressed, the process proceeds to S434. In S433, the system control unit 50 determines whether to maintain the half-pressed state of the shutter button 61 based on whether signal SW1 is received. If it is determined that the half-pressed state is maintained, the system control unit 50 returns the process to S431, and if it is determined that the half-pressed state is not maintained, the process proceeds to S436.

[0142] In S434, the system control unit 50 performs SDR image processing and advances the processing to S435. In this SDR image processing, the system control unit 50 develops the RAW image data obtained through SDR shooting at SDR image quality, generates JPEG image data by encoding the SDR quality image in JPEG format, and records this data as a JPEG file on the recording medium. If the recording setting is configured to record only SDR images as JPEG files, then only JPEG files are recorded. If the recording setting is configured to record both JPEG files and RAW image files, then JPEG files are recorded, and both the data obtained by encoding the RAW image data obtained through SDR shooting and the JPEG image data are recorded on the recording medium as a JPEG file. Figure 8A The image shown is a RAW image file in RAW image file format. In this RAW image file, Figure 8A The ImageData 809 in the data structure shown has Figure 8B The format shown is as follows: That is, the images of each size used for display are encoded in 8-bit precision using JPEG, and the encoded data is integrated and stored as a single file. Then, in S435, the system control unit 50 performs fast playback display processing and advances the processing to S436. Details of this fast playback display processing will be discussed later. Figure 4B-1 and Figure 4B-2 Describe it.

[0143] In S436, the system control unit 50 determines whether the LV button 76 is pressed. If it is determined that the button is pressed, the system control unit 50 proceeds the process to S437; if it is determined that the button is not pressed, the process proceeds to S438. In S437, the system control unit 50 compresses the SDR image obtained through the SDR quality imaging process in S425 using H264 compression, records the compressed image as an SDR motion picture file, and proceeds the process to S438.

[0144] In S438, the system control unit 50 determines whether the playback button 79 has been pressed. If it is determined that the playback button 79 has been pressed, the system control unit 50 advances the process to S439; if it is determined that the playback button 79 has not been pressed, the process advances to S440. In S439, the system control unit 50 performs playback mode processing and advances the process to S440. Details of this playback mode processing will be discussed later. Figures 9A to 9H and Figure 10A and Figure 10B Describe it.

[0145] In S440, the system control unit 50 determines whether an LV mode termination command exists. If it is determined that no LV mode termination command exists, the system control unit 50 returns the processing to S401; if it is determined that a termination command exists, the processing is terminated.

[0146] Figure 4B-1 and Figure 4B-2 This is a flowchart illustrating the fast-rewind display process of the system control unit 50. This process is achieved by expanding the program recorded in the non-volatile memory 56 in the system memory 52 and executing the program by the system control unit 50.

[0147] In S451, the system control unit 50 determines whether a fast playback display is set. If it is determined that a fast playback display is set, the system control unit 50 advances the process to S452; otherwise, it terminates the process.

[0148] In S452, the system control unit 50 determines whether to shoot in HDR shooting mode. If it is determined that shooting is in HDR shooting mode, the system control unit 50 causes the processing to proceed to S453, and if it is determined that shooting is in SDR shooting mode, the processing proceeds to S460.

[0149] In S453, the system control unit 50 determines whether the device used for fast playback display (display unit 28 or external device 300) supports HDR. If it is determined that the device does not support HDR, the system control unit 50 causes the processing to proceed to S454, and if it is determined that the device supports HDR, it causes the processing to proceed to S457.

[0150] In S454, the system control unit 50 determines whether to capture images using RAW still images. If it is determined that the image will be captured using RAW still images, the system control unit 50 initiates processing in S455; and if it is determined that the image will be captured using HEIF still images, the processing proceeds to S456.

[0151] In S455, the system control unit 50 performs HDR→SDR conversion on the HDR image 828 for display in the HDR RAW image through the same process as in S406 to S408, adjusts the image size to be suitable for the output target device (display unit 28 or external device 300), displays the image with SDR image quality, and then proceeds to S463.

[0152] In S456, the system control unit 50 performs HDR→SDR conversion on the HDR image for display in the HEIF image through the same process as in S406 to S408, adjusts the image size to be suitable for the output target device (display unit 28 or external device 300), displays the image with SDR image quality, and then proceeds to S463.

[0153] In S457, the system control unit 50 determines whether shooting is performed via RAW still image capture. If it is determined that shooting is performed via RAW still image capture, the system control unit 50 proceeds to S458; and if it is determined that shooting is performed via HEIF still image capture, it proceeds to S459. In S458, the system control unit 50 adjusts the size of the HDR image 828 in the HDR RAW image for display to a size suitable for the output target device (display unit 28 or external device 300), displays the image with HDR image quality, and proceeds to S463. In S459, the system control unit 50 adjusts the size of the HDR image in the HEIF image for display to a size suitable for the output target device (display unit 28 or external device 300), displays the image with HDR image quality, and proceeds to S463.

[0154] In S460, the external device 300 determines whether shooting is performed via RAW still image capture. If it is determined that shooting is performed via RAW still image capture, the external device 300 proceeds the processing to S461; and if it is determined that shooting is performed via HEIF still image capture, the processing proceeds to S462. In S461, the system control unit 50 adjusts the size of the SDR image 823 for display in the SDR RAW image to fit the size of the output target device (display unit 28 or external device 300), displays the image with SDR image quality, and proceeds the processing to S463. In S462, the system control unit 50 adjusts the size of the SDR image for display in the JPEG image to fit the size of the output target device (display unit 28 or external device 300), displays the image with SDR image quality, and proceeds the processing to S463.

[0155] In S463, the system control unit 50 determines whether the shutter button 61 has been pressed. If it is determined that the button has not been pressed, the system control unit 50 causes the process to proceed to S464, and if it is determined that the button has been pressed, the process terminates.

[0156] In S464, the system control unit 50 determines whether the time set for the fast replay display has elapsed. If it is determined that the time has not yet elapsed, the system control unit 50 returns the process to S463; if it is determined that the time has elapsed, the process is terminated.

[0157] Figure 5A This is a sequence diagram illustrating the control process of the digital camera 100 and the external device 300 when they are connected. The explanation will assume that the digital camera 100 and the external device 300 are connected via an HDMI connection.

[0158] In S501, the system control unit 50 instructs the digital output I / F 90 to start sending a +5V signal. As a result, the digital output I / F 90 starts sending a +5V signal. The sent +5V signal is transmitted to the external device 300 via the +5V signal line (not shown) of the connection cable 302. The external device 300 receives the +5V signal from the connection cable 302 and proceeds to S502.

[0159] In S502, the external device 300 determines that the digital camera 100 has confirmed the connection of the external device 300 and proceeds the process to S503.

[0160] In S503, the external device 300 begins transmitting an HPD signal from the HPD signal line (not shown) of the connection cable 302. The digital output I / F 90 of the digital camera 100 receives the transmitted HPD signal via the connection cable 302. Upon receiving the HPD signal, the digital output I / F 90 notifies the system control unit 50 of the HPD reception.

[0161] In S504, the system control unit 50 detects the connection response from the external device 300 via the notification of HPD and proceeds the processing to S505.

[0162] In S505, the system control unit 50 controls the digital output I / F 90 to send an EDID request signal from the connection cable 302. The sent EDID request signal is transmitted to the external device 300 via the EDID signal line (not shown) of the connection cable 302. The external device 300 receives the EDID request signal and proceeds to S506.

[0163] In S506, the external device 300 transmits an EDID signal from the EDID signal line (not shown) of the connection cable 302. The digital output I / F 90 of the digital camera 100 receives the EDID via the connection cable 302. Upon receiving the EDID, the digital output I / F 90 notifies the system control unit 50 of the reception of the EDID.

[0164] In S507, the system control unit 50 receives a notification of EDID reception and instructs the digital output I / F 90 to copy the EDID received in S506 into the memory 32. After the copying is completed, the system control unit 50 analyzes the expanded EDID in the memory 32, determines the performance of the video signal acceptable to the external device 300, and initiates processing in S508.

[0165] In S508, if the main setting is HDR enabled and the performance of the video signal acceptable to the external device 300 as determined in S507 corresponds to the HDR signal, the system control unit 50 determines to output the HDR signal to the external device 300; otherwise, it determines to output the SDR signal and proceeds to S509.

[0166] In S509, the system control unit 50 instructs the digital output I / F 90 to begin transmitting the HDR or SDR video signal determined in S508. Having received the video signal transmission start command, the digital output I / F 90 begins transmitting the video signal via the connection cable 302, and the processing proceeds to S510.

[0167] In S510, the digital camera 100 outputs a video signal to the TMDS signal line (not shown) of the connecting cable 302. The external device 300 receives the video signal via the TMDS signal line (not shown) of the connecting cable 302 and advances the processing to S511.

[0168] In S511, the external device 300 analyzes the video signal received in S510, switches the drive condition of the display 301 to a setting that enables the display of the video signal, and advances the processing to S512. In S512, the external device 300 displays the video signal received in S510 on the display 301 of the external device 300.

[0169] Figure 5B This is a sequence diagram illustrating the process of switching the video output of digital camera 100 and external device 300 from SDR images to HDR images.

[0170] In this sequence, assuming at the reference Figure 5A The connection between the digital camera 100 and the external device 300 is completed in the described sequence.

[0171] In S521, the system control unit 50 instructs the digital output I / F 90 to send an SDR video signal and advances the processing to S522.

[0172] In S522, the digital camera 100 outputs an SDR video signal to the TMDS signal line (not shown) of the connecting cable 302. The external device 300 receives the SDR video signal via the TMDS signal line (not shown) of the connecting cable 302 and initiates processing in S523.

[0173] In S523, the external device 300 displays the SDR video received in S522 on the display 301 of the external device 300.

[0174] When the digital camera 100 is outputting an SDR signal, the display 301 of the external device 300 displays the SDR image by repeating steps S521 to S523.

[0175] When the digital camera 100 switches the video output to the external device 300 from an SDR image to an HDR image, processing starting from S524 is performed.

[0176] In S524, the system control unit 50 instructs the digital output I / F 90 to stop the SDR video signal and advances the processing to S525.

[0177] In S525, the system control unit 50 stops sending video signals to the TMDS signal line (not shown) of the connection cable 302. The external device 300 stops receiving SDR video signals via the TMDS signal line (not shown) of the connection cable 302 and proceeds to S526.

[0178] In S526, because the reception of video from digital camera 100 has stopped, external device 300 stops displaying video on display 301 of external device 300.

[0179] In S527, the system control unit 50 instructs the digital output I / F 90 to send an HDR video signal and advances the processing to S528.

[0180] In S528, the system control unit 50 outputs the HDR video signal to the TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the HDR video signal via the TMDS signal line (not shown) of the connection cable 302 and advances the processing to S529.

[0181] In S529, the external device 300 analyzes the video signal received in S528, switches the driving conditions of the display 301 to a setting that enables the display of HDR video signals, and advances the processing to S530.

[0182] In S530, the external device 300 displays the HDR video signal received in S528 on the display 301 of the external device 300.

[0183] The processing time from S529 to S530 varies depending on the performance of the external device 300, so it takes approximately 1 to 5 seconds before the video is displayed.

[0184] Figure 5C This is a sequence diagram illustrating the process of switching the video output of digital camera 100 and external device 300 from HDR images to SDR images.

[0185] In this sequence, assuming at the reference Figure 5A The connection between the digital camera 100 and the external device 300 is completed in the described sequence.

[0186] In S541, the system control unit 50 instructs the digital output I / F 90 to send an HDR video signal and advances the processing to S542. In S542, the system control unit 50 outputs the HDR video signal to the TMDS signal line (not shown) of the connection cable 302. Furthermore, the external device 300 receives the HDR video signal via the TMDS signal line (not shown) of the connection cable 302 and advances the processing to S543.

[0187] In S543, the external device 300 displays the HDR video received in S542 on the display 301 of the external device 300.

[0188] When the digital camera 100 is outputting an HDR signal, the display 301 of the external device 300 displays the HDR signal by repeating steps S541 to S543.

[0189] When the digital camera 100 switches the video output to the external device 300 from an HDR image to an SDR image, processing starting from S544 is performed.

[0190] In S544, the system control unit 50 instructs the digital output I / F 90 to stop the HDR video signal and advances the processing to S545. In S545, the system control unit 50 stops the video signal to the TMDS signal line (not shown) of the connection cable 302. The external device 300 stops receiving the HDR video signal via the TMDS signal line (not shown) of the connection cable 302 and advances the processing to S546.

[0191] In S546, because the reception of video from digital camera 100 has stopped, external device 300 stops displaying video on display 301 of external device 300.

[0192] In S547, the system control unit 50 instructs the digital output I / F 90 to send an SDR video signal and advances the processing to S548.

[0193] In S548, the system control unit 50 outputs the SDR video signal to the TMDS signal line (not shown) of the connection cable 302. The external device 300 receives the SDR video signal via the TMDS signal line (not shown) of the connection cable 302 and advances the processing to S549.

[0194] In S549, external device 300 analyzes the video signal received in S548, switches the drive condition of display 301 to a setting that enables the display of SDR video signals, and advances the processing to S550. In S550, external device 300 displays the SDR video signal received in S528 on display 301 of external device 300.

[0195] Note that the processing time from S549 to S550 varies depending on the performance of the external device 300, so it takes approximately 1 to 5 seconds before the video is displayed.

[0196] Figure 6A-1 and Figure 6A-2 as well as Figure 6B-1 and Figure 6B-2 It is shown Figures 4A-1 to 4A-4 The flowcharts detail the shooting menu processing in S411 and S427. This processing is achieved by expanding the program recorded in the non-volatile memory 56 in the system memory 52 and executing the program by the system control unit 50.

[0197] In S601, the system control unit 50 determines whether to perform HDR shooting based on whether the user has enabled HDR shooting mode. If it is determined that HDR shooting will not be performed, the system control unit 50 advances the processing to S602 and displays the menu for normal SDR shooting. If it is determined that HDR shooting will be performed, the system control unit 50 advances the processing to S603 and displays the menu for HDR shooting. In S603, the system control unit 50 displays the functions not used in HDR shooting in an invalid state, such as a grayed-out state, on the menu.

[0198] In S604, the system control unit 50 determines whether the user has selected a setting indicating whether to perform HDR shooting. If the setting is selected, the system control unit 50 proceeds to S605; otherwise, it proceeds to S611. In S605, the system control unit 50 determines whether the user has enabled the setting indicating whether to perform HDR shooting. If the setting is enabled, the system control unit 50 proceeds to S606; otherwise, it proceeds to S607. In S606, the system control unit 50 changes the setting indicating whether to perform HDR shooting to enabled and stores the setting value in the system memory 52.

[0199] If the setting indicating whether to perform HDR shooting is "enabled," the system control unit 50 determines in S607 whether the user has switched the HDR auxiliary display setting to "change." If it is determined that the setting has been switched to "change," the system control unit 50 proceeds the processing to S608; otherwise, it proceeds the processing to S609. Note that if the setting indicating whether to perform HDR shooting is "disabled," it is expected that the HDR auxiliary display setting cannot be changed.

[0200] In S608, the system control unit 50 changes the HDR assist display setting used for shooting to "perform" or "not perform", and stores the setting value in the system memory 52. ​​The "perform" HDR assist setting can have two or more variations.

[0201] Please note that when changing HDR shooting settings or HDR auxiliary display settings on the menu screen, the change in display settings will also be reflected on the monitor at the time the menu screen changes to the live view screen. When these settings are changed not on the menu screen but on the live view screen by, for example, using a specific button on the operation unit 70, the change will also be reflected at the time the button is pressed.

[0202] In S609, the system control unit 50 determines whether the user has issued a termination command for the HDR settings menu display processing. If it is determined that a termination command has been issued, the system control unit 50 proceeds the processing to S610.

[0203] In S610, the system control unit 50 determines whether the user has selected a setting for still image recording image quality. If a selection is confirmed, the system control unit 50 proceeds the process to S611; otherwise, it proceeds to S651.

[0204] In S611, the system control unit 50 determines whether the user has entered an HDR shooting command. If it is determined that a command input for HDR shooting has been issued, the system control unit 50 proceeds the processing to S612, and if it is determined that no command input has been issued, it proceeds the processing to S614.

[0205] The system control unit 50 displays the image for HDR shooting in S612 and accepts user selection of the recorded image quality for HDR shooting in S613. As the set recorded image quality for HDR shooting, two images can be simultaneously output as file formats: RAW, HDR still image file, and RAW+HDR still image file. Examples of image sizes are Large (close to the number of pixels read by the sensor), Middle (slightly smaller than Large), and Small (even smaller than Large). Examples of compression ratios for compressing file size are High image quality (low compression ratio), Standard (high compression ratio), and Low image quality (high compression ratio).

[0206] The system control unit 50 displays the image for SDR shooting in S614 and accepts user selection of the recorded image quality for SDR shooting in S615. The same options as those for HDR shooting are also prepared as settings for the recorded image quality for SDR shooting.

[0207] In S651, the system control unit 50 determines whether the user has selected the setting item for motion picture recording image quality. If it is determined that the setting item for motion picture recording image quality has been selected, the system control unit 50 proceeds the process to S652; otherwise, it proceeds the process to S657.

[0208] In S652, the system control unit 50 checks whether the setting for HDR shooting has been enabled by the user. If it is set to "yes", the system control unit 50 proceeds to S653; otherwise, it proceeds to S655.

[0209] The system control unit 50 displays the image for HDR shooting in S653 and accepts user selection of the recorded image quality for HDR shooting in S654. As set for the recorded image quality for HDR shooting, three motion images are prepared for simultaneous output: RAW motion image, RAW motion image + proxy motion image, HDR motion image file, and RAW + proxy motion image + HDR motion image file. Examples of image sizes are 8K, 4K, FullHD, HD, and VGA. Examples of compression ratios for reducing file size are high image quality (low compression ratio) such as ALL-I and standard to low image quality (high compression ratio) such as IPB. Frame rate and broadcast system such as NTSC / PAL can also be selected.

[0210] The system control unit 50 displays the image for SDR shooting in S655 in the same manner as in S653, and accepts user selection of the recorded image quality for SDR shooting in S656. Similarly, the same options as those for HDR shooting are prepared as settings for the recorded image quality for SDR shooting.

[0211] In S657, the system control unit 50 determines whether the user has selected the HDR output setting. If it is determined that the user has selected the HDR output setting, the system control unit 50 proceeds the process to S658; otherwise, it proceeds to S660. In S658, the system control unit 50 determines whether the user has switched the HDR output setting to "enabled". If it is determined that the setting has been switched to "enabled", the system control unit 50 proceeds the process to S659; otherwise, it proceeds to S660. In S659, the system control unit 50 changes the HDR output setting to "enabled" and stores the setting value in the system memory 52.

[0212] In S660, the system control unit 50 determines whether the user has selected a playback viewing assistance setting. If it is determined that the playback viewing assistance setting has been selected, the system control unit 50 proceeds the process to S661; otherwise, it proceeds to S663. In S661, the system control unit 50 determines whether the user has switched the playback viewing assistance setting to "enabled". If it is determined that the setting has been switched to "enabled", the system control unit 50 proceeds the process to S662; otherwise, it proceeds to S663. In S662, the system control unit 50 changes the setting for playback viewing assistance to "enabled" and stores the setting value in the system memory 52.

[0213] In S663, the system control unit 50 determines whether the user has selected a setting item for SDR conversion used for transmission. If it is determined that this selection has been made, the system control unit 50 proceeds the process to S664; otherwise, it proceeds the process to S665. In S663, the system control unit 50 determines whether the user has switched the setting for SDR conversion used for transmission to "enabled". If it is determined that the setting has been switched to "enabled", the system control unit 50 changes the SDR conversion setting for transmission to "enabled" in S664 and proceeds the process to S665.

[0214] In S665, the system control unit 50 determines whether the user has selected any settings related to other HDR shooting. If a selection is confirmed, the system control unit 50 proceeds the processing to S666; otherwise, it proceeds to S667. In S666, the system control unit 50 changes the other processing to "enabled" and proceeds the processing to S667.

[0215] In S667, the system control unit 50 determines whether the user has indicated that they will leave the menu. If it is determined that they will not leave, the system control unit 50 returns the processing to S660, and if it is determined that they will leave, the processing is terminated.

[0216] Figure 7A and Figure 7B This is a flowchart illustrating the details of the HDR shooting process in the system control unit 50. In this process, RAW data written to memory 32 is processed by HDR imaging in image processing unit 24.

[0217] Video recording devices such as digital cameras or digital camcorders have a white balance function that corrects the tone of the captured image based on the light source used in the shooting. The white balance function corrects for differences in tone between light sources (such as natural light sources like clear skies and cloudy skies, and artificial light sources like fluorescent lamps and incandescent lamps), ensuring that white appears uniform regardless of the light source. In steps S701 to S703, the white balance coefficient required for white balance processing is calculated. In this embodiment, to prevent the loss of highlight details in bright areas such as the sky, it is assumed that the shot is taken with an exposure lower than the appropriate brightness for the subject, etc.

[0218] First, in S701, the system control unit 50 obtains RAW image data via the memory control unit 15.

[0219] In S702, the system control unit 50 performs a white search box determination process on the acquired RAW image data to determine pixels that appear white, in order to calculate the white balance coefficient.

[0220] In S703, the system control unit 50 calculates the white balance coefficient based on the result determined within the white search box.

[0221] Reference Figure 12 The flowchart shown illustrates the details of the processing in S702 and S703.

[0222] As mentioned earlier, each pixel in the RAW image data has only one of the signal components R, G, and B. To perform a white search, the system control unit 50 performs a debayering process S1201 (because a color signal conversion is required), thereby generating all channels of R, G, and B signals for each pixel. There are various methods for debayering, and signal generation can be performed, for example, by using linear interpolation with a low-pass filter. RAW image data is generally affected by noise, so optical black is not 0 but has a value. Therefore, the system control unit 50 performs a process of subtracting the OB value from the debayered signal (S1202). Then, the system control unit 50 calculates the color signal C based on the obtained RGB signal using the following equation. x and C y (S1203).

[0223]

[0224]

[0225]

[0226] G1 and G2 are the two G component values ​​in a 2×2 pixel array of the Bayer array. x Represents color temperature, C y Represents the green direction correction amount, and Y i It is the brightness value.

[0227] Figure 13 C is shown x -C y Plane. For example... Figure 13 As shown, white can be pre-captured using a camera device at various color temperatures, from high color temperature (e.g., during the day) to low color temperature (e.g., at sunset), and the color evaluation value C can be plotted on a coordinate system. x and C y The system control unit 50 obtains a white axis 1200 as a reference for detecting white. Since the actual light source has slight variations in whiteness, the system control unit 50 imparts a certain degree of width to both sides of the white axis 1200 (S1204). The box obtained by widening the white axis in this way is called the white search box 1201.

[0228] In S1205, the system control unit 50 is in C x-C y The pixels after Bayer calibration are plotted in the coordinate system, and it is determined whether these pixels exist in the white search box. In S1206, the system control unit 50 performs a luminance exclusion process to limit the pixels that are the objects of integration in the luminance direction among the pixels present in the white search box. This process is performed to prevent the accuracy of the white balance coefficient calculation from being reduced due to the susceptibility of too dark colors to noise. Similarly, this process is performed to prevent the accuracy of the white balance coefficient calculation from being reduced due to the fact that too bright colors may disrupt the R / G ratio or B / G ratio balance due to sensor saturation in one of the channels, and separate from the correct color. The luminance values ​​of the pixels that are the objects of the luminance exclusion process differ between SDR and HDR. That is, the pixels to be used in the white balance coefficient calculation (described later) differ between SDR and HDR. This is because HDR has higher reproducibility in the high-brightness areas compared to SDR. In this embodiment, in SDR, for example, luminance up to +1EV is the object on the bright side, while in HDR, luminance up to +2EV is the object. This makes it possible to calculate a white balance coefficient optimized for HDR.

[0229] In S1207, the system control unit 50 determines the C based on the information present in the white search box and after brightness and darkness elimination processing. x and C y The integral values ​​SumR, SumG, and SumB of the color evaluation value are calculated. Then, in S1208, the system control unit 50 calculates the white balance coefficient WBCo based on the calculated integral values ​​using the following equation. R WBCo G and WBCo B .

[0230]

[0231]

[0232]

[0233]

[0234] The “1024” on the right side of each equation indicates that the precision of a color component is 10 bits.

[0235] Note that the white balance factor can be calculated for the shooting mode (SDR or HDR shooting) set by the user, or for both SDR and HDR.

[0236] The instructions will return to Figure 7AIn S704 to S706, the system control unit 50 calculates the tone correction table required for tone correction processing. Details of tone correction will be referred to... Figure 14A and Figure 14B The flowchart shown is used for illustration.

[0237] In S1221, the system control unit 50 uses a method consisting of... Figure 7A The WB coefficients generated in S701 to S703 are used for WB processing. In S1222, the system control unit 50 performs histogram detection. More specifically, the system control unit 50 applies the white balance gain value obtained in S1221 to the entire image data and generates a histogram as brightness information for the pixel values ​​that have undergone gamma correction processing. The gamma correction processing can be performed using a known lookup table method, but it is preferable to use the same gamma characteristics as those used in the display. However, simplified gamma characteristics, such as those utilizing line approximation gamma characteristics, can also be used to save processing time and storage. Note that the portions at the edges of the image are often not important in many cases and are also affected by the reduction in edge illumination depending on the camera lens; therefore, the histogram can also be formed by excluding pixels in the peripheral portions.

[0238] In S1223, the system control unit 50 performs face detection preprocessing. This preprocessing makes faces easier to detect by performing image data reduction, gamma processing, etc. In S1224, the system control unit 50 performs face detection processing on the preprocessed image data using known methods. In this face detection processing, the position and size of the face-like region (face region) and the reliability of the detection can be obtained.

[0239] In S1225, the system control unit 50 calculates a tone correction amount (tone correction amount (A)) as a first tone correction amount to compensate for the exposure correction amount (reduction amount). In this step, the system control unit 50 calculates a tone correction amount having input / output characteristics such that the dark areas of the image are properly exposed and high-brightness pixels with a predetermined brightness level or higher are not corrected (at least not fully compensated for the exposure correction amount). This allows for further suppression of highlight detail loss in the tone-corrected bright areas. This tone correction amount can be prepared as multiple correction tables corresponding to the exposure correction amount.

[0240] In S1226, if a facial region with a reliability higher than a preset evaluation threshold exists among the facial regions detected by the facial detection process in S1224, the system control unit 50 determines that a face has been detected. If a face is determined to be detected, the system control unit 50 proceeds the process to S1227, and if a face is determined not to be detected, the process proceeds to S1231.

[0241] In S1227, the system control unit 50 calculates a portion of the detected facial region as a facial brightness acquisition region. The facial brightness acquisition region is a region used to acquire the brightness of the bright parts of the face, and there are no particular restrictions on the number or location of this region. In S1228, the system control unit 50 calculates the average values ​​of each type of R, G, and B pixels contained in each facial brightness acquisition region. Furthermore, the system control unit 50 performs gamma correction on the average values ​​of the R, G, and B pixels by applying a white balance gain value in the same manner as histogram detection, and converts the result into a brightness value Y using the following equation.

[0242] Y=0.299×R+0.587×G+0.114×B

[0243] Note that the white balance gain value to be applied in histogram detection and face detection is preferably the gain value used in WB processing for the same image data. Ideally, the same luminance gamma as used in the imaging is also preferred, but simplified gamma characteristics, such as those using line approximations, can also be used to save processing time and storage.

[0244] In S1229, the system control unit 50 converts the brightness values ​​obtained in S1228 for each facial brightness acquisition area into values ​​corresponding to appropriate exposure. This is a process for correcting facial brightness, where, since the image data was captured with an exposure lower than appropriate, it is detected that the facial brightness is lower than the brightness when the image was captured with appropriate exposure. The brightness value conversion can be performed to compensate for the exposure correction amount (reduction amount) determined by the exposure control, or the brightness value conversion can be performed using the tone correction amount calculated in S1225.

[0245] In S1230, the system control unit 50 calculates a typical value for the detected facial brightness. For example, a typical value can be obtained by acquiring the maximum brightness value of each detected facial region within that region.

[0246] When the system control unit 50 determines in S1226 that no facial region has been detected, the processing in S1231 is performed. In S1231, the system control unit 50 detects a histogram feature value. The histogram feature value may be, for example, the class (SD) to which pixels with a cumulative frequency of 1% from the dark side belong, or the class (HL) to which pixels with a cumulative frequency of 1% from the bright side belong. Subsequently, in S1232, the system control unit 50 converts the histogram feature value calculated in S1231 into a value corresponding to the image captured with proper exposure. This is a process for correcting the histogram feature value, wherein, since the image data was captured with an exposure lower than proper exposure, it is detected that the histogram feature value is lower than the histogram feature value when the image was captured with proper exposure. The brightness value conversion can be performed to compensate for the exposure correction amount (reduction amount) determined by the exposure control, or the brightness value conversion can be performed using the tone correction amount calculated in S1225.

[0247] In S1233, the system control unit 50 calculates the target correction amount. The system control unit 50 calculates the typical brightness value of the face or the target brightness level relative to histogram features. Then, the system control unit 50 generates a lookup table (input / output characteristic) defining the output brightness level relative to the input brightness level, as the tone correction amount (B), using spline interpolation based on the target brightness level and the minimum and maximum brightness values ​​in the image data. The tone correction amount (B) is the second tone correction amount.

[0248] The target tone correction amount for HDR can differ from that for SDR. For example, Figure 16A The appearance of the SDR is shown, and Figure 16B This demonstrates what HDR looks like. Although the subject (person) has the same brightness value, the background is at most 100 dc / m² in SDR. 2 However, it exceeds 100 cd / m² in HDR. 2 Therefore, even if the subject's brightness value is the same, the subject sometimes appears darker in HDR. This is called brightness contrast and is a phenomenon caused by the characteristics of human vision. For example, the subject's brightness value at... Figure 16A and Figure 16B The middle is the same, but with Figure 16A In comparison, Figure 16B In this case, the difference between the subject's brightness and the background brightness is even greater. The user perceives... Figure 16B The subject ratio in Figure 16A The subject appears darker in HDR. In other words, HDR can render bright areas such as the sky as brighter, which increases the likelihood that the subject will appear darker than in SDR. Therefore, in this embodiment, in the case of SDR, a method such as... Figure 15AThe tonal characteristics shown. However, in the case of HDR, by using, as Figure 15B The tonal characteristics shown are used to apply tonal correction to enhance the shadows, thus achieving a good appearance. Note that the tonal correction in this embodiment is illustrated using underexposure compensation as an example. However, for image production purposes, similar tonal correction can be performed during luminance correction.

[0249] The target brightness level can be set to a fixed value that may be empirically advantageous, relative to the typical brightness value of the face and the histogram feature value of the image data. However, different target brightness levels can also be set based on the values ​​of the typical brightness value and the histogram feature value. In this case, a lookup table defining the relationship between the input level and the target brightness level can be prepared for each parameter (typical brightness value and histogram feature value) for which the target brightness level is set.

[0250] The correction characteristics used to achieve the conversion to the target brightness level defined above are obtained by methods such as spline interpolation, and if necessary, the correction characteristics are saved as a lookup table (or relational expression) with the applied hue correction amount (B).

[0251] In S1234, the system control unit 50 synthesizes the hue correction amount (A) calculated in S1225 and the hue correction amount (B) calculated in S1233. For example, the system control unit 50 first applies the hue correction amount (A) to each input brightness level, and then applies the hue correction amount (B) to the corrected brightness level to obtain the result brightness value and form a lookup table of output brightness values ​​relative to each input brightness level.

[0252] In S1235, the system control unit 50 performs a process (limiter processing) to limit the upper limit of the composite correction amount (composite tone correction amount) obtained in S1234. The composite tone correction amount (A) and tone correction amount (B) increase the correction amount and significantly increase the noise in the corrected image, thus limiting the overall correction amount. This limiter processing can be implemented by preparing the maximum allowable correction amount for each brightness value in the form of a table, and replacing the output level in the lookup table formed in S1234 that exceeds the maximum correction amount with the output level corresponding to the maximum correction amount. Note that the tone correction amount can be calculated as a value for the shooting mode (SDR shooting or HDR shooting) set by the user, or a value for both SDR and HDR.

[0253] The instructions will return to Figure 7A and Figure 7BIn S707, the system control unit 50 performs imaging using calculated white balance coefficients, tone correction parameters, and various HDR parameters. HDR images are generated using, for example, color matrices, camera OETF curve data, color adjustment parameters, noise reduction parameters, and sharpness parameters as other imaging parameters. As an example of camera OETF (gamma curve), the inverse characteristic of the EOTF (electro-optical transfer function) of the PQ (perceptual quantization) of ITU-R recommendation BT.2100 is assumed. However, the OOTF (optical-optical transfer function) can also be combined as the camera-side experience. Alternatively, the HLG (hybrid log-gamma) OETF of the same ITU-R recommendation BT.2100 can be used.

[0254] In S708, the system control unit 50 generates an MPF ​​(Multiple Picture Format) image (such as a two-screen comparison image) for simple display by adjusting the size of the image displayed in S707, and compresses and encodes the generated image using HEVC format.

[0255] In S709, the system control unit 50 further adjusts the size of the MPF image generated in S708, thereby generating a thumbnail image with a smaller number of pixels than the MPF image for index display, etc., and compresses the generated image.

[0256] In S710, system control unit 50 compresses the HDR image displayed in S707 into the main image. Various methods can be used for this compression. For example, 10-bit YUV422 data can be compressed using H.265 (ISO / IEC 23008-2 HEVC).

[0257] In S711, the system control unit 50 determines the recorded image quality set by the user. If it is determined that the setting is to record only RAW images, the system control unit 50 causes the processing to proceed to S712; if it is determined that the setting is to record only HDR images, the system control unit 50 causes the processing to proceed to S713; and if it is determined that the setting is to record both RAW and HDR images, the system control unit 50 causes the processing to proceed to S714.

[0258] In S712, the system control unit 50 generates a file with a header by compressing the RAW image and adding a header. Figure 8AThe RAW image file with the structure shown is recorded on the recording medium 200 via the recording medium I / F 18. Various compression methods can be used, including, for example, lossless compression without degradation or lossy compression that reduces file size but is irreversible. Furthermore, the white balance white search box determination result obtained in S1205, the histogram obtained in S704, and the face detection result obtained in S705 are recorded in the header. The white search box determination result to be detected here is the determination result before the brightness and darkness elimination processing in S1206. Therefore, the same determination result is recorded in both HDR and SDR shooting. Additionally, when the user sets the HDR shooting mode, HDR imaging parameters (such as...) are... Figure 12 The obtained white balance coefficient and Figure 14A and Figure 14B The tone correction amount obtained, and the MPF image generated in the S708 for display by encoding the HDR image data in HEVC format, are also recorded as follows: Figure 8C The metadata shown. As mentioned earlier, the content of this data varies depending on whether the shooting mode is HDR or SDR. Note that in SDR shooting, the display parameters obtained by using the white search box and tonal characteristics described above for SDR are recorded. Note that even when shooting in HDR, SDR display parameters can be generated and recorded by performing the processing in S702 to S706 on the SDR. Also note that the processing load increases when generating display parameters for both HDR and SDR; therefore, this processing can be performed when the processing load is relatively low, such as when shooting in a single burst instead of continuously.

[0259] When processing power is sufficient, such as during a single shot, in addition to the HDR display image, an SDR-quality main image, MPF image, and thumbnail image can be generated using SDR display parameters, and the HDR display image and SDR display image can be recorded in the same file. Figure 8D ).

[0260] When displaying thumbnail images, the images are small, so only the image type needs to be identifiable. Therefore, only thumbnail images can be created and saved as SDR display images in S709. Figure 8E In such an arrangement, a display device or PC that does not correspond to H.265 decoding, which is an HDR compression method, can display only a thumbnail image.

[0261] In S713, the system control unit 50 adds static or dynamic metadata by compressing and encoding the displayed HDR image in HECV format, and records the encoded image as an HEIF (High Efficiency Image File Format) file on the recording medium 200 via the recording medium I / F 18. Examples of static metadata include the x and y coordinates of the three primary colors and white point of a CEA-861.3 compliant display, as well as the maximum brightness value, minimum brightness value, maximum content brightness level, and maximum frame average brightness level of the mastering display. Examples of dynamic metadata include dynamic tone mapping metadata based on color capacity conversion as defined by SMPTE ST 2094. Note that a depth of 10 bits or more is preferred when expressing HDR characteristics via PQ signals. However, since the traditional JPEG format only has 8 bits, a new container is needed for still image HDR. This embodiment uses the HEIF container as an image file format developed by MPEG (Moving Picture Experts Group) and defined by MPEG-H Part 12 (ISO / IEC 23008-12). The key feature of HEIF is that it can store not only the main image, but also thumbnails, multiple time-related images, and metadata such as EXIF ​​or XMP in a single file. This makes HEIF convenient because it can also store 10-bit image sequences encoded by HEVC.

[0262] In S714 and S715, the processes in S712 and S713 are performed sequentially to record both the RAW image and the HDR image.

[0263] Figure 8A The structure of a RAW image file, which contains still RAW image data to be recorded on recording medium 200 in the above recording process, is shown. The container file format of the RAW image file, which will be described below, is the ISO Basic Media File Format as defined by ISO / IEC 14496-12. Therefore, the container format of this file has a tree structure and has nodes, each called a box. Furthermore, each box can have multiple boxes as child elements.

[0264] The RAW image data file 801 has a header containing a box `ftyp` 802 describing the file type, and also contains a box `moov` 803 containing all metadata, a box `mdat` 808 containing the main track media data (image data), and other boxes 807. Box `moov` 803 has a box `uuid` 804 for storing MetaData 805 and a box `trak` 806 for storing information referencing ImageData as child elements. MetaData 805 describes the image's metadata, such as the image's creation date / time, shooting conditions, information indicating whether it was shot using HDR or SDR, the aforementioned detection metadata, and other shooting information. Box `mdat` 808 has an ImageData 809 as a child element, representing the captured still image data.

[0265] Note that the image data to be recorded in ImageData 809 for RAW images captured by SDR is different from the image data for RAW images captured by HDR.

[0266] Figure 8B ImageData 809 is shown to be recorded in a RAW image taken by SDR. Figure 8B The ImageData 809 shown includes a THM image 821, an MPF ​​image 822, and a main image 823 (each rendered in SDR image quality and compressed via JPEG), as well as a RAW image 824 and RAW rendering parameters 825. Each SDR quality image is an image with 8 bits (256 hues) for each color component. Note that... Figure 8B The RAW display parameters 825 shown include at least the display parameters used for SDR display.

[0267] Figure 8C ImageData 809 is shown in a RAW image that is recorded when shooting in HDR, with only the HDR image as the display image. Figure 8C The ImageData 809 shown has a THM image 826, an MPF ​​image 827, and a main image 828 (each of which is rendered in HDR image quality and compressed using HEVC), as well as a RAW image 824 and RAW rendering parameters 825. Each HDR quality image is an image with 10 bits (1024 hues) for each color component. Figure 8C , Figure 8D and Figure 8E The RAW display parameters 825 shown have at least the display parameters required for HDR display.

[0268] Figure 8DImageData 809 is shown as the image data to be recorded in a RAW image that displays both the HDR and SDR images when shooting in HDR. Figure 8D The ImageData 809 shown has a THM image 821, an MPF ​​image 822 and a main image 823 (each of which is displayed in SDR image quality and compressed by JPEG), a THM image 826, an MPF ​​image 827 and a main image 828 (each of which is displayed in HDR image quality and compressed by HEVC), a RAW image 824 and RAW display parameters 825.

[0269] Figure 8E The image data 809 shown is to be recorded in a RAW image when shooting in HDR, where only the THM image is used as an SDR image, and the MPF image and the main image are used as HDR quality display images. Figure 8E The ImageData 809 shown includes: a THM image 821 rendered in SDR image quality and compressed by JPEG, an MPF ​​image 827 and a main image 828 rendered in HDR image quality and compressed by HEVC, and a RAW image 824 and RAW rendering parameters 825.

[0270] The file formats illustrated in this example are one embodiment; other boxes may be used as needed. Additionally, the display image may be stored in a box of the moov 803 or another box 807.

[0271] Because the file format described above is used, the display parameters for SDR images are recorded in the RAW image file captured as SDR, and the display parameters for HDR images are recorded in the RAW image file captured as HDR. Therefore, even when displaying RAW images later, the display can be performed by reflecting the shooting settings. For example, the device used for RAW display (which could be the digital camera 100 or another device such as a PC) refers to the MetaData 805 of the RAW image and determines whether the image was captured as HDR or SDR. If it is determined that the RAW image was captured as HDR, the RAW image is displayed as an HDR image using the display parameters for HDR images included in the file. If it is determined that the RAW image was captured as SDR, the RAW image is displayed as an SDR image using the display parameters for SDR images included in the file. To enable this processing, the digital camera 100 records the display parameters for SDR images in the RAW image file captured as SDR, and the display parameters for HDR images in the RAW image file captured as HDR. Note that the device used for RAW imaging can record still HDR images that have been developed using the HEIF container described above.

[0272] Furthermore, the same determination results are recorded as detection metadata for both HDR and SDR shooting. Therefore, by using the recorded detection data, even RAW image files shot in HDR shooting mode can be displayed as SDR images. Thus, even devices that only support SDR images can properly display RAW image files shot in HDR shooting mode.

[0273] Figure 9A This is a flowchart illustrating the details of the playback mode processing using the display unit 28 of the system control unit 50. This processing is achieved by expanding the program recorded in the non-volatile memory 56 in the system memory 52 and executing the program by the system control unit 50.

[0274] In S901, the system control unit 50 determines whether the playback is index playback or normal playback. If it is determined in S901 that the playback is index playback, the system control unit 50 advances the processing to S902. In S902, the system control unit 50 determines the number of images to be played back.

[0275] In S903, the system control unit 50 determines the image to be played back. In S904, the system control unit 50 performs the process of drawing the image to be played back.

[0276] In S905, the system control unit 50 determines whether the drawing of all images to be displayed is complete. If it is determined that the drawing is incomplete, the system control unit 50 returns the processing to S903 and continues the drawing process. If it is determined that the drawing process is complete, the system control unit 50 advances the processing to S906. In S906, the system control unit 50 performs the image output processing of S906 on the display unit 28 and terminates the display process. Afterwards, the system control unit 50 performs operation acceptance processing.

[0277] Figure 9B This is a flowchart illustrating the details of the drawing process in the playback mode processing using the display unit 28 of the system control unit 50.

[0278] In S911, the system control unit 50 obtains information about the image to be played back. In S912, the system control unit 50 determines the image to be played back. In S913, the system control unit 50 reads the image to be played back from the recording medium 200. In S914, the system control unit 50 performs decompression processing on the image to be played back. In S915, the system control unit 50 collects data of each pixel from the image data that has been decompressed in S914. This image data includes brightness data, etc., and is used for histogram processing or highlight warning processing.

[0279] In S916, the system control unit 50 determines whether the image to be played back is an HDR image or an SDR image. If it is determined that the image to be played back is an HDR image, the system control unit 50 proceeds the processing to S917; and if it is determined that the image to be played back is an SDR image, it proceeds the processing to S920. In S917, the system control unit 50 checks the HDR auxiliary display settings for playback. If Auxiliary 1 is set, the system control unit 50 proceeds the processing to S918; if Auxiliary 2 is set, it proceeds the processing to S919. In S918, the system control unit 50 performs HDR→SDR conversion processing on the image decompressed in S914 according to the setting of Auxiliary 1. In S919, the system control unit 50 performs HDR→SDR conversion processing on the image decompressed in S914 according to the setting of Auxiliary 2.

[0280] In S920, the system control unit 50 performs a process of enlarging or reducing the size of the image decompressed in S914 or the image that has undergone SDR conversion in S918 or S919 to a size suitable for the display unit 28. Then, in S921, the system control unit 50 determines the placement of the generated image and terminates the drawing process.

[0281] Figures 9C to 9H This is a flowchart illustrating the details of the object image selection process performed by the system control unit 50.

[0282] In S926, the system control unit 50 checks the information of the acquired image and determines whether the image can be played back. If it is determined that the image can be played back, the system control unit 50 advances the process to S927, and if it is determined that the image cannot be played back, the process advances the process to S936.

[0283] In S927, the system control unit 50 determines whether the image to be played back is a still image. If it is determined that the image to be played back is a still image, the system control unit 50 proceeds the process to S928; otherwise, it proceeds the process to S935.

[0284] In S928, the system control unit 50 determines whether the image to be played back is a RAW image. If it is determined that the image to be played back is a RAW image, the system control unit 50 proceeds the processing to S929; otherwise, it proceeds the processing to S930.

[0285] In S929, the system control unit 50 determines whether the RAW image was captured using HDR or SDR. The system control unit 50 uses a reference... Figures 8A to 8E The determination is made using the metadata in the RAW file. If the RAW image is determined to be a RAW image captured in HDR, the system control unit 50 proceeds the processing to S931, and if the RAW image is determined to be an SDR image, the processing proceeds to S932.

[0286] In S930, the system control unit 50 determines whether a still image that is not a RAW image is an HDR image or an SDR image. In this embodiment, HDR images are recorded using HEIF, and SDR images are recorded using JPEG; therefore, whether an image is an HDR image or an SDR image is determined by whether the format is HEIF or JPEG. However, it is also possible to determine whether an image is an HDR image or an SDR image by using metadata in HEIF.

[0287] In S931, the system control unit 50 selects image data to be used in playback from RAW images captured by HDR. In S932, the system control unit 50 selects image data to be used in playback from RAW images captured by SDR. In S933, the system control unit 50 selects image data to be used in playback from still images developed by HDR. In S934, the system control unit 50 selects image data to be used in playback from still images captured by SDR. In S935, the system control unit 50 selects image data to be displayed from a moving image file. In S936, the system control unit 50 performs a non-display processing step for the playback image. In this process, a message indicating that the image cannot be played is displayed to notify the user that the image cannot be played back.

[0288] Figure 9D This is a flowchart of the system control unit 50 selecting image data to be used in playback from RAW images captured by HDR.

[0289] In S941, the system control unit 50 determines whether the playback is index playback or normal playback. If it is determined that the playback is index playback, the system control unit 50 advances the process to S942, and if it is determined that the playback is normal playback, it advances the process to S943.

[0290] In S942, the system control unit 50 determines the image data to be used based on the number of images to be played back via index playback. Although the threshold is 36 in this embodiment, this number is only an example, and therefore can be appropriately set by the user or determined according to the size of the display unit 28. If it is determined that the number of images to be displayed is 36 or more, the system control unit 50 causes the processing to proceed to S945, and if it is determined that the number is less than 36, it causes the processing to proceed to S944.

[0291] In S943, the system control unit 50 determines "HDR main image (HEVC) for display" (828) as the image data to be used in playback. In S944, the system control unit 50 determines "HDR MPF image (HEVC) for display" (827) as the image data to be used in playback. In S945, the system control unit 50 determines "HDRTHM image (HEVC) for display" (826) as the image data to be used in playback.

[0292] Figure 9E-1 and Figure 9E-2 This is a flowchart illustrating the process of selecting image data to be used in playback from RAW images captured by HDR when RAW images have SDR images for display.

[0293] In S951, the system control unit 50 determines whether the playback is index playback or normal playback. If it is determined that the playback is index playback, the system control unit 50 advances the processing to S952, and if it is determined that the playback is normal playback, it advances the processing to S953.

[0294] In S952, the system control unit 50 determines the image data to be used based on the number of playback images used for indexing playback. Assume the threshold determined in this case is 36. If the number of playback images is determined to be 36 or more, the system control unit 50 proceeds the process to S955; if the number is determined to be less than 36, the process proceeds to S954.

[0295] In S953, S954, and S955, the system control unit 50 determines whether the RAW image to be played back contains an SDR image. This determination uses a reference... Figures 8A to 8E Metadata in the RAW file described.

[0296] In S956, the system control unit 50 determines "HDR main image (HEVC) for display" (828) as the image data to be used in playback. In S957, the system control unit 50 determines "SDR main image (JPEG) for display" (823) as the image data to be used in playback. In S958, the system control unit 50 determines "HDRMPF image (HEVC) for display" (827) as the image data to be used in playback. In S959, the system control unit 50 determines "SDR MPF image (JPEG) for display" (822) as the image data to be used in playback. In S960, the system control unit 50 determines "HDR THM image (HEVC) for display" (826) as the image data to be used in playback. In S961, the system control unit 50 determines "SDR THM image (JPEG) for display" (821) as the image data to be used in playback.

[0297] Figure 9F This is a flowchart of the system control unit 50 selecting image data to be used in playback from still images displayed by HDR.

[0298] In S971, the system control unit 50 determines whether the playback is index playback or normal playback. If it is determined that the playback is index playback, the system control unit 50 advances the process to S972, and if it is determined that the playback is normal playback, it advances the process to S973.

[0299] In S972, the system control unit 50 determines the image data to be used based on the number of images to be played back in the index playback. In this embodiment, the threshold for the number is 36. If it is determined that the number of images to be played back is 36 or more, the system control unit 50 advances the processing to S975, and if it is determined that the number is less than 36, the system control unit 50 advances the processing to S974.

[0300] In S973, the system control unit 50 determines "HDR main image (HEVC)" (not shown) as the image data to be used in playback. In S974, the system control unit 50 determines "HDR MPF image (HEVC)" (not shown) as the image data to be used in playback. In S975, the system control unit 50 determines "HDR THM image (HEVC)" (not shown) as the image data to be used in playback.

[0301] Figure 9G This is a flowchart for selecting image data to be used in playback from RAW images captured by SDR.

[0302] In S981, the system control unit 50 determines whether the playback is index playback or normal playback. If it is determined that the playback is index playback, the system control unit 50 proceeds to S982, and if it is determined that the playback is normal playback, it proceeds to S983.

[0303] In S982, the system control unit 50 determines the image data to be used based on the number of playback images in the index playback. In this embodiment, the threshold for the number is 36. If it is determined that the number of images to be displayed is 36 or more, the system control unit 50 proceeds the process to S985, and if it is determined that the number is less than 36, it proceeds the process to S984.

[0304] In S983, the system control unit 50 determines "SDR main image (JPEG) for display" (823) as the image data to be used in playback. In S984, the system control unit 50 determines "SDR MPF image (JPEG) for display" (822) as the image data to be used in playback. In S985, the system control unit 50 determines "SDR DHM image (JPEG) for display" (821) as the image data to be used in playback.

[0305] Figure 9H It is a flowchart for selecting image data to be used in playback from still images developed by SDR.

[0306] In S991, the system control unit 50 determines whether the playback is index playback or normal playback. If it is determined that the playback is index playback, the system control unit 50 advances the process to S992, and if it is determined that the playback is normal playback, it advances the process to S993.

[0307] In S992, the system control unit 50 determines the image data to be used based on the number of images to be played back in the index playback. In this embodiment, the threshold for the number is 36. If it is determined that the number of images to be played back is 36 or more, the system control unit 50 proceeds to S995, and if it is determined that the number of images to be played back is less than 36, the system control unit 50 proceeds to S994.

[0308] In S993, the system control unit 50 determines "SDR master image (JPEG)" (not shown) as the image data to be used in playback. In S994, the system control unit 50 determines "SDR MPF image (JPEG)" (not shown) as the image data to be used in playback. In S995, the system control unit 50 determines "SDR THM image (JPEG)" (not shown) as the image data to be used in playback.

[0309] Figure 10A This is a flowchart illustrating the details of the playback mode processing using external device 300. This processing is achieved by expanding a program recorded in non-volatile memory 56 in system memory 52 and executing the program by system control unit 50.

[0310] In S1001, the system control unit 50 determines whether the external device 300 is connected to the digital camera 100. If it is determined that the external device 300 is connected, the system control unit 50 proceeds the process to S1002, and if it is determined that the external device 300 is not connected, the process proceeds to S1005.

[0311] In S1002, the system control unit 50 determines whether the HDR playback setting is valid. As playback settings, one can select "Perform HDR playback," "Do not perform HDR playback," or "Synchronize with shooting mode." When "Perform HDR playback" is set, if the external device 300 supports HDR, the mode is HDR output mode, regardless of whether the image to be played back is an HDR or SDR image. "Do not perform HDR playback" is SDR output mode. "Synchronize with shooting mode" is a mode where playback output is synchronized with the shooting mode. That is, in HDR shooting mode where "HDR shooting" is set to "Perform," HDR output occurs during playback. In SDR shooting mode where "HDR shooting" is set to "Do not perform," SDR output occurs during playback. Note that "Synchronize with shooting mode" is set as the default mode and remains so even if the user changes the shooting mode. This synchronization is only canceled when the user changes the playback setting from "Synchronize with shooting mode" to "Perform HDR playback" or "Do not perform HDR playback." File formats such as "HEIF (playback)" and "JPEG (playback)" can also be used as options to replace "perform HDR playback" and "do not perform HDR playback". Similarly, file formats such as "HEIF (shoot)" and "JPEG (shoot)" can also be used as options to replace "perform HDR shooting" and "do not perform HDR shooting".

[0312] In S1002, if "Perform HDR playback" is selected, the system control unit 50 moves the processing to S1003, while if "Do not perform HDR playback" is selected, the processing moves to S1005. Furthermore, if "Synchronize with shooting mode" is selected and "HDR shooting" is set to "Perform" in S606, the system control unit 50 advances the processing to S1003, and if "HDR shooting" is set to "Do not perform", the processing advances to S1005.

[0313] In S1003, the system control unit 50 determines whether the external device 300 is an HDR-enabled display. If it is determined that the external device 300 is an HDR-enabled display, the system control unit 50 proceeds the processing to S1004; otherwise, it proceeds the processing to S1005.

[0314] In S1004, the system control unit 50 outputs an HDR signal to the external device 300. In S1005, the system control unit 50 outputs an SDR signal to the external device 300.

[0315] S1006 to S1011 and Figure 9A S901 to S906 are the same, so their description will be omitted.

[0316] Figure 10BThis is a flowchart showing the details of the drawing process (S1009) when the HDR signal is output to the external device 300.

[0317] S1021 to S1025, S1028 and S1029 and reference Figure 9B The descriptions of S911 to S915, S920 and S921 are the same, so their descriptions will be omitted.

[0318] In S1026, the system control unit 50 determines whether the image to be played back is an HDR image or an SDR image. If it is determined that the image to be played back is an HDR image, the system control unit 50 advances the processing to S1028, and if it is determined that the image to be played back is an SDR image, it advances the processing to S1027.

[0319] In S1027, the system control unit 50 performs SDR→HDR conversion processing. Subsequent steps S1028 and S1029... Figure 9B S920 and S921 are the same. Note the details of the drawing process (S1009) when the SDR signal is output to the external device 300. Figure 9B Since they are the same, their descriptions will be omitted.

[0320] Figure 11A This is a flowchart illustrating the details of the playback menu processing. This processing is achieved by expanding the program recorded in the non-volatile memory 56 in the system memory 52 and executing the program by the system controller 50.

[0321] In S1101, the system control unit 50 determines whether the user has set up RAW display through the RAW display settings (not shown). If it is determined that RAW display has not been set, the system control unit 50 proceeds the process to S1103, and if it is determined that RAW display has been set, it proceeds the process to S1102.

[0322] In S1103, the system control unit 50 determines whether HDR→SDR conversion has been set via a setting item (not shown) for SDR conversion of HDR files. If it is determined that HDR→SDR conversion has not been set, the system control unit 50 proceeds the processing to S1105, and if it is determined that HDR→SDR conversion has been set, the processing proceeds to S1104.

[0323] In S1105, the system control unit 50 determines whether file transfer has been set via a file transfer setting item (not shown). If it is determined that file transfer has not been set, the system control unit 50 proceeds the process to S1107, and if it is determined that file transfer has been set, it proceeds the process to S1106.

[0324] In S1107, the system control unit 50 determines whether to exit the menu. If it is determined that "do not exit", the system control unit 50 returns the processing to S1101, and if it is determined that "exit", the system control unit 50 terminates the playback menu processing.

[0325] In S1106, the system control unit 50 performs transmission processing on the user-specified image file. If the receiving destination can only display SDR when transmitting an HDR image file, the HDR→SDR conversion shown in S1104 can be performed in the camera, and the HDR image file can be transmitted as an SDR image file.

[0326] In S1102, the system control unit 50 performs RAW image processing on the user-specified RAW image file. The following will refer to... Figure 11B The block diagram shown illustrates the details of this RAW image processing. Note that the image processing unit 24 includes... Figure 11B The processing units shown are shown, but each processing unit can also be implemented by a program to be executed by the system control unit 50.

[0327] The system control unit 50 causes the image processing unit 24 to perform RAW development processing on the captured RAW image 1101 recorded on the recording medium 200. A RAW image is a set of pixels with a Bayer array, so each pixel has intensity data for only one color component. Note that RAW images include RAW (SDR) obtained during SDR shooting and RAW (HDR) obtained during HDR shooting. Furthermore, RAW (SDR) can be developed using either SDR or HDR development. Conversely, RAW (HDR) can be developed using either HDR or SDR development. The white balance unit 1102 performs a process to whiten white. When performing HDR development on RAW (HDR), white balance processing is performed using HDR white balance coefficients as HDR pixels recorded in the file. On the other hand, when performing SDR development, white balance processing is performed by generating SDR white balance coefficients based on the determination results within the white search box as detection pixels stored in the file. When both HDR and SDR white balance coefficients are recorded in RAW, one can, of course, be used as needed.

[0328] Color interpolation unit 1103 interpolates the noise-reducing and color mosaic image to generate a color image where each pixel has three components (e.g., color information of R, G, and B). The generated color image is processed by matrix transformation unit 1104 and gamma transformation unit 1105 to generate a basic color image. Then, color brightness adjustment unit 1106 performs image correction on the generated color image to improve its appearance. For example, it performs image correction based on scene detection (night scene) and emphasizes saturation. Tone correction is performed in the same manner. However, when performing HDR rendering on RAW (HDR), tone correction is performed by using the tone correction amount for HDR as HDR pixels stored in the file. Conversely, when performing SDR rendering, tone correction is performed by calculating the tone correction amount for SDR using face detection results and histograms as detection pixels recorded in the file. When both HDR and SDR tone correction amounts are recorded in RAW, one can, of course, be used as needed.

[0329] For an image with the desired color adjustment, compression unit 1107 compresses the high-resolution image using methods such as JPEG or HEVC, and recording unit 1108 generates the display image to be recorded on a recording medium such as flash memory. Note that the HEIF container described above can store multiple images, so in addition to HDR display images, SDR display images can also be stored.

[0330] In S1104, the system control unit 50 performs SDR conversion on the user-specified HDR image file. Since the HDR image is generated in a color space with OETF PQ and a color gamut of BT.2020, tone mapping and gamut mapping are required in a color space such as γ2.2 or sGRB for SDR. As a practical approach, well-known methods can be used. For example, when performing tone mapping that matches the appropriate exposure to SDR, a result with increased brightness compared to SDR can be obtained.

[0331] (Variant Example)

[0332] In the above embodiments, when recording RAW images using HDR shooting, the main image developed by HDR, the MPF image for display developed by HDR, and the THM (thumbnail) image developed by HDR are recorded together with the RAW image data in a RAW image file, such as... Figure 8C As shown. Alternatively, such as... Figure 8D As shown, not only the main image, MPF image, and THM image displayed via HDR, but also the main image, MPF image, and THM image displayed via SDR are recorded along with the RAW image. Otherwise, as... Figure 8EAs shown, the main image and MPF image, which are rendered in HDR, and the THM image, which is rendered in SDR, are recorded together with the RAW image.

[0333] In this variant, during the shooting process, an HDR image file or an SDR image file is associated with a RAW image file and recorded together with it. When a RAW image file is recorded in association with an HDR image file or an SDR image file, if there are differences in the developing method or compression encoding method between the HDR image or SDR image contained in the RAW image file and the HDR image or SDR image contained in the image file associated with the RAW image file, management becomes complicated or playback compatibility cannot be maintained. Therefore, in this variant, for both the RAW image file and the image file associated with it, images generated by developing using the same method and encoding using the same compression encoding method are recorded as the main image, MPF image, and THM image.

[0334] Figure 19A and Figure 19B This is a flowchart of the shooting process according to this variant example, which corresponds to the HDR shooting process in S418. Figure 7A and Figure 7B The SDR imaging process is described in S434. This process is achieved by expanding the program recorded in the non-volatile memory 56 in the system memory 52 and executing the program by the system control unit 50.

[0335] First, in S1901, the system control unit 50 obtains the RAW data. This process is the same as in S701.

[0336] Then, in S1902, the system control unit 50 determines whether the image quality setting is set to HDR shooting mode. If it is determined that HDR shooting mode is set, the system control unit 50 proceeds the processing to S1903, and if it is determined that SDR shooting mode is set, it proceeds the processing to S1909.

[0337] In S1903, the system control unit 50 performs the same processing as in S702 to S710. That is, using the RAW data obtained in S1901, various parameters are calculated and detection processing is performed, and HDR main image, HDR MPF image, and HDR THM image are generated by performing HDR imaging processing on the obtained RAW data. Then, the 10-bit data of each of the HDR main image, HDR MPF image, and HDR THM image are compressed and encoded using HECV format to generate HDR compressed image data (HEVC).

[0338] Subsequently, in S1904, the system control unit 50 determines the recorded image quality set by the user in the same manner as in S711. If it is determined that only RAW image files will be recorded by setting, the system control unit 50 causes the processing to proceed to S1905; if it is determined that only HDR image files will be recorded by setting, the processing proceeds to S1906; and if it is determined that both RAW image files and HDR image files will be recorded by setting, the processing proceeds to S1907.

[0339] In S1905, the system control unit 50 records the RAW image data obtained in S1901 in the same manner as in S712, as having the following characteristics: Figure 8A The RAW image file in the container file format shown. In S1905, the HDR main image, HDR MPF image, and HDR THM image generated and compressed in HEVC format in S1903 are recorded together with the RAW image data in ImageData 809 as an image for display, such as... Figure 8C As shown.

[0340] In S1906, the system control unit 50 records the HDR main image, HDR MPF image, and HDR THM image generated and compressed in S1903 into image files with HEIF format, in the same manner as in S713. That is, the HDR main image is recorded as the main image of the HEIF file, and the HDR MPF image and HDR THM image are recorded in the HEIF file as images for display.

[0341] The HEIF file will be explained. Figure 17AThe structure of an image file in HEIF format is shown. The container file format of the image file illustrated below is the ISO Basic Media File Format defined by ISO / IEC 14496-12. Therefore, the container format of this file has a tree structure and nodes called boxes. Furthermore, each box can have multiple boxes as child elements. The HEIF image data file 1701 has a box ftyp 1702 in the header for describing the file type, and also has a box meta 1703 containing metadata and a box mdat 1708 containing the media data body (image data) of the tracks. Box meta 1703 has a trak box 1706 as a child element for storing information referencing ImageData and a MetaData 1705-1 for storing metadata other than the metadata defined by EXIF. Box mdat 1708 has boxes MetaData 1705-2 and ImageData 1709 as child elements. Box MetaData 1705-2 is used to store metadata such as the shooting date / time and shooting conditions of the image as defined by EXIF. ImageData 1709 stores the captured still image data. ImageData 1709 stores image data. The image data to be recorded in ImageData 1709 when shooting with SDR is different from the image data to be recorded when shooting with HDR. In S1906, as... Figure 17C The image shown is recorded in ImageData 1709. In this case, THM image 1726 and MPF image 1727 for display, which are displayed in HDR image quality and compressed by HEVC, and main image 1728, which is displayed in HDR image quality and compressed by HEVC, are recorded in ImageData 1709.

[0342] In S1907, the system control unit 50 records the RAW image data obtained in S1901 in the same manner as in S1905, as having the following characteristics: Figure 8AThe RAW image file in the container file format shown is then recorded in S1908 as an image file in HEIF format, in the same manner as in S1906. This includes the HDR main image, HDR MPF image, and HDR THM image generated and compressed in HEVC format in S1903. That is, when recording RAW image data in a RAW file and an image file (HEIF file) for HDR display, the same images, processed in the same way and encoded in the same encoding format (HEVC), are recorded as MPF and THM images for display. Furthermore, the image data for the main image to be recorded as a RAW image file for display and the image data for the main image to be recorded as a display image file (HEIF file) are the same images encoded in the same encoding format (HEVC). Note that the system control unit 50 associates the RAW image file to be recorded in S1907 with the display image file (HEIF file) to be recorded in S1908.

[0343] As described above, identical images, processed with the same HDR imaging and encoded in the same encoding format (HEVC), are recorded simultaneously in interconnected RAW and HEIF files and used for display. This prevents management complexity and maintains playback compatibility.

[0344] Furthermore, as images to be recorded for display in the RAW file recorded in S1905 and the HEIF file recorded in S1906, images processed using the same HDR image processing and HEVC encoding as in S1907 and S1908 are also recorded. Therefore, even when the recording format changes, the inconvenience of other playback devices being unable to play back images from specific image files captured using the same HDR shooting settings is prevented, thus maintaining playback compatibility.

[0345] In S1909, the system control unit 50 instructs the image processing unit 24 to perform SDR image processing on the RAW image data obtained in S701, thereby generating an SDR main image, an SDR MPF image, and an SDR THM image. Then, SDR compressed image data (JPEG) is generated by compressing and encoding each image in JPEG format. Since the image is an SDR image, the image data for image development and compression encoding is 8-bit YUV420 data. Furthermore, the calculation and detection processing of various parameters are performed in the same manner as in HDR shooting.

[0346] Subsequently, in S1910, the system control unit 50 determines the recorded image quality set by the user. If it is determined that only RAW image files are recorded by the setting, the system control unit 50 proceeds the process to S1911; and if it is determined that only SDR image files are recorded by the setting, the process proceeds to S1912. Furthermore, if it is determined that both RAW image files and SDR image files are recorded by the setting, the system control unit 50 proceeds the process to S1913; and if it is determined that both RAW image files and various types of SDR image files are recorded by the setting, the process proceeds to S1914.

[0347] In S1911, the system control unit 50 compresses the RAW image, adds a header, and, in the same manner as in S1905, displays the RAW image as having... Figure 8A The RAW image file with the container file structure shown is recorded on the recording medium 200 via the recording medium I / F18. In S1911, unlike S1905, the SDR image generated in S1909 is recorded as image data for display, and the SDR display parameters generated in S1909 are recorded as RAW display parameters. That is, in S1911, as... Figure 8B As shown, the data is recorded in ImageData 809 of the RAW image file. In S1911, the raw image data (lossless or lossy compressed) 824 obtained in S1901 and the SDR RAW display parameters generated in S1909 are recorded in ImageData 809 of the RAW image file 801. Furthermore, the SDR THM image (JPEG) 821, SDR MPF image (JPEG) 822, and SDR main image (JPEG) 823 generated in S1909 are recorded in ImageData 809 of the RAW image file 801 as images for display.

[0348] In S1912, the system control unit 50 records the SDR display image data (THM image, MPF image and main image) generated and compressed by JPEG in S1909 as a JPEG file on the recording medium 200 via the recording medium I / F 18. Figure 18 The file structure of a JPEG format is shown. An image data file 1800 in JPEG format has metadata 1804 in the header, such as EXIF, and also includes a THM image 1801, an MPF ​​image 1802, and a main image 1803, each with SDR image quality and compressed via JPEG. The file format shown in this example is an embodiment; other information may be included as needed.

[0349] In S1913, the system control unit 50 records the RAW image data obtained in S1901 and the SDR image data generated and compressed by JPEG in S1909 as a JPEG image file on the recording medium 200 via the recording medium I / F 18 in the same manner as in S1911. Then, in S1914, the system control unit 50 records the SDR image data generated and compressed by JPEG in S1909 as a JPEG image file on the recording medium 200 via the recording medium I / F 18 in the same manner as in S1912. As described above, when recording the RAW image file of RAW image data and the image file (JPEG file) developed by SDR, the same images that have undergone the same development processing and are encoded in the same encoding format (JPEG) are recorded as MPF image and THM image for display. Furthermore, the image data of the image to be recorded as a RAW image file for display and the image data of the main image to be recorded as a developed image file (JPEG file) are the same images that have undergone the same development processing and are encoded in the same encoding format (JPEG). Note that the system control unit 50 associates the RAW image file to be recorded in S1913 with the display image file (JPEG file) to be recorded in S1914, and records them.

[0350] As described above, in RAW and JPEG files that are recorded simultaneously and are interconnected, the same image, processed by the same SDR imaging and encoded in the same encoding format (JPEG), is recorded as the image used for display. This prevents management complexity and maintains playback compatibility.

[0351] In S1915, the system control unit 50 records the RAW image data obtained in S1901 and the JPEG compressed SDR image data generated in S1909 as a JPEG file onto the recording medium 200 via the recording medium I / F 18, in the same manner as in S1911. Then, in S1916, the system control unit 50 records the JPEG compressed SDR image data generated in S1909 as a JPEG image file onto the recording medium 200 via the recording medium I / F 18, in the same manner as in S1912. Furthermore, in S1917, the system control unit 50 records the JPEG compressed SDR image data generated in S1909 as an HEIF file onto the recording medium 200 via the recording medium I / F 18. In S1917, unlike S1906, the image data is recorded in ImageData 1709, as shown below. Figure 17BAs shown. That is, the SDR THM image (JPEG) 1721 and SDRMPF image (JPEG) 1722 generated in S1909 are recorded as images for display, and the SDR main image (JPEG) 1723 generated in S1909 is recorded as the main image.

[0352] Note that the system control unit 50 associates the RAW file to be recorded in S1915, the JPEG file to be recorded in S1916, and the HEIF file to be recorded in S1917. As described above, in the RAW, JPEG, and HEIF files recorded simultaneously in these interconnected files, the same image processed by the same SDR image processing and encoded in the same encoding format (JPEG) is recorded as the image for display. This prevents management complexity and maintains playback compatibility. Furthermore, in the RAW files to be recorded in S1911, S1913, and S1915, the JPEG files to be recorded in S1912, S1914, and S1916, and the HEIF file to be recorded in S1917, the images processed by the same HDR image processing and encoded in the same format (JPEG) for display are recorded. Therefore, even if the recording format changes, it prevents the inconvenience of other playback devices being unable to play back images from specific image files captured using the same SDR shooting settings, thus maintaining playback compatibility.

[0353] Please note that in the above description, SDR images were recorded in JPEG files in S1912 and S1914, but SDR images can also be recorded as HEIF files in the same way as in S1917.

[0354] It should also be noted that, as explained above, the HDR THM and HDR MPF images to be recorded in the RAW file for display in S1907 are the same image data as those to be recorded in the HEIF file for display in S1908. However, these images do not need to be exactly the same, as long as they were generated by encoding the HDR-processed images using the same encoding format. Similarly, the images to be recorded in the RAW file for display in S1913 and the images to be recorded in the JPEG file for display in S1914 do not need to be exactly the same, as long as they were generated by encoding the SDR-processed images using the same encoding format. This also applies to S1915.

[0355] Furthermore, in S1907, the HDR master image data that will be recorded as the master image in the HEIF file in S1908 is recorded as a RAW image file for display. However, it is also possible to record image data of different sizes instead of recording identical image data. Similarly, in S1913, the SDR master image data that will be recorded as the master image in the JPEG file in S1914 does not need to be recorded as a RAW image file for display; instead, different image data can be recorded. This also applies to S1915.

[0356] Furthermore, when recording HEIF files, JPEG images can be recorded as thumbnail images for display instead of HEVC images, such as... Figure 17D As shown, when a JPEG image is recorded as a thumbnail for display, even display devices or PCs that do not support H.265 decoding as an HDR compression method can display only the thumbnail image.

[0357] (Other embodiments)

[0358] The preferred embodiments of the present invention have been described above, but the present invention is not limited to these embodiments, and various modifications and changes can be made within the spirit and scope of the present invention. For example, in the above embodiments, HEVC (High Efficiency Video Coding) is used to encode image data with a color component exceeding 8 bits. However, the type of method is not particularly limited, as long as the method can encode an image with a color component exceeding 8 bits. Furthermore, the above embodiments have been described by assuming that the present invention is applied to a digital camera. However, the present invention is not limited to the above embodiments and can also be applied to computers with camera capabilities (such as smartphones or laptops with cameras).

[0359] This invention is not limited to the embodiments described above, and various changes and modifications can be made within the spirit and scope of this invention. Therefore, the appended claims are made to disclose the scope of the invention.

[0360] This application claims priority to Japanese Patent Application 2019-36396, filed February 28, 2019, and Japanese Patent Application 2019-85969, filed April 26, 2019, the entire contents of which are incorporated herein by reference.

Claims

1. A camera device, characterized in that, include: Camera components; A processing unit is used to process the RAW image data obtained by the camera unit; An encoding unit is used to encode image data that has been processed by the processing unit; as well as The control unit is configured to perform the following control when recording the RAW image data obtained by the camera unit as a RAW image file: If the image data processed for the first dynamic range by the processing unit is to be recorded as an image file separate from the RAW image file, then the following encoded image data is recorded as an image for display in the RAW image file, wherein the encoded image data is obtained by the encoding unit encoding the image data processed for the first dynamic range using a first encoding format, and If the image data processed for the second dynamic range by the processing unit is to be recorded as an image file separate from the RAW image file, then the following encoded image data is recorded as an image for display in the RAW image file, wherein the encoded image data is obtained by the encoding unit encoding the image data processed for the second dynamic range in a second encoding format.

2. The camera device according to claim 1, wherein, When the control unit records the RAW image data obtained by the camera unit as a RAW image file, it performs the following control: If the image data processed for the first dynamic range is to be recorded as an image file separate from the RAW image file, then the encoded image data obtained by the encoding unit through encoding the image data processed for the first dynamic range in the first encoding format is also recorded into the image file. If the image data processed for the second dynamic range is to be recorded as an image file separate from the RAW image file, then the encoded image data obtained by the encoding unit through encoding the image data processed for the second dynamic range in the second encoding format shall also be recorded into the image file.

3. The camera device according to claim 1, wherein, When the control unit records the RAW image data obtained by the camera unit as a RAW image file, it performs the following control: If the image data processed for the first dynamic range is to be recorded as an image file separate from the RAW image file, then the encoded image data obtained by the encoding unit through encoding the image data processed for the first dynamic range in the first encoding format is to be recorded as an image for display in both the image file and the RAW image file. If the image data processed for the second dynamic range is to be recorded as an image file separate from the RAW image file, then the encoded image data obtained by the encoding unit through encoding the image data processed for the second dynamic range in the second encoding format is to be recorded as an image for display in both the image file and the RAW image file.

4. The camera device according to claim 1, wherein, When the control unit records the RAW image data obtained by the camera unit as a RAW image file, it performs the following control: The encoded image data obtained by encoding the image data processed for the first dynamic range using the first encoding format shall be recorded as an image file in a first file format separate from the RAW image file. The encoded image data obtained by encoding the image data processed for the second dynamic range using the second encoding format shall be recorded as an image file in a second file format separate from the RAW image file.

5. The camera device according to claim 4 further includes a setting component, the setting component being used to set: in addition to the RAW image file, to record which of the first file format image file and the second file format image file.

6. The camera device according to claim 5, wherein, The control unit performs the following control: If the setting includes recording image files of the first file format in addition to the RAW image files, then the encoded image data obtained by encoding the image data processed for the first dynamic range using the first encoding format will be recorded as an image for display in the RAW image files. If the setting is to record image files in the second file format in addition to the RAW image files, then the encoded image data obtained by encoding the image data processed for the second dynamic range using the second encoding format will be recorded as an image for display in the RAW image files.

7. The camera device according to claim 4, wherein, The second file format is the High Efficiency Image File Format (HEIF).

8. The camera device according to claim 4, wherein, The first file format is the Joint Image Experts Group format, i.e., JPEG format.

9. The camera device according to claim 1, wherein, The processing for the first dynamic range is the processing for generating image data with 8 bits for a color component, and The processing used for the second dynamic range is the processing used to generate image data with a color component having more than 8 bits.

10. The camera device according to claim 1, wherein, The first dynamic range is the standard dynamic range, i.e., SDR, and the second dynamic range is the high dynamic range, i.e., HDR.

11. The camera device according to claim 1, wherein, The first encoding format is JPEG, and the second encoding format is a high-efficiency video encoding format, namely HEVC.

12. The camera device according to claim 1, further comprising a selection component, the selection component being used to select the file format of an image file to be recorded separately from the RAW image file. in, Depending on the file format selected by the selection component, the control component switches between the following two: The encoded image data obtained by encoding the image data processed for the first dynamic range using the first encoding format is recorded as an image for display in the RAW image file. as well as The encoded image data obtained by encoding the image data that has been processed for the second dynamic range using the second encoding format is recorded as an image for display in the RAW image file.

13. The camera device according to claim 1, wherein, The control unit performs the following control: if the image data processed by the processing unit is to be recorded as an image file separate from the RAW image file, then the image data to be recorded in the image file is also recorded as the RAW image data for display.

14. A control method for a camera device, the camera device comprising a camera unit, the control method comprising: The RAW image data acquired by the camera unit is processed; Encode the image data that has undergone the processing described above; as well as When recording the RAW image data obtained by the camera unit as a RAW image file, the following control is performed: If the image data processed for the first dynamic range is to be recorded as an image file separate from the RAW image file, then encoded image data is recorded as an image for display in the RAW image file, wherein the encoded image data is obtained by encoding the image data processed for the first dynamic range in the encoding process using a first encoding format, and If the image data processed for the second dynamic range in the aforementioned processing is to be recorded as an image file separate from the RAW image file, then the following encoded image data is recorded as an image for display in the RAW image file, wherein the encoded image data is obtained by encoding the image data processed for the second dynamic range in the encoding with a second encoding format.

15. A computer-readable storage medium storing a program to be loaded into and executed by a computer including a camera component, thereby causing the computer to perform the control method according to claim 14.

Images