Video control device, video recording device, video control method, video recording method, and video control program

Abstract

The present invention provides a video control device, a video recording device, a video control method, a video recording method, and a video control program that can perform imaging with an appropriate exposure value depending on whether or not pixel data is recorded before demosaicing. [Solution] The temporary storage unit 106 temporarily stores the pixel data output from the imaging unit 119. The development processing unit 107 outputs video data obtained by performing development processing, including demosaicing, on the pixel data stored in the temporary storage unit 106. The output control unit 109 switches between a first recording mode in which only the video data is recorded as the video data and a second recording mode in which both the video data and the pixel data are recorded as the video data. The output control unit 109 also controls the imaging unit 119 to perform imaging with different exposure values ​​in the first recording mode and the second recording mode.

Description

Technical Field

[0004] , , , , ,

[0006] , , , , , , , , , , , , ,

[0005] , ,

[0001] The present invention relates to a video control device, a video recording device, a video control method, a video recording method, and a video control program.

Background Art

[0002] Patent Document 1 describes a camera system having a recording mode for storing RAW data and YC data for a thumbnail image converted from the RAW data.

[0003] Patent Document 2 describes an imaging device that starts recording a RAW video while continuing to record a FullHD video when a user presses a RAW video recording start button during recording of a FullHD (Full High Definition) video.

[0004] Patent Document 3 describes an imaging device that switches between a first mode for recording a moving image file with digital development images together with RAW data of an image and a second mode for recording digital development images.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

[0006] One embodiment of the technology according to the present disclosure provides a video control device, a video recording device, a video control method, a video recording method, and a video control program capable of performing imaging with an appropriate exposure value according to the presence or absence of recording of pixel data before demosaicing processing.

Means for Solving the Problems

[0007] A video imaging device according to one embodiment of the technology of the present disclosure includes: a storage unit for temporarily storing pixel data output from an imaging unit; a development processing unit for outputting video data obtained by performing a development process including demosaicing on the pixel data stored in the storage unit; and a mode switching control unit for switching between a first recording mode in which only the video data is used as recording data from the video data and the pixel data stored in the storage unit, and a second recording mode in which both the video data and the pixel data stored in the storage unit are used as recording data, and for controlling the imaging unit to perform imaging with different exposure values ​​in the first recording mode and the second recording mode.

[0008] A video recording device according to one embodiment of the technology of this disclosure comprises the video control device and a recording unit that records recording data output from the development processing unit.

[0009] A video control method according to one embodiment of the technology of the present disclosure is a video control method for a video control device that includes a storage unit for temporarily storing pixel data output from an imaging unit, and outputs video data obtained by performing a development process including demosaicing on the pixel data stored in the storage unit, and switches between a first recording mode in which only the video data is used as recording data from the video data and the pixel data stored in the storage unit, and a second recording mode in which both the video data and the pixel data stored in the storage unit are used as recording data, and controls the imaging unit to perform imaging with different exposure values ​​in the first recording mode and the second recording mode.

[0010] One embodiment of the video recording method relating to the technology of this disclosure is a video control method in which the recording data output from the development processing unit is recorded.

[0011] A video control program according to one embodiment of the technology of the present disclosure is a video control program for a video control device that includes a storage unit for temporarily storing pixel data output from an imaging unit, and causes the processor of the video control device to output video data obtained by performing a development process including demosaicing on the pixel data stored in the storage unit, and to switch between a first recording mode in which only the video data is used as recording data from the video data and the pixel data stored in the storage unit, and a second recording mode in which both the video data and the pixel data stored in the storage unit are used as recording data, and to control the imaging unit to perform imaging with different exposure values ​​in the first recording mode and the second recording mode. [Effects of the Invention]

[0012] According to one embodiment of the technology of this disclosure, it is possible to provide a video control device, a video recording device, a video control method, a video recording method, and a video control program that can perform imaging with an appropriate exposure value depending on whether or not pixel data is recorded before demosaicing. [Brief explanation of the drawing]

[0013] [Figure 1] This figure shows an example of an imaging device 100 according to Embodiment 1. [Figure 2] This flowchart shows an example of processing performed by the imaging device 100 according to Embodiment 1. [Figure 3] This figure shows an example of the gamma characteristics of BT.709. [Figure 4] This figure shows the relationship between the subject reflectance and the signal value after gamma correction in the gamma characteristic 31 shown in Figure 3. [Figure 5] This flowchart shows an example of processing performed by the imaging device 100 according to Embodiment 2. [Figure 6] This figure shows an example of a gamma characteristic that is brighter than that of BT.709. [Figure 7] This figure shows the relationship between the subject reflectance and the signal value after gamma correction in the gamma characteristic 61 shown in Figure 6. [Figure 8] It is a diagram showing another example of gamma characteristics brighter than those of BT.709. [Figure 9] It is a diagram showing the relationship between the subject reflectance and the signal value after gamma correction in the gamma characteristic 81 shown in FIG. 8. [Figure 10] It is a flowchart showing an example of the processing by the imaging device 100 according to Embodiment 3. [Figure 11] It is a diagram showing an example of the imaging device 100 according to Embodiment 4.

Embodiments for Carrying Out the Invention

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0015] (Embodiment 1) <Imaging device 100 according to Embodiment 1> FIG. 1 is a diagram showing an example of the imaging device 100 according to Embodiment 1. The imaging device 100 is a video control device capable of generating a video by performing continuous imaging. However, the imaging device 100 may have a function of generating a still image in addition to the function of generating a video.

[0016] The external recording device 120 is provided outside the imaging device 100 and records the RAW pixel data and video data described later output from the imaging device 100. The data output terminal of the imaging device 100 is connected to the input terminal of the external recording device 120 via a communication interface such as HDMI (High-Definition Multimedia Interface). Note that HDMI is a registered trademark. The data output from the imaging device 100 is not limited to HDMI, and may be output using wireless communication (for example, UWB (Ultra Wide Band) or wireless HDMI-SDI (High-Definition Multimedia Interface-Serial Digital Interface)).

[0017] The imaging device 100 includes an imaging unit 119, an imaging control unit 104, a RAW correction unit 105, a temporary storage unit 106, a development processing unit 107, a monitor 108, an output control unit 109, and an external output I / F 110.

[0018] The imaging unit 119 generates video image data as a series of temporally continuous RAW pixel data by performing continuous imaging. Specifically, the imaging unit 119 includes an imaging lens system 101, an image sensor 102, and an ADC (Analog / Digital Converter) 103. The ADC 103 may be built into the image sensor 102.

[0019] The imaging lens system 101 includes lenses for transmitting light from the subject and forming an image on the image sensor 102. The imaging lens system 101 may also include an aperture, an ND (Neutral Density) filter, a focus lens, a zoom lens, a shift lens, etc. The movable parts of these imaging lens systems 101 are controlled by the imaging control unit 104.

[0020] The image sensor 102 converts the optical image from the imaging lens system 101 into an analog image signal and outputs the converted analog image signal to the ADC 103. The image sensor 102 is composed of an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) type image sensor or a CCD (Charge-Coupled Device) type image sensor.

[0021] Furthermore, the image sensor 102 is equipped with an electronic shutter, which is a shutter for adjusting the exposure time. Image capture by the image sensor 102 is controlled by the image control unit 104. For example, when capturing video, the image sensor 102 continuously captures images in time under the control of the image control unit 104, and sequentially outputs the resulting analog image signals to the ADC 103.

[0022] The ADC103 converts the analog image signal from the image sensor 102 into digital RAW pixel data and outputs the converted RAW pixel data to the RAW correction unit 105. The RAW pixel data continuously output from the ADC103 is the pixel data before demosaicing.

[0023] The imaging control unit 104 controls imaging by the imaging lens system 101 and the image sensor 102. For example, the imaging control unit 104 controls the focus, exposure, etc. of imaging by the imaging unit 119 in response to instructions from the user. Alternatively, the imaging control unit 104 may automatically control the exposure, etc. of imaging by the imaging unit 119 based on the demosaicing image obtained by the development processing unit 107, which will be described later.

[0024] Furthermore, the imaging control unit 104 generates metadata indicating the imaging conditions by the imaging lens system 101 and the image sensor 102, and outputs the generated metadata to the temporary storage unit 106. The temporary storage unit 106 is basically composed of memory (any type). The metadata is used when performing development processing to generate a demosaicing image based on RAW pixel data. For example, metadata may include black offset level, coefficient for converting RAW values ​​to a color system, white balance parameters, lens correction parameters, color conversion parameters, gamma correction parameters, noise correction parameters, time code, imaging date and time, and product name.

[0025] The RAW correction unit 105 performs corrections on the RAW pixel data output from the ADC 103. The corrections performed by the RAW correction unit 105 are applied to the RAW pixel data before demosaicing, and include, for example, pixel value correction, defective pixel correction, and shading correction according to the characteristics of the image sensor 102. The RAW correction unit 105 outputs the corrected RAW pixel data to the temporary storage unit 106.

[0026] The temporary storage unit 106 temporarily stores the RAW pixel data output from the RAW correction unit 105 and the metadata output from the imaging control unit 104. For example, the temporary storage unit 106 can be implemented using memory such as RAM (Random Access Memory). Alternatively, the temporary storage unit 106 may be implemented using multiple memories. For example, the temporary storage unit 106 may be implemented using a memory for storing RAW pixel data and a memory for storing metadata.

[0027] The development processing unit 107 generates a demosaiced video by performing development processing, including demosaicing, based on the RAW pixel data and metadata stored by the temporary storage unit 106, and outputs the generated demosaiced video to the monitor 108. In RAW pixel data that has undergone defective pixel correction and shading correction, each individual pixel has only one of the colors R, G, or B. Therefore, the other two colors are interpolated from surrounding pixels so that each pixel has data for three colors. For example, if a pixel only has the R color, then there is no G and B data, so the G color data for that pixel is interpolated from the data of surrounding G pixels or predicted from surrounding G pixels. As a result, every pixel has all three colors: R, G, and B. This is called demosaicing.

[0028] Furthermore, the development processing unit 107 generates video data by performing various image processing operations on the image data generated by demosaicing, and outputs the generated video data to the monitor 108. The image processing performed by the development processing unit 107 after demosaicing includes, for example, gain correction, gamma correction, peripheral light falloff correction, color correction, edge enhancement, noise reduction, edge enhancement, debayering, and compression.

[0029] For example, the development processing unit 107 generates video data compatible with BT.709 by performing various image processing operations on the image data. BT.709 is a video standard commonly used in televisions and monitors. In this case, as one of the image processing operations, the development processing unit 107 performs gamma correction based on gamma characteristics compatible with BT.709.

[0030] The monitor 108 displays the video to the user based on the video data output from the development processing unit 107. This allows the user to view the video being captured as a live image while video is being captured.

[0031] The output control unit 109 controls the output of the demosaicing video data corrected by the development processing unit 107 and the RAW pixel data before demosaicing stored in the temporary storage unit 106. The output control unit 109 also constitutes a mode switching control unit of one embodiment of the technology of this disclosure, which switches between a first recording mode and a second recording mode.

[0032] The first recording mode is a mode in which only video data from among video data and RAW pixel data is recorded as recording data (data to be recorded). The second recording mode is a mode in which both video data and RAW pixel data are recorded as recording data. Recording data refers to the data to be recorded on the recording medium 123, which will be described later. For example, the output control unit 109 may switch between the first recording mode and the second recording mode based on instructions from the user, or it may do so automatically based on various data such as the free space on the recording medium 123, which will be described later.

[0033] In the first recording mode, the output control unit 109 outputs the demosaicing video data corrected by the development processing unit 107 to the external recording device 120 via the external output I / F 110. In addition, in the first recording mode, the output control unit 109 does not output the RAW pixel data stored in the temporary storage unit 106 to the external recording device 120.

[0034] In the second recording mode, the output control unit 109 outputs the demosaiced video data corrected by the development processing unit 107 and the RAW pixel data before demosaicing stored in the temporary storage unit 106 to the external recording device 120 via the external output interface 110. Furthermore, when outputting the RAW pixel data via the external output interface 110, the output control unit 109 adds the metadata stored in the temporary storage unit 106 to the RAW pixel data.

[0035] Furthermore, the output control unit 109 controls the imaging unit 119 to capture images with different exposure values ​​for the first recording mode and the second recording mode. Specifically, the output control unit 109 controls the imaging unit 119 via the imaging control unit 104 so that the exposure value for imaging in the second recording mode is lower than the exposure value for imaging in the first recording mode. The exposure value can be controlled, for example, by adjusting at least one of the aperture value (F-number), exposure time (shutter speed), and ISO (International Organization for Standardization) sensitivity.

[0036] For example, the output control unit 109 controls the imaging unit 119 to perform imaging using a first exposure value in the first recording mode. The first exposure value is, for example, an exposure value that yields video data with appropriate brightness when generating BT.709 video data.

[0037] Furthermore, in the second recording mode, the output control unit 109 controls the imaging unit 119 to perform imaging with a second exposure value that is lower (darker) than the first exposure value. The second exposure value is, for example, the amount of brightness increase due to the gain correction performed in the second recording mode, which will be described later, and is lower than the first exposure value.

[0038] Furthermore, the output control unit 109 controls the development processing performed by the development processing unit 107 so that it differs between the first recording mode and the second recording mode. For example, the output control unit 109 controls the image processing related to brightness performed by the development processing unit 107 so that it differs between the first recording mode and the second recording mode.

[0039] The image processing performed by the development processing unit 107 regarding brightness may include, for example, gamma correction, gain correction, LUT processing, or a combination of several of these processes. LUT processing is a process that takes the three primary colors RGB as input values ​​and converts them into predetermined output values ​​for each input value. For example, there are LUTs that use luminance Y as input and output, and 3D LUTs that use RGB as input and output. Furthermore, this image processing related to brightness may be performed before or after demosaicing among the development processes performed by the development processing unit 107.

[0040] The output control unit 109 controls the system so that the increase in brightness of the video data due to the brightness-related image processing performed by the development processing unit 107 in the second recording mode is greater than the increase in brightness of the video data due to the brightness-related image processing performed by the development processing unit 107 in the first recording mode.

[0041] A large increase in brightness means, for example, that the increase in output value is large for all input values. However, a large increase in brightness may also mean, for example, that the average increase in output value relative to input values ​​is large, and that the output value may be small for some input values.

[0042] For example, the output control unit 109 causes the development processing unit 107 to perform the same gamma correction using the gamma characteristics corresponding to BT.709 in both the first and second recording modes. Furthermore, the output control unit 109 causes the development processing unit 107 to perform gain correction to increase brightness in the second recording mode, while this gain correction is not performed by the development processing unit 107 in the first recording mode. As a result, the increase in brightness of the video data is greater in the second recording mode than in the first recording mode due to the gain correction.

[0043] For example, this gain correction is a gain correction that, in the second recording mode, uses RAW pixel data obtained by imaging with a second exposure value lower than the first exposure value to obtain BT.709 video data with the appropriate brightness.

[0044] The external output interface 110 is a communication interface for communicating with the external recording device 120. For example, the external output interface 110 communicates via HDMI. The external output interface 110 outputs RAW pixel data and video data to the external recording device 120 according to the control from the output control unit 109.

[0045] The imaging device 100 may also be equipped with an internal memory for storing video data obtained by the development processing unit 107. Furthermore, the imaging device 100 may be equipped with a user interface for receiving various instructions from the user and outputting various data to the user.

[0046] Furthermore, the imaging device 100 may be equipped with a microphone that converts ambient sound into electrical signals. In this case, the electrical signals obtained by the microphone may be converted into digital acoustic data and output together with RAW pixel data and video data from the external output I / F 110 to the external recording device 120, where they may be recorded.

[0047] The external recording device 120 comprises an external input interface 121, a recording control unit 122, and a recording medium 123. The external input interface 121 acquires RAW pixel data and video data output from the external output interface 110 of the imaging device 100, and outputs the acquired RAW pixel data and video data to the recording control unit 122.

[0048] The recording control unit 122 controls the recording of RAW pixel data and video data output from the external input I / F 121 onto the recording medium 123. The recording medium 123 is a high-capacity recording medium capable of high-speed writing so that it can record the large amount of RAW pixel data and video data continuously output from the imaging device 100 in real time. For example, the recording medium 123 can be realized by a memory card or an SSD (Solid State Drive).

[0049] In the first recording mode, video data is sequentially recorded on the recording medium 123, but RAW pixel data is not recorded. In the second recording mode, video data and RAW pixel data are recorded in parallel on the recording medium 123. For example, in the second recording mode, video data and RAW pixel data may be recorded on the recording medium 123 simultaneously, or video data and RAW pixel data may be recorded with a time difference, or video data and RAW pixel data may be recorded alternately.

[0050] Although not shown in the diagram, the external recording device 120 has an external output interface that outputs RAW pixel data and video data stored on the recording medium 123 to an external data processing device (e.g., a personal computer) that is different from the imaging device 100 and the external recording device 120. This allows the data processing device to manage the RAW pixel data and video data stored on the recording medium 123 and to perform development processing based on the RAW pixel data.

[0051] Alternatively, the external recording device 120 may be a data processing device (e.g., a personal computer) having a processor and memory for developing the image. In this case, the external recording device 120 can manage the RAW pixel data and video data stored in the recording medium 123, and perform developing the image based on the RAW pixel data.

[0052] <Hardware configuration of imaging device 100> The imaging control unit 104, RAW correction unit 105, development processing unit 107, and output control unit 109 in the imaging device 100 are implemented by a processor that operates in coordination with the memory of the imaging device 100.

[0053] This processor is, for example, a CPU (Central Processing Unit), MPU (Micro Processing Unit), FPGA (Field Programmable Gate Array), PLD (Programmable Logic Device), or ASIC (Application Specific Integrated Circuit). This processor functions as the processing unit in the imaging device 100 by reading and executing a program stored in memory. More specifically, the structure of the above-mentioned various processors is an electrical circuit combining circuit elements such as semiconductor elements. This processor may be a combination of multiple processors of the same or different types.

[0054] Memory is implemented using RAM, ROM (Read Only Memory), flash memory, etc. Memory stores programs executed by the processor, or data used by the processor. This memory may be a combination of multiple memory types, either identical or different.

[0055] <Processing by the imaging device 100 according to Embodiment 1> Figure 2 is a flowchart showing an example of processing by the imaging device 100 according to Embodiment 1. The imaging device 100, for example, executes the processing shown in Figure 2 for each frame of imaging while capturing a video. The processing shown in Figure 2 is executed, for example, by the output control unit 109.

[0056] First, the output control unit 109 determines whether the current mode is the first recording mode (step S21). If it is the first recording mode (step S21: Yes), the output control unit 109 controls the imaging unit 119 via the imaging control unit 104 to capture one frame at the first exposure value corresponding to BT.709 (step S22).

[0057] Next, the output control unit 109 controls the development processing unit 107 to perform development processing, including gamma correction using the gamma characteristics of BT.709, based on the RAW pixel data obtained in step S22, and generates BT.709 video data (step S23).

[0058] Next, the output control unit 109 records the video data generated in step S23 (step S24) and completes a series of processes related to that frame. In step S24, the output control unit 109 outputs the video data from the external output I / F 110 to the external recording device 120, thereby recording the video data on the recording medium 123 of the external recording device 120.

[0059] In step S21, if the recording mode is not the first recording mode (step S21: No), i.e., the recording mode is the second recording mode, the output control unit 109 controls the imaging unit 119 via the imaging control unit 104 to capture one frame at a second exposure value lower than the first exposure value (step S25).

[0060] Next, the output control unit 109 records the RAW pixel data obtained in step S25 (step S26). In step S26, the output control unit 109 outputs the RAW pixel data from the external output I / F 110 to the external recording device 120, thereby recording the RAW pixel data on the recording medium 123 of the external recording device 120.

[0061] Furthermore, the output control unit 109 performs gain correction on the RAW pixel data obtained in step S25 to increase brightness (step S27). Next, the output control unit 109 controls the development processing unit 107 to perform development processing, including gamma correction using the gamma characteristics of BT.709, based on the RAW pixel data that has undergone gain correction in step S27, and generates BT.709 video data (step S28). Next, the output control unit 109 moves to step S24, records the video data generated in step S28, and ends the series of processing related to that frame.

[0062] <Gamma characteristics of BT.709> FIG. 3 is a diagram showing an example of the gamma characteristics of BT.709. In FIG. 3, the horizontal axis represents the signal value of RAW pixel data, and the vertical axis represents the signal value of video data after gamma correction of BT.709. The gamma characteristic 31 is the gamma characteristic in the gamma correction of BT.709. In the development processes of steps S23 and S28 shown in FIG. 2, the imaging device 100 performs gamma correction using, for example, the gamma characteristic 31.

[0063] FIG. 4 is a diagram showing the relationship between the subject reflectance and the signal value after gamma correction in the gamma characteristic 31 shown in FIG. 3. In FIG. 4, the horizontal axis represents the subject reflectance [%] (i.e., the brightness of the subject being imaged), and the vertical axis represents the signal value of video data after gamma correction of BT.709.

[0064] The signal value characteristic 41 is the characteristic of the signal value of video data after gamma correction of BT.709 when imaging is performed at the above-mentioned first exposure value corresponding to BT.709. The signal value characteristic 41 has, for example, a signal value at a subject reflectance 43 of 18% gray close to 0.5, indicating a characteristic of appropriate brightness.

[0065] The signal value characteristic 42 is shown for reference as the characteristic of the signal value of video data after gamma correction of BT.709 when imaging is performed at a second exposure value lower than the above-mentioned first exposure value. The signal value characteristic 42 has a darker characteristic compared to the signal value characteristic 41 as the exposure value of imaging is lower.

[0066] In the first recording mode, since imaging is performed at the first exposure value, video data with appropriate brightness such as the signal value characteristic 41 can be obtained.

[0067] In the second recording mode, imaging is performed at a second exposure value lower than the first exposure value. However, in addition to gamma correction based on the gamma characteristics 31, gain correction is performed to increase brightness, so that video data with appropriate brightness is recorded, just like in the first recording mode. Furthermore, for RAW pixel data, RAW pixel data with a wide dynamic range, such as the signal value characteristics 42, is recorded. This increases the degree of freedom in processing during development after the data is stored on the recording medium 123.

[0068] Thus, according to the imaging device 100 of Embodiment 1, the imaging unit 119 performs imaging with different exposure values ​​in a first recording mode in which only video data is recorded as the recording data, and in a second recording mode in which both video data and RAW pixel data are recorded as the recording data.

[0069] This allows for imaging with an appropriate first exposure value in the first recording mode, which does not store RAW pixel data, and imaging with a lower second exposure value that yields RAW pixel data with greater flexibility in post-processing in the second recording mode, which does store RAW pixel data. Therefore, imaging can be performed with an appropriate exposure value depending on whether or not RAW pixel data is being recorded.

[0070] By obtaining RAW pixel data with a wide dynamic range and high degree of processing flexibility, it becomes possible to generate video data capable of expressing a wide range of brightness, such as HLG (Hybrid Log Gamma), through subsequent development processing.

[0071] Furthermore, the imaging device 100 may have different development processes for generating demosaiced video data between the first recording mode and the second recording mode. For example, the imaging device 100 may have different image processing related to brightness included in the development process between the first recording mode and the second recording mode.

[0072] This allows the increase in brightness of the video data due to image processing related to brightness in the second recording mode to be greater than the increase in brightness of the video data due to image processing related to brightness in the first recording mode, thereby suppressing the decrease in brightness of the video data obtained in the second recording mode. In other words, in the second recording mode, it is possible to obtain RAW pixel data with a high degree of freedom in processing during development while suppressing the decrease in brightness of the video data.

[0073] Although the configuration described above involves performing gain correction in the second recording mode (for example, step S27 in Figure 2) on the RAW pixel data before demosaicing, the imaging device 100 may also perform gain correction in the second recording mode on the image data after demosaicing.

[0074] (Embodiment 2) The differences between Embodiment 2 and Embodiment 1 will now be described.

[0075] <Processing by the imaging device 100 according to Embodiment 2> Figure 5 is a flowchart showing an example of processing by the imaging device 100 according to Embodiment 2. The imaging device 100 according to Embodiment 2, for example, executes the processing shown in Figure 5 for each frame of imaging while capturing video. The processing shown in Figure 5 is executed, for example, by the output control unit 109.

[0076] Steps S51 to S57 shown in Figure 5 are the same as steps S21 to S26 and S28 shown in Figure 2. However, in the development process of step S57, the output control unit 109 performs gamma correction using a gamma characteristic brighter than that of BT.709 based on the RAW pixel data obtained in step S55, and generates BT.709 video data (step S57). A gamma characteristic brighter than that of BT.709 is, for example, a gamma characteristic in which the average ratio of output values ​​to input values ​​is higher than that of BT.709 (see, for example, Figure 6).

[0077] That is, in the process shown in FIG. 5, instead of performing the process of performing gain correction on RAW pixel data (for example, step S27 in FIG. 2), the gamma characteristic used for the development process in the second recording mode is made brighter than the gamma characteristic used for the development process in the first recording mode.

[0078] <Gamma characteristic brighter than the gamma characteristic of BT.709> FIG. 6 is a diagram showing an example of a gamma characteristic brighter than the gamma characteristic of BT.709. In FIG. 6, the same parts as those shown in FIG. 3 are denoted by the same reference numerals and the description thereof is omitted. The gamma characteristic 61 is a gamma characteristic brighter than the gamma characteristic 31 of BT.709. Specifically, the gamma characteristic 61 is obtained by compressing the gamma characteristic 31 in the horizontal axis direction.

[0079] For example, the output control unit 109 uses the gamma characteristic 31 for the correction of the development process in step S53 shown in FIG. 5 (that is, the development process in the first recording mode), and uses the gamma characteristic 61 for the correction of the development process in step S57 shown in FIG. 5 (that is, the development process in the second recording mode).

[0080] FIG. 7 is a diagram showing the relationship between the subject reflectance and the signal value after gamma correction in the gamma characteristic 61 shown in FIG. 6. In FIG. 7, the same parts as those shown in FIG. 4 are denoted by the same reference numerals and the description thereof is omitted.

[0081] The signal value characteristic 71 is a characteristic of the signal value of the moving image data after gamma correction using the gamma characteristic 61 when imaging is performed with a second exposure value lower than the first exposure value. The signal value characteristic 71 has the same characteristic as the signal value characteristic 41 when imaging is performed with the above first exposure value corresponding to BT.709. Therefore, as in the first embodiment, moving image data with appropriate brightness is also recorded in the second recording mode.

[0082] Figure 8 shows another example of a gamma characteristic that is brighter than that of BT.709. In Figure 8, parts that are the same as those shown in Figure 3 are denoted by the same reference numerals and their explanations are omitted. Gamma characteristic 81 is a gamma characteristic that is brighter than the gamma characteristic 31 of BT.709. Specifically, gamma characteristic 81 has a steeper rise in the dark region compared to gamma characteristic 31.

[0083] For example, the output control unit 109 uses the gamma characteristic 31 to correct the development process in step S53 shown in Figure 5 (i.e., the development process for the first recording mode), and uses the gamma characteristic 81 to correct the development process in step S57 shown in Figure 5 (i.e., the development process for the second recording mode).

[0084] Figure 9 shows the relationship between the subject reflectance and the signal value after gamma correction in the gamma characteristic 81 shown in Figure 8. In Figure 9, parts that are the same as those shown in Figure 4 are denoted by the same reference numerals and their explanations are omitted.

[0085] The signal value characteristic 91 is the signal value characteristic of the video data after gamma correction using the gamma characteristic 81, performed by imaging at a second exposure value lower than the first exposure value described above. The signal value characteristic 91 has a brightness suitable for BT.709 while having a wide dynamic range. Therefore, as with Embodiment 1, video data with appropriate brightness is recorded in the second recording mode as well. Furthermore, this video data has a wide dynamic range, allowing for greater flexibility in processing during development.

[0086] Thus, according to the imaging device 100 of Embodiment 2, by performing different gamma corrections for the first recording mode and the second recording mode, it is possible to obtain RAW pixel data with a high degree of freedom for processing during development in the second recording mode, while suppressing a decrease in the brightness of the video data, similar to the imaging device 100 of Embodiment 1.

[0087] (Embodiment 3) The differences between Embodiment 3 and Embodiments 1 and 2 will be described below.

[0088] <Processing by the imaging device 100 according to Embodiment 3> Figure 10 is a flowchart showing an example of processing by the imaging device 100 according to Embodiment 3. The imaging device 100 according to Embodiment 3, for example, executes the processing shown in Figure 10 for each frame of imaging while capturing video. The processing shown in Figure 10 is executed, for example, by the output control unit 109.

[0089] Steps S101 to S108 shown in Figure 10 are the same as steps S21 to S28 shown in Figure 2. However, in the development process of steps S103 and S108, the output control unit 109 performs noise reduction and edge enhancement processing.

[0090] Furthermore, in the development process of step S108, the output control unit 109 performs noise reduction that is more effective in reducing noise than the noise reduction in the development process of step S103. This allows for appropriate noise reduction to be applied to the noise amplified by the gain correction in step S107.

[0091] Furthermore, in the development process of step S108, the output control unit 109 performs noise reduction with a weaker edge enhancement effect than the edge enhancement in the development process of step S103. This prevents further deterioration of the image quality of the image, which has been amplified by the gain correction in step S107, due to edge enhancement.

[0092] Thus, the imaging device 100 according to Embodiment 3 performs different noise processing for the first recording mode and the second recording mode. This makes it possible to perform appropriate noise reduction on the noise in the second recording mode, which is greater than that in the first recording mode.

[0093] Furthermore, the imaging device 100 according to Embodiment 3 performs different contour processing for the first recording mode and the second recording mode. This makes it possible to suppress the deterioration of image quality in video data in the second recording mode, which has more noise than the first recording mode, due to contour enhancement.

[0094] (Embodiment 4) The differences between Embodiment 4 and Embodiments 1 to 3 will be described below.

[0095] <Imaging device 100 according to Embodiment 4> Figure 11 shows an example of an imaging device 100 according to Embodiment 4. In Figure 11, parts that are the same as those shown in Figure 1 are denoted by the same reference numerals and their descriptions are omitted.

[0096] The imaging device 100 according to Embodiment 4 is a video recording device that, instead of the external output I / F 110 shown in Figure 1, is equipped with an internal output I / F 124, an internal input I / F 125, a recording control unit 122, and a recording medium 123.

[0097] The internal output I / F 124 is an interface such as HDMI, similar to the external output I / F 110 shown in Figure 1, but differs from the external output I / F 110 in that it communicates with the internal input I / F 125 located inside the imaging device 100. The internal input I / F 125 is an interface such as HDMI, similar to the external input I / F 121 of the external recording device 120 shown in Figure 1, but differs from the external input I / F 121 in that it is located inside the imaging device 100.

[0098] The recording control unit 122 and recording medium 123 shown in Figure 11 have the same configuration as the recording control unit 122 and recording medium 123 shown in Figure 1, but are located inside the imaging device 100. In other words, the imaging device 100 shown in Figure 11 has a built-in high-speed, high-capacity recording medium 123, and outputs RAW pixel data and video data to the recording medium 123 using an interface inside the imaging device 100.

[0099] In this case, for example, in step S24 shown in Figure 2, the output control unit 109 records the video data on the recording medium 123 of the imaging device 100 by outputting the video data from the internal output I / F 124 to the internal input I / F 125. Also, in step S26 shown in Figure 2, the output control unit 109 records the RAW pixel data on the recording medium 123 of the imaging device 100 by outputting the RAW pixel data from the internal output I / F 124 to the internal input I / F 125.

[0100] Thus, even in a configuration in which the imaging device 100 incorporates a recording medium 123 for recording RAW pixel data and video data, the same effects as those of the imaging device 100 according to Embodiment 1 can be obtained.

[0101] (Combinations of each embodiment) The embodiments described above can also be implemented in combination. For example, in Embodiment 2, different gamma correction may be performed for the first recording mode and the second recording mode, while gain correction as in Embodiment 1 may be performed. Also, in Embodiment 2, noise processing and contour processing as in Embodiment 3 may be performed. Furthermore, it is possible to implement the same processing as in Embodiments 2 and 3 in the configuration of the imaging device 100 according to Embodiment 4.

[0102] (modified version) Although BT.709 was used as an example of a video data format in this explanation, the video data format is not limited to BT.709; various other video formats can be used.

[0103] As explained above, this specification contains at least the following information:

[0104] (1) A storage unit that temporarily stores pixel data output from the imaging unit, A development processing unit that outputs video data obtained by performing development processing, including demosaicing, on the pixel data stored in the above-mentioned storage unit, A mode switching control unit switches between a first recording mode in which only the video data is used as recording data from the video data and the pixel data stored in the storage unit, and a second recording mode in which both the video data and the pixel data stored in the storage unit are used as recording data, and controls the imaging unit to perform imaging with different exposure values ​​in the first recording mode and the second recording mode. A video control device equipped with the following features.

[0105] (2) (1) The video control device described above, The mode switching control unit controls the exposure value of the image in the second recording mode to be lower than the exposure value of the image in the first recording mode. Video control device.

[0106] (3) (1) or (2) The video control device described above, The mode switching control unit controls the development process so that it differs between the first recording mode and the second recording mode. Video control device.

[0107] (4) (3) The video control device described above, The mode switching control unit controls the image processing related to brightness included in the development process so that it differs between the first recording mode and the second recording mode. Video control device.

[0108] (5) (4) The video control device described above, The mode switching control unit controls the system so that the increase in brightness of the video data due to the brightness-related image processing in the second recording mode is greater than the increase in brightness of the video data due to the brightness-related image processing in the first recording mode. Video control device.

[0109] (6) (4) or (5) The video control device described above, The above image processing related to brightness includes gamma correction. Video control device.

[0110] (7) A video control device according to any one of (4) to (6), The above image processing related to brightness includes gain correction. Video control device.

[0111] (8) A video control device according to any one of (4) to (7), The above image processing related to brightness includes LUT processing. Video control device.

[0112] (9) A video control device according to any one of (1) to (8), The mode switching control unit controls the development processing unit to perform different noise processing on the pixel data in the first recording mode and the second recording mode. Video control device.

[0113] (10) A video control device according to any one of (1) to (9), The mode switching control unit controls the development processing unit to perform different contour processing on the pixel data in the first recording mode and the second recording mode. Video control device.

[0114] (11) A video control device according to any one of (1) to (10), Output unit that outputs the above recorded data A video control device equipped with the following features.

[0115] (12) A video control device as described in any one of (1) to (10), A recording unit for recording the above-mentioned data, A video recording device equipped with the following features.

[0116] (13) A video control method for a video control device that includes a storage unit for temporarily storing pixel data output from an imaging unit, The pixel data stored in the above-mentioned memory unit is subjected to development processing, including demosaicing, to obtain video data, which is then output. The system switches between a first recording mode, in which only the video data is used as recording data from the video data and the pixel data stored in the storage unit, and a second recording mode, in which both the video data and the pixel data stored in the storage unit are used as recording data. The system controls the imaging unit to perform imaging with different exposure values ​​in the first recording mode and the second recording mode. Video control methods.

[0117] (14) (13) The video control method described above, The system controls the exposure value of the image in the second recording mode to be lower than the exposure value of the image in the first recording mode. Video control methods.

[0118] (15) (13) or (14) The video control method described above, The development process is controlled to be different between the first recording mode and the second recording mode. Video control methods.

[0119] (16) (15) The video control method described above, The image processing related to brightness in the development process is controlled to be different between the first recording mode and the second recording mode. Video control methods.

[0120] (17) (16) The video control method described above, The control is performed so that the increase in brightness of the video data due to the image processing related to brightness in the second recording mode is greater than the increase in brightness of the video data due to the image processing related to brightness in the first recording mode. Video control methods.

[0121] (18) A video control method according to (16) or (17), The above image processing related to brightness includes gamma correction. Video control methods.

[0122] (19) A video control method described in any one of (16) to (18), The above image processing related to brightness includes gain correction. Video control methods.

[0123] (20) A video control method according to any one of (16) to (19), The above image processing related to brightness includes LUT processing. Video control methods.

[0124] (twenty one) A video control method described in any one of (13) to (20), The noise processing included in the development process differs between the first recording mode and the second recording mode described above. Video control methods.

[0125] (twenty two) A video control method described in any one of (13) to (21), The contour processing included in the development process differs between the first recording mode and the second recording mode described above. Video control methods.

[0126] (twenty three) A video control method described in any one of (13) to (22), An output unit that outputs the above recorded data, A video control method including...

[0127] (twenty four) In the video control method described in any one of (13) to (22), Record the above data. A video recording method that includes this.

[0128] (twenty five) A video control program for a video control device that includes a storage unit for temporarily storing pixel data output from an imaging unit, The processor of the above video control device, The pixel data stored in the above-mentioned memory unit is subjected to development processing, including demosaicing, to obtain video data, which is then output. The system switches between a first recording mode, in which only the video data is used as recording data from the video data and the pixel data stored in the storage unit, and a second recording mode, in which both the video data and the pixel data stored in the storage unit are used as recording data. The system controls the imaging unit to perform imaging with different exposure values ​​in the first recording mode and the second recording mode. A video control program for executing processing. [Explanation of symbols]

[0129] 31,61,81 Gamma characteristics 41, 42, 71, 91 Signal Value Characteristics 43 Subject reflectance 100 Imaging device 101 Imaging lens system 102 Image sensor 103 ADC 104 Imaging control unit 105 RAW Correction Section 106 Temporary storage 107 Development Processing Unit 108 monitors 109 Output Control Unit 110 External Output Interface 119 Imaging Unit 120 External recording device 121 External Input Interface 122 Recording Control Unit 123 Recording media 124 Internal Output Interface 125 Internal Input Interface

Claims

1. The system includes a first recording mode that records only video data obtained by developing the pixel data of a video based on the output from the imaging unit, and a second recording mode that records the video data and the pixel data, and a mode control unit that controls the imaging unit to perform imaging at a lower exposure value in the second recording mode than in the first recording mode. The mode control unit controls the system so that the amount of increase in brightness of the video data due to the brightness-related image processing included in the development process in the second recording mode is greater than the amount of increase in brightness of the video data due to the brightness-related image processing included in the development process in the first recording mode. The mode control unit controls the noise reduction process included in the development process in the second recording mode to have a stronger noise reduction effect than the noise reduction process included in the development process in the first recording mode. Video control device.

2. A video control device according to claim 1, The aforementioned image processing related to brightness includes gain correction. Video control device.

3. A video control device according to claim 1, The mode control unit controls the contour enhancement effect of the contour enhancement process included in the development process in the first recording mode to be stronger than the contour enhancement effect of the contour enhancement process included in the development process in the second recording mode. Video control device.

4. A video control device according to any one of claims 1 to 3, The mode control unit controls the image processing related to brightness according to whether it is the first recording mode or the second recording mode. Video control device.

5. The system includes a first recording mode that records only video data obtained by developing the pixel data of a video based on the output from the imaging unit, and a second recording mode that records the video data and the pixel data, wherein the imaging unit is controlled to perform imaging at a lower exposure value in the second recording mode than in the first recording mode. A video control method, The control is performed such that the amount of increase in brightness of the video data due to the brightness-related image processing included in the development process in the second recording mode is greater than the amount of increase in brightness of the video data due to the brightness-related image processing included in the development process in the first recording mode. The noise reduction effect of the noise reduction process included in the development process in the second recording mode is controlled to be stronger than that of the noise reduction process included in the development process in the first recording mode. Video control methods.

6. A video control method according to claim 5, The image processing related to brightness is controlled according to whether the first recording mode or the second recording mode is selected. Video control methods.

7. On the computer, The system switches between a first recording mode, which records only video data obtained by developing the pixel data of the video output from the imaging unit, and a second recording mode, which records the video data and the pixel data, and controls the imaging unit to perform imaging at a lower exposure value in the second recording mode than in the first recording mode. The control is performed such that the amount of increase in brightness of the video data due to the brightness-related image processing included in the development process in the second recording mode is greater than the amount of increase in brightness of the video data due to the brightness-related image processing included in the development process in the first recording mode. The noise reduction effect of the noise reduction process included in the development process in the second recording mode is controlled to be stronger than that of the noise reduction process included in the development process in the first recording mode. A video control program for executing processing.

8. A video control program according to claim 7, To the aforementioned computer, The image processing related to brightness is controlled according to whether the first recording mode or the second recording mode is selected. A video control program for executing processing.

Images