Video control device, video recording device, video control method, video recording method, and video control program

Abstract

Provided are a video control device, a video recording device, a video control method, a video recording method, and a video control program with which it is possible to perform imaging using an appropriate exposure value corresponding to whether or not pixel data prior to demosaic processing are to be recorded. A temporary storage unit 106 temporarily stores pixel data output from an imaging unit 119. A development processing unit 107 outputs video data obtained by subjecting the pixel data stored in the temporary storage unit 106 to development processing including demosaic processing. An output control unit 109 switches between a first recording mode in which only the video data, among the video data and the pixel data, are used as recording data, and a second recording mode in which the video data and the pixel data are used as the recording data. Further, the output control unit 109 performs control such that the imaging unit 119 performs imaging using different exposure values in the first recording mode and the second recording mode.

Description

[Technical field]

[0001] The present invention relates to a video control device, a video recording device, a video control method, a video recording method, and a video control program. [Background technology]

[0002] Patent Document 1 describes a camera system having a recording mode that stores RAW data and YC data for thumbnail images converted from the RAW data.

[0003] Patent Document 2 describes an imaging device that, when a user presses a RAW video recording start button while recording a FullHD (Full High Definition) video, starts recording the RAW video while continuing recording the FullHD video.

[0004] Patent Document 3 describes an imaging device that switches between a first mode in which a moving image file is recorded by a digitally developed image together with RAW data of the image, and a second mode in which the digitally developed image is recorded. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2007-166193 A [Patent Document 2] JP 2017-28466 A [Patent Document 3] JP 2010-21710 A Summary of the Invention

[0006] One embodiment of the technology disclosed herein provides a video control device, a video recording device, a video control method, a video recording method, and a video control program that can capture images using an appropriate exposure value depending on whether or not pixel data before demosaic processing is recorded. [Means for solving the problem]

[0007] A video imaging device according to one embodiment of the technology disclosed herein includes a memory unit that temporarily stores pixel data output from an imaging unit, a development processing unit that outputs video data obtained by performing development processing including demosaic processing on the pixel data stored in the memory unit, and a mode switching control unit that switches between a first recording mode in which only the video data is used as recording data out of the video data and the pixel data stored in the memory unit, and a second recording mode in which the video data and the pixel data stored in the memory unit are used as recording data, and controls the imaging unit to capture images using different exposure values ​​in the first recording mode and the second recording mode.

[0008] A moving image recording device according to one embodiment of the technique disclosed herein includes the moving image control device and a recording unit that records the recording data output from the development processing unit.

[0009] A video control method according to one embodiment of the technology disclosed herein is a video control method for a video control device having a memory unit that temporarily stores pixel data output from an imaging unit, which outputs video data obtained by performing a development process including a demosaic process on the pixel data stored in the memory unit, and switches between a first recording mode in which only the video data is used as recording data out of the video data and the pixel data stored in the memory unit, and a second recording mode in which the video data and the pixel data stored in the memory unit are used as recording data, and controls the imaging unit to capture images using different exposure values ​​in the first recording mode and the second recording mode.

[0010] A moving image recording method according to one embodiment of the technique of the present disclosure includes recording record data output from the development processing unit in the moving image control method.

[0011] A video control program according to one embodiment of the technology disclosed herein is a video control program for a video control device having a memory unit that temporarily stores pixel data output from an imaging unit, and causes a processor of the video control device to execute a process of outputting video data obtained by performing a development process including a demosaic process on the pixel data stored in the memory unit, switching between a first recording mode in which only the video data is used as recording data out of the video data and the pixel data stored in the memory unit, and a second recording mode in which the video data and the pixel data stored in the memory unit are used as recording data, and controlling the imaging unit to capture images using different exposure values ​​in the first recording mode and the second recording mode. Effect of the Invention

[0012] According to one embodiment of the technology of the present disclosure, it is possible to provide a video control device, a video recording device, a video control method, a video recording method, and a video control program that can capture images using an appropriate exposure value depending on whether or not pixel data before demosaic processing is recorded. [Brief description of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an example of an imaging device 100 according to a first embodiment. [Diagram 2] 4 is a flowchart showing an example of processing performed by the imaging device 100 according to the first embodiment. [Diagram 3] FIG. 1 is a diagram showing an example of the gamma characteristic of BT.709. [Figure 4] 4 is a diagram showing the relationship between the subject reflectance and the signal value after gamma correction in the gamma characteristic 31 shown in FIG. [Diagram 5] 11 is a flowchart showing an example of processing performed by the imaging device 100 according to the second embodiment. [Figure 6] FIG. 1 is a diagram showing an example of a gamma characteristic brighter than the gamma characteristic of BT.709. [Figure 7] 7 is a diagram showing the relationship between the subject reflectance and the signal value after gamma correction in the gamma characteristic 61 shown in FIG. 6. FIG. [Figure 8] FIG. 13 is a diagram showing another example of gamma characteristics brighter than the gamma characteristics of BT.709. [Figure 9] 9 is a diagram showing the relationship between the subject reflectance and the signal value after gamma correction in the gamma characteristic 81 shown in FIG. 8. FIG. [Figure 10] 13 is a flowchart showing an example of processing performed by the imaging device 100 according to the third embodiment. [Figure 11] FIG. 13 is a diagram illustrating an example of an imaging device 100 according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0015] (Embodiment 1) <Imaging device 100 according to embodiment 1> 1 is a diagram showing an example of an imaging device 100 according to embodiment 1. The imaging device 100 is a moving image control device capable of generating a moving image by performing continuous imaging. However, the imaging device 100 may have a function of generating still images in addition to a function of generating moving images.

[0016] The external recording device 120 is provided outside the imaging device 100, and records RAW pixel data and video data (described later) output from the imaging device 100. A data output terminal of the imaging device 100 is connected to an input terminal of the external recording device 120 via a communication interface such as HDMI (High-Definition Multimedia Interface). HDMI is a registered trademark. Data output from the imaging device 100 is not limited to HDMI, and may be output using wireless communication (for example, UWB (Ultra Wide Band) or wireless HDMI-SDI (High-Definition Multimedia Interface-Serial Digital Interface).

[0017] The imaging device 100 includes an imaging unit 119 , an imaging control unit 104 , a RAW correction unit 105 , a temporary storage unit 106 , a development processing unit 107 , a monitor 108 , an output control unit 109 , and an external output I / F 110 .

[0018] The imaging unit 119 performs continuous imaging to generate moving image data as multiple pieces of RAW pixel data that are continuous in time. Specifically, the imaging unit 119 has an imaging lens system 101, an imaging element 102, and an ADC (Analog / Digital Converter) 103. The ADC 103 may be built into the imaging element 102.

[0019] The imaging lens system 101 includes a lens for transmitting light from a subject and forming an image on the imaging element 102. The imaging lens system 101 may also include an aperture, an ND (Neutral Density) filter, a focus lens, a zoom lens, a shift lens, etc. These movable parts of the imaging lens system 101 are controlled by an imaging control unit 104.

[0020] The imaging element 102 converts an optical image captured by the imaging lens system 101 into an analog image signal, and outputs the converted analog image signal to the ADC 103. The imaging element 102 is configured by an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) type image sensor or a CCD (Charge-Coupled Device) type image sensor.

[0021] The image sensor 102 also includes an electronic shutter for adjusting the exposure time. Image capture by the image sensor 102 is controlled by an image capture control unit 104. For example, during moving image capture, the image sensor 102 captures images continuously over time under the control of the image capture control unit 104, and sequentially outputs the analog image signals obtained thereby to the ADC 103.

[0022] The ADC 103 converts the analog image signal from the image sensor 102 into digital RAW pixel data, and outputs the converted RAW pixel data to a RAW correction unit 105. The RAW pixel data continuously output from the ADC 103 is pixel data before demosaic processing.

[0023] The imaging control unit 104 controls the imaging lens system 101 and the image sensor 102 to control imaging by the imaging unit 119. For example, the imaging control unit 104 controls the focus, exposure, and the like of imaging by the imaging unit 119 in response to an instruction from a user. Furthermore, the imaging control unit 104 may automatically control the exposure, and the like of imaging by the imaging unit 119 based on a demosaic image obtained by a development processing unit 107 described below.

[0024] Furthermore, the imaging control unit 104 generates metadata indicating the conditions of imaging by the imaging lens system 101 and the imaging element 102, and outputs the generated metadata to the temporary storage unit 106. The temporary storage unit 106 is basically composed of a memory (any type). The metadata is used when performing development processing to generate a demosaic image based on the RAW pixel data. For example, the metadata includes a black offset level, a coefficient for converting a RAW numerical value into a color system, a white balance parameter, a lens correction parameter, a color conversion parameter, a gamma correction parameter, a noise correction parameter, a time code, an imaging date and time, a product name, and the like.

[0025] The RAW correction unit 105 performs correction on the RAW pixel data output from the ADC 103. The correction performed by the RAW correction unit 105 is correction performed on the RAW pixel data before demosaic processing, such as pixel value correction, defective pixel correction, shading correction, etc., according to the characteristics of the image sensor 102. The RAW correction unit 105 outputs the corrected RAW pixel data to the temporary storage unit 106.

[0026] The temporary storage unit 106 temporarily stores the RAW pixel data output from the RAW correction unit 105 and the metadata output from the imaging control unit 104. For example, the temporary storage unit 106 is realized by a memory such as a RAM (Random Access Memory). The temporary storage unit 106 may also be realized by a plurality of memories. For example, the temporary storage unit 106 may be realized by a memory that stores the RAW pixel data and a memory that stores the metadata.

[0027] The development processing unit 107 generates a demosaic moving image by performing development processing including demosaic processing based on the RAW pixel data and metadata stored by the temporary storage unit 106, and outputs the generated demosaic moving image to the monitor 108. In the RAW pixel data that has been subjected to defective pixel correction and shading correction, each pixel has only one of the colors R, G, and B. Therefore, the other two colors are complemented from the surrounding pixels so that each pixel has three-color data. For example, if a pixel only has R color, there is no G and B data, so the G color data of the pixel is complemented from the data of the surrounding G pixels or predicted from the data of the surrounding G pixels. As a result, each of all pixels has the three colors R, G, and B. This is called demosaic processing.

[0028] Furthermore, the development processing unit 107 generates moving image data by performing various image processing on the image data generated by the demosaic processing, and outputs the generated moving image data to the monitor 108. Image processing performed by the development processing unit 107 after the demosaic processing includes, for example, gain correction, gamma correction, peripheral light falloff correction, color correction, edge enhancement, noise reduction, edge enhancement, debayering, compression, and the like.

[0029] As an example, the development processor 107 performs various image processing on the image data to generate video data that conforms to BT.709. BT.709 is a video standard that is used in general televisions and monitors. In this case, the development processor 107 performs gamma correction based on gamma characteristics that conform to BT.709 as one of the image processes.

[0030] The monitor 108 displays to the user the moving image based on the moving image data output from the development processing unit 107. This allows the user to check the moving image being captured as a live image while the moving image is being captured.

[0031] The output control unit 109 controls the output of the demosaic processed video data corrected by the development processing unit 107 and the RAW pixel data before demosaic processing stored in the temporary storage unit 106. The output control unit 109 also constitutes a mode switching control unit of one embodiment of the technology of the present disclosure, which switches between a first recording mode and a second recording mode.

[0032] The first recording mode is a mode in which only the moving image data out of the moving image data and the RAW pixel data is used as the recording data (data to be recorded). The second recording mode is a mode in which the moving image data and the RAW pixel data are used as the recording data. The recording data is data to be recorded on the recording medium 123 described below. For example, the output control unit 109 may switch between the first recording mode and the second recording mode based on an instruction from a user, or may automatically switch based on various data such as the free space of the recording medium 123 described below.

[0033] In the first recording mode, the output control unit 109 outputs the demosaic processed video data corrected by the development processing unit 107 from the external output I / F 110 to the external recording device 120. In addition, in the first recording mode, the output control unit 109 does not output the RAW pixel data stored in the temporary storage unit 106 to the external recording device 120.

[0034] In the second recording mode, the output control unit 109 outputs the demosaic processed video data corrected by the development processing unit 107 and the RAW pixel data before demosaic processing stored in the temporary storage unit 106 from the external output I / F 110 to the external recording device 120. In addition, when outputting the RAW pixel data from the external output I / F 110, the output control unit 109 adds the metadata stored in the temporary storage unit 106 to the RAW pixel data.

[0035] Furthermore, the output control unit 109 controls the imaging unit 119 to capture images with different exposure values ​​in the first recording mode and the second recording mode. Specifically, the output control unit 109 controls the imaging unit 119 via the imaging control unit 104 so that the exposure value of imaging in the second recording mode is lower than the exposure value of imaging in the first recording mode. The control of the exposure value can be performed by adjusting at least one of the aperture value (F value), the exposure time (shutter speed), and the ISO (International Organization for Standardization) sensitivity, for example.

[0036] As an example, the output control unit 109 controls the imaging unit 119 to capture an image with a first exposure value in the first recording mode. The first exposure value is an exposure value that provides video data with appropriate brightness when generating video data in BT.709 format, for example.

[0037] Furthermore, in the second recording mode, the output control unit 109 controls the imaging unit 119 to capture an image with a second exposure value that is lower (darker) than the first exposure value. The second exposure value is an exposure value that is lower than the first exposure value by, for example, an increase in brightness due to a gain correction, which will be described later, performed in the second recording mode.

[0038] In addition, the output control unit 109 controls the development processing unit 107 to perform different development processes in the first recording mode and the second recording mode. For example, the output control unit 109 controls the development processing unit 107 to perform different image processes related to brightness in the first recording mode and the second recording mode.

[0039] The image processing related to brightness performed by the development processing unit 107 is, for example, gamma correction, gain correction, lookup table (LUT) processing, or a combination of a plurality of these processes. The LUT processing is a process in which the three primary colors of RGB are used as input values ​​and each input value is converted into a predetermined output value. For example, there is an LUT with luminance Y as input / output, and a three-dimensional (3D) LUT with RGB as input / output. Furthermore, this image processing related to brightness may be a process performed before or after the demosaic processing in the development processing performed by the development processing unit 107.

[0040] The output control unit 109 controls the amount of increase in brightness of the moving image data due to the image processing related to brightness performed by the development processing unit 107 in the second recording mode to be greater than the amount of increase in brightness of the moving image data due to the image processing related to brightness performed by the development processing unit 107 in the first recording mode.

[0041] A large increase in brightness means, for example, a large increase in the output value for all input values. However, a large increase in brightness may also mean, for example, a large average increase in the output value for the input values, or a small output value for some input values.

[0042] For example, the output control unit 109 causes the development processing unit 107 to execute the same gamma correction using gamma characteristics corresponding to BT.709 in both the first recording mode and the second recording mode. Also, the output control unit 109 causes the development processing unit 107 to further execute gain correction for increasing brightness in the second recording mode, but does not cause the development processing unit 107 to execute this gain correction in the first recording mode. As a result, the increase in brightness of the moving image data is greater in the second recording mode than in the first recording mode by the amount of gain correction.

[0043] For example, this gain correction is a gain correction that enables BT.709 video data of appropriate brightness to be obtained in the second recording mode based on RAW pixel data obtained by capturing an image with a second exposure value that is lower than the first exposure value.

[0044] The external output I / F 110 is a communication interface for communicating with the external recording device 120. As an example, the external output I / F 110 communicates via HDMI. The external output I / F 110 outputs RAW pixel data and video data to the external recording device 120 under the control of the output control unit 109.

[0045] The imaging device 100 may include an internal memory that stores video data obtained by the development processing unit 107. The imaging device 100 may also include a user interface that receives various instructions from a user and outputs various data to the user.

[0046] The imaging device 100 may also include a microphone that converts ambient sound into an electrical signal. In this case, the electrical signal obtained by the microphone may be converted into digital acoustic data and output from the external output I / F 110 to the external recording device 120 together with the RAW pixel data and the video data, and may be recorded by the external recording device 120.

[0047] The external recording device 120 includes an external input I / F 121, a recording control unit 122, and a recording medium 123. The external input I / F 121 receives RAW pixel data and video data output from the external output I / F 110 of the imaging device 100, and outputs the received RAW pixel data and video data to the recording control unit 122.

[0048] The recording control unit 122 controls recording of the RAW pixel data and video data output from the external input I / F 121 onto the recording medium 123. The recording medium 123 is a large-capacity recording medium onto which data can be written at high speed so that large amounts of RAW pixel data and video data continuously output from the imaging device 100 can be recorded in real time. For example, the recording medium 123 can be realized by a memory card or an SSD (Solid State Drive).

[0049] In the first recording mode, moving image data is recorded sequentially, and RAW pixel data is not recorded, on the recording medium 123. In the second recording mode, moving image data and RAW pixel data are recorded in parallel on the recording medium 123. For example, in the second recording mode, moving image data and RAW pixel data may be recorded simultaneously on the recording medium 123, the moving image data and RAW pixel data may be recorded with a time difference, or the moving image data and RAW pixel data may be recorded alternately.

[0050] Although not shown, the external recording device 120 has an external output I / F that outputs the RAW pixel data and video data stored in the recording medium 123 to an external data processing device (e.g., a personal computer) different from the imaging device 100 and the external recording device 120. This allows the data processing device to manage the RAW pixel data and video data stored in the recording medium 123 and perform development processing based on the RAW pixel data.

[0051] Alternatively, the external recording device 120 may be a data processing device (such as a personal computer) having a processor and memory for performing development processing. In this case, the external recording device 120 can manage the RAW pixel data and video data stored in the recording medium 123 and perform development processing based on the RAW pixel data.

[0052] <Hardware configuration of the imaging device 100> The imaging control unit 104, the RAW correction unit 105, the development processing unit 107, and the output control unit 109 in the imaging device 100 are realized by a processor that operates in cooperation with the memory of the imaging device 100.

[0053] The processor is, for example, a processor such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an FPGA (Field Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit). The processor functions as the above-mentioned processing unit in the imaging device 100 by reading and executing a program stored in a memory. More specifically, the structure of the above-mentioned various processors is an electric circuit that combines circuit elements such as semiconductor elements. Note that the processor may be a combination of multiple processors of the same type or different types.

[0054] The memory is realized by RAM, ROM (Read Only Memory), flash memory, etc. The memory stores programs executed by the processor, data used by the processor, etc. Note that this memory may be a combination of multiple memories of the same or different types.

[0055] <Processing by Imaging Device 100 According to First Embodiment> Fig. 2 is a flowchart showing an example of processing by the imaging device 100 according to embodiment 1. For example, while capturing a moving image, the imaging device 100 executes the processing shown in Fig. 2 for each captured frame. The processing shown in Fig. 2 is executed by, for example, the output control unit 109.

[0056] First, the output control unit 109 judges whether the current mode is the first recording mode (step S21). If the current mode is the first recording mode (step S21: Yes), the output control unit 109 controls the imaging unit 119 via the imaging control unit 104 to capture one frame with the above-mentioned first exposure value corresponding to BT.709 (step S22).

[0057] Next, the output control unit 109 controls the development processing unit 107 to perform development processing including gamma correction using the gamma characteristics of BT.709 based on the RAW pixel data obtained in step S22, and generates BT.709 video data (step S23).

[0058] Next, the output control unit 109 records the video data generated in step S23 (step S24), and ends a series of processes for that frame. In step S24, the output control unit 109 outputs the video data from the external output I / F 110 to the external recording device 120, thereby recording the video data on the recording medium 123 of the external recording device 120.

[0059] In step S21, if the mode is not the first recording mode (step S21: No), i.e., if the mode is the second recording mode, the output control unit 109 controls the imaging unit 119 via the imaging control unit 104 to capture one frame at a second exposure value that is lower than the first exposure value (step S25).

[0060] Next, the output control unit 109 records the RAW pixel data obtained in step S25 (step S26). In step S26, the output control unit 109 outputs the RAW pixel data from the external output I / F 110 to the external recording device 120, thereby recording the RAW pixel data on the recording medium 123 of the external recording device 120.

[0061] Further, the output control unit 109 performs gain correction to increase the brightness on the RAW pixel data obtained in step S25 (step S27). Next, the output control unit 109 controls the development processing unit 107 to perform development processing including gamma correction using the gamma characteristics of BT.709 on the RAW pixel data that has undergone gain correction in step S27, and generates BT.709 video data (step S28). Next, the output control unit 109 proceeds to step S24, records the video data generated in step S28, and ends a series of processes related to that frame.

[0062] <Gamma characteristics of BT.709> FIG. 3 is a diagram showing an example of the gamma characteristics of BT.709. In FIG. 3, the horizontal axis represents the signal value of the RAW pixel data, and the vertical axis represents the signal value of the video data after gamma correction of BT.709. The gamma characteristic 31 is the gamma characteristic in the gamma correction of BT.709. In the development processing of steps S23 and S28 shown in FIG. 2, the imaging device 100 performs gamma correction using, for example, the gamma characteristic 31.

[0063] FIG. 4 is a diagram showing the relationship between the subject reflectance and the signal value after gamma correction in the gamma characteristic 31 shown in FIG. 3. In FIG. 4, the horizontal axis represents the subject reflectance [%] (i.e., the brightness of the subject being imaged), and the vertical axis represents the signal value of the video data after gamma correction of BT.709.

[0064] The signal value characteristic 41 is the characteristic of the signal value of the video data after gamma correction of BT.709 when imaging is performed at the above-mentioned first exposure value corresponding to BT.709. The signal value characteristic 41 has, for example, a signal value near 0.5 at a subject reflectance 43 of 18% gray, and has an appropriate brightness characteristic.

[0065] The signal value characteristic 42 is shown for reference as the characteristic of the signal value of the video data after gamma correction of BT.709 when imaging is performed at a second exposure value lower than the above-mentioned first exposure value. The signal value characteristic 42 has a darker characteristic compared to the signal value characteristic 41 as the exposure value of the imaging is lower.

[0066] In the first recording mode, imaging is performed with a first exposure value, so that moving image data with appropriate brightness such as the signal value characteristic 41 is obtained.

[0067] In the second recording mode, imaging is performed with a second exposure value that is lower than the first exposure value, but since gain correction for increasing brightness is performed in addition to gamma correction based on the gamma characteristic 31, moving image data with appropriate brightness is recorded as in the first recording mode. Furthermore, RAW pixel data with a wide dynamic range such as signal value characteristic 42 is recorded. This increases the degree of freedom in processing in the development process after storage in the recording medium 123.

[0068] In this way, according to the imaging device 100 of the first embodiment, the imaging unit 119 captures images using different exposure values ​​in a first recording mode in which only the video data of the video data and RAW pixel data is used as the recording data, and in a second recording mode in which the video data and RAW pixel data are used as the recording data.

[0069] As a result, imaging can be performed with an appropriate first exposure value in the first recording mode in which RAW pixel data is not stored, and imaging can be performed with a lower second exposure value that can obtain RAW pixel data with a high degree of freedom in processing in the development process in the second recording mode in which RAW pixel data is stored. Therefore, imaging can be performed with an appropriate exposure value depending on whether or not RAW pixel data is recorded.

[0070] By obtaining RAW pixel data with a wide dynamic range and high degree of freedom in processing, it is possible to generate video data that can express a wide range of brightness, such as HLG (Hybrid Log Gamma), through subsequent development processing.

[0071] Furthermore, the imaging device 100 may perform different development processes for generating demosaiced video data in the first and second recording modes. For example, the imaging device 100 performs different image processes related to brightness included in the development processes in the first and second recording modes.

[0072] This makes it possible to make the amount of increase in brightness of the video data caused by image processing related to brightness in the second recording mode greater than the amount of increase in brightness of the video data caused by image processing related to brightness in the first recording mode, thereby making it possible to suppress a decrease in brightness of the video data obtained in the second recording mode. In other words, in the second recording mode, it is possible to suppress a decrease in brightness of the video data while obtaining RAW pixel data that has a high degree of freedom in processing in the development process.

[0073] Although the above description has been given of a configuration in which gain correction in the second recording mode (e.g., step S27 in FIG. 2) is performed on RAW pixel data before demosaic processing, the imaging device 100 may also be configured to perform gain correction in the second recording mode on image data after demosaic processing.

[0074] (Embodiment 2) The second embodiment will be described with respect to the differences from the first embodiment.

[0075] <Processing by Imaging Device 100 According to Second Embodiment> Fig. 5 is a flowchart showing an example of processing by the imaging device 100 according to the second embodiment. The imaging device 100 according to the second embodiment executes the processing shown in Fig. 5 for each frame of imaging, for example, while capturing a moving image. The processing shown in Fig. 5 is executed by the output control unit 109, for example.

[0076] Steps S51 to S57 shown in FIG. 5 are the same as steps S21 to S26 and S28 shown in FIG. 2. However, in the development process of step S57, the output control unit 109 performs gamma correction using a gamma characteristic brighter than the gamma characteristic of BT.709 on the RAW pixel data obtained in step S55, and generates video data of BT.709 (step S57). The gamma characteristic brighter than the gamma characteristic of BT.709 is, for example, a gamma characteristic in which the average ratio of the output value to the input value is higher than the gamma characteristic of BT.709 (see FIG. 6 for example).

[0077] That is, in the process shown in FIG. 5, instead of performing the process of performing gain correction on the RAW pixel data (for example, step S27 in FIG. 2), the gamma characteristic used for the development process in the second recording mode is made brighter than the gamma characteristic used for the development process in the first recording mode.

[0078] <Gamma characteristic brighter than the gamma characteristic of BT.709> FIG. 6 is a diagram showing an example of a gamma characteristic brighter than the gamma characteristic of BT.709. In FIG. 6, the same parts as those shown in FIG. 3 are denoted by the same reference numerals and the description thereof is omitted. The gamma characteristic 61 is a gamma characteristic brighter than the gamma characteristic 31 of BT.709. Specifically, the gamma characteristic 61 is obtained by compressing the gamma characteristic 31 in the horizontal axis direction.

[0079] For example, the output control unit 109 uses the gamma characteristic 31 for the correction of the development process of step S53 shown in FIG. 5 (that is, the development process in the first recording mode), and uses the gamma characteristic 61 for the correction of the development process of step S57 shown in FIG. 5 (that is, the development process in the second recording mode).

[0080] FIG. 7 is a diagram showing the relationship between the subject reflectance and the signal value after gamma correction in the gamma characteristic 61 shown in FIG. 6. In FIG. 7, the same parts as those shown in FIG. 4 are denoted by the same reference numerals and the description thereof is omitted.

[0081] Signal value characteristic 71 is a characteristic of the signal value of moving image data after imaging with a second exposure value lower than the first exposure value and gamma correction using gamma characteristic 61. Signal value characteristic 71 is similar to signal value characteristic 41 when imaging is performed with the above-mentioned first exposure value corresponding to BT.709. Therefore, similar to the first embodiment, moving image data with appropriate brightness is recorded in the second recording mode as well.

[0082] Fig. 8 is a diagram showing another example of gamma characteristics brighter than the gamma characteristics of BT.709. In Fig. 8, parts similar to those shown in Fig. 3 are given the same reference numerals and description thereof will be omitted. Gamma characteristics 81 are brighter than gamma characteristics 31 of BT.709. Specifically, gamma characteristics 81 are gamma characteristics with a steeper rise in dark areas compared to gamma characteristics 31.

[0083] For example, the output control unit 109 uses gamma characteristic 31 to correct the development process of step S53 shown in FIG. 5 (i.e., the development process in the first recording mode), and uses gamma characteristic 81 to correct the development process of step S57 shown in FIG. 5 (i.e., the development process in the second recording mode).

[0084] Fig. 9 is a diagram showing the relationship between the subject reflectance and the signal value after gamma correction in the gamma characteristic 81 shown in Fig. 8. In Fig. 9, the same parts as those shown in Fig. 4 are given the same reference numerals and the description thereof will be omitted.

[0085] Signal value characteristic 91 is a characteristic of the signal value of the video data after imaging with a second exposure value lower than the above-mentioned first exposure value and gamma correction using gamma characteristic 81. Signal value characteristic 91 has a wide dynamic range while having brightness suitable for BT.709. Therefore, similar to the first embodiment, video data with appropriate brightness is recorded in the second recording mode as well. Furthermore, this video data has a wide dynamic range and a high degree of freedom in processing during development.

[0086] In this way, according to the imaging device 100 of embodiment 2, by performing different gamma corrections in the first recording mode and the second recording mode, in the second recording mode, it is possible to suppress a decrease in brightness of the video data while obtaining RAW pixel data that has a high degree of freedom in processing during development processing, just like the imaging device 100 of embodiment 1.

[0087] (Embodiment 3) The third embodiment will be described with respect to the differences from the first and second embodiments.

[0088] <Processing by Imaging Device 100 According to Third Embodiment> Fig. 10 is a flowchart showing an example of processing by the imaging device 100 according to embodiment 3. The imaging device 100 according to embodiment 3 executes the processing shown in Fig. 10 for each frame of imaging, for example, while capturing a moving image. The processing shown in Fig. 10 is executed by, for example, the output control unit 109.

[0089] Steps S101 to S108 shown in Fig. 10 are similar to steps S21 to S28 shown in Fig. 2. However, in the development processing of steps S103 and S108, the output control unit 109 performs each process of noise reduction and edge enhancement.

[0090] Furthermore, in the development process of step S108, the output control unit 109 performs noise reduction that is more effective at reducing noise than the noise reduction in the development process of step S103. This makes it possible to perform appropriate noise reduction on the noise amplified by the gain correction in step S107.

[0091] Furthermore, in the development process of step S108, the output control unit 109 performs noise reduction with a weaker effect of contour enhancement than the contour enhancement in the development process of step S103. This makes it possible to prevent the image quality of the image in which noise has been amplified by the gain correction of step S107 from being further deteriorated by the contour enhancement.

[0092] In this way, the imaging device 100 according to the third embodiment performs different noise processing in the first recording mode and the second recording mode, thereby making it possible to perform appropriate noise reduction for noise that is greater in the second recording mode than in the first recording mode.

[0093] Moreover, the imaging device 100 according to the third embodiment performs different contour processing in the first recording mode and the second recording mode, which makes it possible to suppress degradation of image quality in moving image data in the second recording mode, which has more noise than the first recording mode, due to contour emphasis.

[0094] (Embodiment 4) The fourth embodiment will be described with respect to the differences from the first to third embodiments.

[0095] <Imaging device 100 according to embodiment 4> Fig. 11 is a diagram showing an example of an imaging device 100 according to embodiment 4. In Fig. 11, the same parts as those shown in Fig. 1 are denoted by the same reference numerals and description thereof will be omitted.

[0096] The imaging device 100 according to the fourth embodiment is a moving image recording device including an internal output I / F 124, an internal input I / F 125, a recording control unit 122, and a recording medium 123, instead of the external output I / F 110 shown in FIG.

[0097] The internal output I / F 124 is an interface such as HDMI similar to the external output I / F 110 shown in Fig. 1, but differs from the external output I / F 110 in that it communicates with an internal input I / F 125 inside the imaging device 100. The internal input I / F 125 is an interface such as HDMI similar to the external input I / F 121 of the external recording device 120 shown in Fig. 1, but differs from the external input I / F 121 in that it is provided inside the imaging device 100.

[0098] The recording control unit 122 and recording medium 123 shown in Fig. 11 have the same configuration as the recording control unit 122 and recording medium 123 shown in Fig. 1, but are provided inside the imaging device 100. That is, the imaging device 100 shown in Fig. 11 has a high-speed, large-capacity recording medium 123 built in, and outputs RAW pixel data and video data to the recording medium 123 using an interface inside the imaging device 100.

[0099] 2, the output control unit 109 outputs the moving image data from the internal output I / F 124 to the internal input I / F 125, thereby recording the moving image data on the recording medium 123 of the imaging device 100. Also, in step S26 shown in FIG 2, the output control unit 109 outputs the RAW pixel data from the internal output I / F 124 to the internal input I / F 125, thereby recording the RAW pixel data on the recording medium 123 of the imaging device 100.

[0100] In this way, even in a configuration in which the imaging device 100 has a built-in recording medium 123 for recording RAW pixel data and moving image data, it is possible to obtain the same effects as those of the imaging device 100 according to the first embodiment.

[0101] (Combination of each embodiment) The above-mentioned respective embodiments can be realized in combination. For example, in the second embodiment, a gain correction like the first embodiment can be performed while performing different gamma corrections in the first recording mode and the second recording mode. In the second embodiment, noise processing and contour processing like the third embodiment can be performed. In the configuration of the imaging device 100 according to the fourth embodiment, it is also possible to realize the same processing as the second and third embodiments.

[0102] (Modification) Although BT.709 has been taken as an example of the video standard for moving image data, the video standard for moving image data is not limited to BT.709 and may be any of a variety of video standards.

[0103] As described above, this specification includes at least the following:

[0104] (1) a storage unit that temporarily stores pixel data output from the imaging unit; a development processing unit that outputs video data obtained by performing development processing including demosaic processing on the pixel data stored in the storage unit; a mode switching control unit that switches between a first recording mode in which only the moving image data is recorded as data out of the moving image data and the pixel data stored in the storage unit, and a second recording mode in which the moving image data and the pixel data stored in the storage unit are recorded as data, and controls the imaging unit to capture images with different exposure values ​​in the first recording mode and the second recording mode; A video control device comprising:

[0105] (2) The video control device according to (1), the mode switching control unit controls an exposure value of the imaging in the second recording mode to be lower than an exposure value of the imaging in the first recording mode; Video control device.

[0106] (3) A video control device according to (1) or (2), the mode switching control unit controls the developing process to be different between the first recording mode and the second recording mode; Video control device.

[0107] (4) The video control device according to (3), the mode switching control unit controls so that image processing related to brightness included in the development processing is different between the first recording mode and the second recording mode; Video control device.

[0108] (5) The video control device according to (4), the mode switching control unit controls so that an increase in brightness of the moving image data by the image processing related to the brightness in the second recording mode is greater than an increase in brightness of the moving image data by the image processing related to the brightness in the first recording mode; Video control device.

[0109] (6) A video control device according to (4) or (5), The brightness-related image processing includes gamma correction. Video control device.

[0110] (7) A video control device according to any one of (4) to (6), The brightness-related image processing includes gain correction. Video control device.

[0111] (8) A video control device according to any one of (4) to (7), The brightness-related image processing includes LUT processing. Video control device.

[0112] (9) A video control device according to any one of (1) to (8), the mode switching control unit controls the development processing unit to perform different noise processing on the pixel data in the first recording mode and the second recording mode; Video control device.

[0113] (10) A video control device according to any one of (1) to (9), the mode switching control unit controls the development processing unit to perform different contour processing on the pixel data in the first recording mode and the second recording mode; Video control device.

[0114] (11) A video control device according to any one of (1) to (10), An output section for outputting the recorded data A video control device comprising:

[0115] (12) A video control device according to any one of (1) to (10); A recording unit for recording the record data; A video recording device comprising:

[0116] (13) A video control method for a video control device having a storage unit that temporarily stores pixel data output from an imaging unit, comprising: outputting video data obtained by performing a development process including a demosaic process on the pixel data stored in the storage unit; switching between a first recording mode in which only the moving image data is recorded as data out of the moving image data and the pixel data stored in the storage unit, and a second recording mode in which the moving image data and the pixel data stored in the storage unit are recorded as data, and controlling the imaging unit to capture images with different exposure values ​​in the first recording mode and the second recording mode; Video control methods.

[0117] (14) (13) A video control method according to the present invention, controlling the exposure value of the imaging in the second recording mode to be lower than the exposure value of the imaging in the first recording mode; Video control methods.

[0118] (15) The video control method according to (13) or (14), controlling the developing process to be different between the first recording mode and the second recording mode; Video control methods.

[0119] (16) (15) A video control method according to the present invention, controlling the image processing related to brightness in the development processing to be different between the first recording mode and the second recording mode; Video control methods.

[0120] (17) (16) A video control method according to the present invention, control so that an increase in brightness of the moving image data by the image processing related to the brightness in the second recording mode is greater than an increase in brightness of the moving image data by the image processing related to the brightness in the first recording mode; Video control methods.

[0121] (18) The video control method according to (16) or (17), The brightness-related image processing includes gamma correction. Video control methods.

[0122] (19) A video control method according to any one of (16) to (18), The brightness-related image processing includes gain correction. Video control methods.

[0123] (20) A video control method according to any one of (16) to (19), The brightness-related image processing includes LUT processing. Video control methods.

[0124] (twenty one) A video control method according to any one of (13) to (20), a noise processing included in the development process is different between the first recording mode and the second recording mode; Video control methods.

[0125] (twenty two) A video control method according to any one of (13) to (21), a contour processing included in the development process differs between the first recording mode and the second recording mode; Video control methods.

[0126] (twenty three) A video control method according to any one of (13) to (22), an output unit for outputting the recorded data; A video control method comprising:

[0127] (twenty four) In the video control method according to any one of (13) to (22), Recording the record data; A video recording method comprising:

[0128] (twenty five) A video control program for a video control device having a storage unit that temporarily stores pixel data output from an imaging unit, The processor of the video control device includes: outputting video data obtained by performing a development process including a demosaic process on the pixel data stored in the storage unit; switching between a first recording mode in which only the moving image data is recorded as data out of the moving image data and the pixel data stored in the storage unit, and a second recording mode in which the moving image data and the pixel data stored in the storage unit are recorded as data, and controlling the imaging unit to capture images with different exposure values ​​in the first recording mode and the second recording mode; A video control program for executing the process. [Explanation of symbols]

[0129] 31,61,81 Gamma characteristics 41,42,71,91 Signal value characteristics 43 Subject reflectance 100 Imaging device 101 Imaging lens system 102 Image sensor 103 ADC 104 Imaging control unit 105 RAW correction section 106 Temporary storage 107 Development Processing Section 108 Monitor 109 Output control section 110 External output I / F 119 Imaging unit 120 External recording device 121 External input I / F 122 Recording control section 123 Recording media 124 Internal output I / F 125 Internal Input I / F

Claims

1. a storage unit that temporarily stores pixel data output from the imaging unit; a development processing unit that outputs video data obtained by performing development processing including demosaic processing on the pixel data stored in the storage unit; a mode switching control unit that switches between a first recording mode in which only the moving image data out of the moving image data and the pixel data stored in the storage unit is used as recording data, and a second recording mode in which the moving image data and the pixel data stored in the storage unit are used as recording data, and controls the imaging unit to capture images with different exposure values ​​in the first recording mode and the second recording mode; A video control device comprising: the mode switching control unit controls the image processing related to brightness depending on whether the recording mode is the first recording mode or the second recording mode so that an increase in brightness of the moving image data due to the image processing related to brightness included in the development processing in the second recording mode is greater than an increase in brightness of the moving image data due to the image processing related to brightness included in the development processing in the first recording mode. Video control device.

2. 2. The video control device according to claim 1, the mode switching control unit controls an exposure value of the imaging in the second recording mode to be lower than an exposure value of the imaging in the first recording mode. Video control device.

3. 3. The video control device according to claim 1, The brightness-related image processing includes gamma correction. Video control device.

4. 4. The video control device according to claim 1, The brightness-related image processing includes gain correction. Video control device.

5. 5. The video control device according to claim 1, The brightness-related image processing includes lookup table processing. Video control device.

6. The video control device according to any one of claims 1 to 5, the mode switching control unit controls the development processing unit to perform different noise processing on the pixel data in the first recording mode and the second recording mode. Video control device.

7. 7. The video control device according to claim 1, the mode switching control unit controls the development processing unit to perform different contour processing on the pixel data in the first recording mode and the second recording mode. Video control device.

8. The video control device according to any one of claims 1 to 7, An output unit that outputs the recording data A video control device comprising:

9. A video control device according to any one of claims 1 to 8, a recording unit for recording the record data; A video recording device comprising:

10. A video control method for a video control device having a storage unit that temporarily stores pixel data output from an imaging unit, comprising: outputting video data obtained by performing a development process including a demosaic process on the pixel data stored in the storage unit; switching between a first recording mode in which only the moving image data is recorded as data out of the moving image data and the pixel data stored in the storage unit, and a second recording mode in which the moving image data and the pixel data stored in the storage unit are recorded as data, and controlling the imaging unit to capture images with different exposure values ​​in the first recording mode and the second recording mode; A video control method, comprising: controlling the image processing related to brightness depending on whether the recording mode is the first recording mode or the second recording mode, so that an increase in brightness of the moving image data due to the image processing related to brightness included in the development processing in the second recording mode is greater than an increase in brightness of the moving image data due to the image processing related to brightness included in the development processing in the first recording mode. Video control methods.

11. The video control method according to claim 10, an exposure value of the imaging in the second recording mode is controlled to be lower than an exposure value of the imaging in the first recording mode; Video control methods.

12. 12. The video control method according to claim 10, further comprising: The brightness-related image processing includes gamma correction. Video control methods.

13. 13. A video control method according to any one of claims 10 to 12, comprising: The brightness-related image processing includes gain correction. Video control methods.

14. A video control method according to any one of claims 10 to 13, comprising: The brightness-related image processing includes lookup table processing. Video control methods.

15. 15. A video control method according to any one of claims 10 to 14, comprising: a noise processing included in the development process is different between the first recording mode and the second recording mode; Video control methods.

16. 16. A video control method according to any one of claims 10 to 15, comprising: a contour processing included in the development process is different between the first recording mode and the second recording mode; Video control methods.

17. 17. A video control method according to any one of claims 10 to 16, comprising: an output unit that outputs the recording data; A video control method comprising:

18. 18. The video control method according to claim 10, Recording the record data; A video recording method comprising:

19. A video control program for a video control device having a storage unit that temporarily stores pixel data output from an imaging unit, A processor of the video control device outputting video data obtained by performing a development process including a demosaic process on the pixel data stored in the storage unit; switching between a first recording mode in which only the moving image data is recorded as data among the moving image data and the pixel data stored in the storage unit, and a second recording mode in which the moving image data and the pixel data stored in the storage unit are recorded as data, and controlling the imaging unit to capture images with different exposure values ​​in the first recording mode and the second recording mode; controlling the image processing related to brightness depending on whether the recording mode is the first recording mode or the second recording mode, so that an increase in brightness of the moving image data due to the image processing related to brightness included in the development processing in the second recording mode is greater than an increase in brightness of the moving image data due to the image processing related to brightness included in the development processing in the first recording mode. A video control program for executing the process.

Images