Dynamic image control device, dynamic image recording device, dynamic image control method, dynamic image recording method, and recording medium

Abstract

The present application provides a kind of dynamic image control device, dynamic image recording device, dynamic image control method, dynamic image recording method and recording medium, which can be exposed according to the appropriate exposure value corresponding to the record of the pixel data before demosaicing processing.The temporary storage unit (106) temporarily stores the pixel data output from the camera unit (119).The developing processing unit (107) outputs the dynamic image data obtained by developing processing including demosaicing processing on the pixel data stored in the temporary storage unit (106).The output control unit (109) switches the first recording mode in which only dynamic image data in the dynamic image data and pixel data is recorded data, and the second recording mode in which dynamic image data and pixel data are recorded data.Furthermore, the output control unit (109) controls the camera unit (119) to shoot according to different exposure values in the first recording mode and the second recording mode.

Description

Technical Field

[0001] This invention relates to a dynamic image control device, a dynamic image recording device, a dynamic image control method, a dynamic image recording method, and a computer-readable recording medium storing a dynamic image control program. Background Technology

[0002] Patent document 1 describes a camera system having a recording mode that stores RAW data and YC data for thumbnail images converted from RAW data.

[0003] Patent document 2 describes a camera device in which, if a user presses the RAW video recording start button while recording Full HD (Full High Definition) video, recording Full HD video continues while recording RAW video begins simultaneously.

[0004] Patent document 3 describes a camera device that switches between a first mode that records dynamic image files based on digitally developed images along with RAW data of the images and a second mode that records digitally developed images.

[0005] Previous technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2007-166193

[0008] Patent Document 2: Japanese Patent Application Publication No. 2017-28466

[0009] Patent Document 3: Japanese Patent Application Publication No. 2010-21710 Summary of the Invention

[0010] One embodiment of the present invention provides a dynamic image control device, a dynamic image recording device, a dynamic image control method, a dynamic image recording method, and a computer-readable recording medium storing a dynamic image control program, which are capable of taking pictures based on an appropriate exposure value corresponding to the presence or absence of a recording of pixel data before de-mosaic processing.

[0011] means for solving technical problems

[0012] A dynamic image control device according to one embodiment of the present invention includes: a storage unit for temporarily storing pixel data output from a camera unit; a development processing unit for outputting dynamic image data obtained by performing a development process, including demosaic processing, on the pixel data stored in the storage unit; and a mode switching control unit for switching between a first recording mode that records only the dynamic image data and the dynamic image data stored in the storage unit and a second recording mode that records only the dynamic image data and the pixel data stored in the storage unit, and controlling the camera unit to take pictures according to different exposure values ​​in the first recording mode and the second recording mode.

[0013] One embodiment of the moving image recording apparatus according to the technology of the present invention includes: the moving image control device described above; and a recording unit for recording recording data output from the developing unit described above.

[0014] One embodiment of the dynamic image control method involved in the present invention is a dynamic image control method of a dynamic image control device having a storage unit for temporarily storing pixel data output from a camera unit. The method includes the following steps: outputting dynamic image data obtained by performing a development process including demosaic processing on the pixel data stored in the storage unit; and switching between a first recording mode that records only the dynamic image data and the dynamic image data stored in the storage unit and a second recording mode that records only the dynamic image data and the pixel data stored in the storage unit, and controlling the camera unit to take pictures according to different exposure values ​​in the first recording mode and the second recording mode.

[0015] The dynamic image recording method of one embodiment of the present invention records recording data in the above-described dynamic image control method.

[0016] The present invention relates to a motion picture control program stored in a computer-readable recording medium, which is a motion picture control program for a motion picture control device having a storage unit for temporarily storing pixel data output from a camera unit. The program is configured to cause the processor of the motion picture control device to perform the following processing: outputting motion picture data obtained by performing a developing process, including de-mosaic processing, on the pixel data stored in the storage unit; and switching between a first recording mode that records only the motion picture data and the motion picture data stored in the storage unit, and a second recording mode that records only the motion picture data and the pixel data stored in the storage unit, and controlling the camera unit to take pictures according to different exposure values ​​in the first and second recording modes.

[0017] Invention Effects

[0018] According to one embodiment of the present invention, a dynamic image control device, a dynamic image recording device, a dynamic image control method, a dynamic image recording method, and a recording medium readable by a computer that stores a dynamic image control program are provided, which can perform shooting based on an appropriate exposure value corresponding to the presence or absence of a recording of pixel data before de-mosaic processing. Attached Figure Description

[0019] Figure 1 This is a diagram illustrating an example of the camera device 100 according to the first embodiment.

[0020] Figure 2 This is a flowchart illustrating an example of the processing performed by the camera device 100 according to the first embodiment.

[0021] Figure 3 This is a diagram illustrating an example of the gamma characteristics of BT.709.

[0022] Figure 4 It means Figure 3 The graph shows the relationship between the subject reflectance and the gamma-corrected signal value in the gamma characteristic 31.

[0023] Figure 5 This is a flowchart illustrating an example of the processing performed by the camera device 100 according to the second embodiment.

[0024] Figure 6 This is a diagram illustrating an example of a gamma characteristic that is brighter than that of BT.709.

[0025] Figure 7 It means Figure 6 The graph shows the relationship between the subject reflectance and the gamma-corrected signal value in the gamma characteristic 61 shown.

[0026] Figure 8 This is another example of a gamma characteristic that is brighter than that of BT.709.

[0027] Figure 9 It means Figure 8 The graph shows the relationship between the subject reflectance and the gamma-corrected signal value in the gamma characteristic 81.

[0028] Figure 10 This is a flowchart illustrating an example of the processing performed by the camera device 100 according to the third embodiment.

[0029] Figure 11 This is a diagram illustrating an example of the camera device 100 according to the fourth embodiment. Detailed Implementation

[0030] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0031] (First Embodiment)

[0032] <The camera device 100 according to the first embodiment>

[0033] Figure 1 This diagram illustrates an example of the camera device 100 according to the first embodiment. The camera device 100 is a motion picture control device capable of generating motion pictures by performing continuous shooting. However, in addition to the function of generating motion pictures, the camera device 100 may also have the function of generating still images.

[0034] An external recording device 120 is disposed outside the camera device 100 and records the RAW pixel data and motion image data output from the camera device 100 (described later). The data output terminal of the camera device 100 is connected to the input terminal of the external recording device 120, for example, via a communication interface such as HDMI (High-Definition Multimedia Interface). HDMI is a registered trademark. Data output from the camera device 100 is not limited to HDMI; it can also be output wirelessly (e.g., UWB (Ultra Wide Band) or Wireless HDMI-SDI (High-Definition Multimedia Interface-Serial Digital Interface)).

[0035] The camera device 100 includes a camera unit 119, a camera control unit 104, a RAW correction unit 105, a temporary storage unit 106, a developing unit 107, a monitor 108, an output control unit 109, and an external output I / F 110.

[0036] The camera unit 119 generates image data for moving images by continuously capturing multiple RAW pixel data in a temporally consecutive manner. Specifically, the camera unit 119 includes an imaging lens system 101, an imaging element 102, and an ADC (Analog / Digital Converter) 103. The ADC 103 may also be built into the imaging element 102.

[0037] The imaging lens system 101 includes a lens for transmitting light from the subject and imaging it onto the imaging element 102. Furthermore, the imaging lens system 101 may include an aperture, an ND (Neutral Density) filter, a focusing lens, a zoom lens, a shifting lens, etc. These movable parts of the imaging lens system 101 are controlled by the camera control unit 104.

[0038] Imaging element 102 converts the optical image formed by imaging lens system 101 into an analog image signal and outputs the converted analog image signal to ADC 103. Imaging element 102 is composed of imaging elements such as CMOS (Complementary Metal-Oxide-Semiconductor) type image sensor or CCD (Charge-Coupled Device) type image sensor.

[0039] Furthermore, the imaging element 102 is equipped with an electronic shutter that serves as a shutter for adjusting the exposure time. The shooting performed by the imaging element 102 is controlled by the camera control unit 104. For example, when shooting moving images, the imaging element 102 continuously shoots in time under the control of the camera control unit 104, and sequentially outputs the resulting analog image signals to the ADC 103.

[0040] The ADC103 converts the analog image signal from the imaging element 102 into digital RAW pixel data and outputs the converted RAW pixel data to the RAW correction unit 105. The RAW pixel data continuously output from the ADC103 is the pixel data before demosaicing.

[0041] The camera control unit 104 controls the shooting performed by the camera unit 119 by controlling the imaging lens system 101 and the imaging element 102. For example, the camera control unit 104 controls the focus or exposure of the shooting performed by the camera unit 119 according to instructions from the user. Furthermore, the camera control unit 104 can also automatically control the exposure of the shooting performed by the camera unit 119 based on the de-mosaic image obtained by the developing processing unit 107 (described later).

[0042] Furthermore, the camera control unit 104 generates metadata representing the shooting conditions performed by the imaging lens system 101 and the imaging element 102, and outputs the generated metadata to the temporary storage unit 106. The temporary storage unit 106 is essentially composed of memory (of any kind). The metadata is used during the development process, in which a de-mosaic image is generated based on the RAW pixel data. For example, the metadata includes black offset level, coefficients for converting RAW values ​​to color scales, white balance parameters, lens correction parameters, color conversion parameters, gamma correction parameters, noise correction parameters, timecode, shooting date and time, product name, etc.

[0043] The RAW correction unit 105 corrects the RAW pixel data output from the ADC 103. The correction performed by the RAW correction unit 105 is a correction of the RAW pixel data before demosaicing, such as pixel value correction, defect pixel correction, and shadow correction corresponding to the characteristics of the imaging element 102. The RAW correction unit 105 outputs the corrected RAW pixel data to the temporary storage unit 106.

[0044] Temporary storage unit 106 temporarily stores RAW pixel data output from RAW correction unit 105 and metadata output from camera control unit 104. For example, temporary storage unit 106 may be implemented using a memory such as RAM (Random Access Memory). Furthermore, temporary storage unit 106 may be implemented using multiple memories. For example, temporary storage unit 106 may be implemented using a memory that stores RAW pixel data and a memory that stores metadata.

[0045] The developing unit 107 generates a de-mosaic motion image by performing developing processing, including de-mosaic processing, based on the RAW pixel data and metadata stored in the temporary storage unit 106, and outputs the generated de-mosaic motion image to the monitor 108. In the RAW pixel data that has undergone defect pixel correction or shadow correction, each pixel is only associated with one of the colors R, G, and B. Therefore, the other two colors are supplemented from surrounding pixels, so that each pixel has data of all three colors. For example, if a pixel only has the R color, then the G and B data are not present, so the G color data of that pixel is supplemented from the data of surrounding G pixels or predicted based on the surrounding G pixels. Thus, all pixels can have the three colors R, G, and B respectively. This is called de-mosaic processing.

[0046] Furthermore, the developing unit 107 generates dynamic image data by performing various image processing operations on the image data generated through the demosaic processing, and outputs the generated dynamic image data to the monitor 108. The image processing performed by the developing unit 107 after the demosaic processing includes, for example, gain correction, gamma correction, peripheral light attenuation correction, color correction, contour enhancement, noise reduction, Bayer de-processing, compression, and other processing.

[0047] As an example, the developing unit 107 generates moving image data corresponding to BT.709 by performing various image processing operations on the image data. BT.709 is an image standard used in general televisions or monitors. In this case, as one of the image processing operations, the developing unit 107 performs gamma correction based on the gamma characteristics corresponding to BT.709.

[0048] The monitor 108 displays a moving image to the user based on the moving image data output from the developing unit 107. Thus, the user can view the moving image being captured in real-time while it is being captured.

[0049] The output control unit 109 controls the output of the motion image data after de-mosaicing processing by the developing processing unit 107 and the RAW pixel data before de-mosaicing processing stored in the temporary storage unit 106. Furthermore, the output control unit 109 constitutes a mode switching control unit in one embodiment of the present invention for switching between the first recording mode and the second recording mode.

[0050] The first recording mode is a mode that records only the moving image data and the moving image data from the RAW pixel data. The second recording mode is a mode that records both the moving image data and the RAW pixel data. The recorded data refers to the data to be recorded onto the recording medium 123, which will be described later. For example, the output control unit 109 can switch between the first and second recording modes based on instructions from the user, or it can automatically switch between the first and second recording modes based on various data such as the available capacity of the recording medium 123, which will be described later.

[0051] In the first recording mode, the output control unit 109 outputs the de-mosaiced motion image data corrected by the developing processing unit 107 to the external recording device 120 from the external output I / F 110. Furthermore, in the first recording mode, the output control unit 109 does not output the RAW pixel data stored in the temporary storage unit 106 to the external recording device 120.

[0052] In the second recording mode, the output control unit 109 outputs, from the external output I / F 110, the de-mosaiced motion image data corrected by the developing processing unit 107 and the un-de-mosaiced RAW pixel data stored in the temporary storage unit 106 to the external recording device 120. Furthermore, when outputting RAW pixel data from the external output I / F 110, the output control unit 109 appends metadata stored in the temporary storage unit 106 to the RAW pixel data.

[0053] Furthermore, the output control unit 109 controls the imaging unit 119 to take pictures according to different exposure values ​​in the first recording mode and the second recording mode. Specifically, the output control unit 109 controls the imaging unit 119 via the imaging control unit 104 to make the exposure value of the picture taken in the second recording mode lower than the exposure value of the picture taken in the first recording mode. The exposure value can be controlled, for example, by adjusting at least one of the aperture value (F-number), exposure time (shutter speed), and ISO (International Organization for Standardization) sensitivity.

[0054] As an example, the output control unit 109 controls the camera unit 119 to take pictures according to a first exposure value in the first recording mode. The first exposure value is, for example, the exposure value that can obtain the appropriate brightness of the moving image data when generating the moving image data of BT.709.

[0055] Furthermore, the output control unit 109 controls the camera unit 119 to capture images in the second recording mode based on a second exposure value that is lower (darker) than the first exposure value. The second exposure value is, for example, an exposure value that is lower than the first exposure value by the amount of brightness increase produced by the gain correction performed later in the second recording mode.

[0056] Furthermore, the output control unit 109 controls the developing process performed by the developing processing unit 107 in different ways in the first recording mode and the second recording mode. For example, the output control unit 109 controls the brightness-related image processing performed by the developing processing unit 107 in different ways in the first recording mode and the second recording mode.

[0057] The brightness-related image processing performed by the developing unit 107 includes, for example, gamma correction, gain correction, lookup table (LUT) processing, or a combination of several of these processes. LUT processing is the process of converting RGB primary colors as input values ​​into preset output values. For example, there are LUTs with luminance Y as input and output, and three-dimensional (3D) LUTs with RGB as input and output. Furthermore, this brightness-related image processing can be performed during the developing process by the developing unit 107 before or after demosaicing.

[0058] The output control unit 109 controls the output to ensure that the increase in brightness of the motion image data generated by the brightness-related image processing performed by the developing unit 107 in the second recording mode is greater than the increase in brightness of the motion image data generated by the brightness-related image processing performed by the developing unit 107 in the first recording mode.

[0059] A large increase in brightness can refer to a large increase in the output value relative to all input values. However, a large increase in brightness can also mean a large average increase in the output value relative to the input values, or a small increase in the output value relative to a subset of input values.

[0060] For example, in both the first and second recording modes, the output control unit 109 causes the developing unit 107 to perform the same gamma correction using the gamma characteristics corresponding to BT.709. Furthermore, in the second recording mode, the output control unit 109 causes the developing unit 107 to further perform gain correction to increase brightness, while in the first recording mode, the developing unit 107 does not perform this gain correction. Therefore, the increase in brightness of the moving image data in the second recording mode is equivalent to the amount of gain correction, which is greater than the increase in brightness of the moving image data in the first recording mode.

[0061] For example, this gain correction is for gain correction of BT.709 moving image data to obtain appropriate brightness based on RAW pixel data in a second recording mode, which is obtained by shooting with a second exposure value lower than the first exposure value.

[0062] External output I / F 110 is a communication interface for communicating with external recording device 120. As an example, external output I / F 110 communicates via HDMI. Under the control of output control unit 109, external output I / F 110 outputs RAW pixel data and motion picture data to external recording device 120.

[0063] Furthermore, the camera device 100 may have an internal memory for storing moving image data obtained through the developing unit 107. Also, the camera device 100 may have a user interface for receiving various instructions from the user or outputting various data to the user.

[0064] Furthermore, the camera device 100 may be equipped with a microphone that converts ambient sound into electrical signals. In this case, the electrical signals obtained through the microphone can be converted into digital audio data, which, along with RAW pixel data and moving image data, is output from the external output I / F 110 to the external recording device 120 and recorded by the external recording device 120.

[0065] The external recording device 120 includes an external input I / F 121, a recording control unit 122, and a recording medium 123. The external input I / F 121 reads RAW pixel data and motion image data output from the external output I / F 110 of the camera device 100, and outputs the read RAW pixel data and motion image data to the recording control unit 122.

[0066] The recording control unit 122 controls the recording of RAW pixel data and video data output from the external input I / F 121 into the recording medium 123. The recording medium 123 is a high-capacity recording medium capable of high-speed writing to record large amounts of RAW pixel data and video data continuously output from the camera device 100 in real time. For example, the recording medium 123 can be implemented as a memory card or an SSD (Solid State Drive).

[0067] In the first recording mode, motion image data is recorded sequentially on the recording medium 123, without recording RAW pixel data. In the second recording mode, motion image data and RAW pixel data are recorded simultaneously on the recording medium 123. For example, in the second recording mode, motion image data and RAW pixel data can be recorded simultaneously on the recording medium 123, or they can be recorded with a time difference, or they can be recorded alternately.

[0068] Furthermore, although not illustrated, the external recording device 120 has an external output I / F that outputs RAW pixel data and moving image data stored in the recording medium 123 to an external data processing device (e.g., a personal computer) that is different from the camera device 100 and the external recording device 120. Therefore, management of the RAW pixel data and moving image data stored in the recording medium 123 or development processing based on the RAW pixel data can be performed in the data processing device.

[0069] Alternatively, the external recording device 120 may be a data processing device (e.g., a personal computer) having a processor and memory for performing development processing. In this case, the external recording device 120 is capable of managing RAW pixel data and moving image data stored in the recording medium 123 or performing development processing based on RAW pixel data.

[0070] <Hardware Structure of Camera Device 100>

[0071] The camera control unit 104, RAW correction unit 105, developing unit 107, and output control unit 109 in the camera device 100 are implemented by a processor that operates in cooperation with the memory of the camera device 100.

[0072] The processor can be, for example, a CPU (Central Processing Unit), an MPU (Microprocessor Unit), a FPGA (Field Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit). This processor executes a program stored in memory to perform the functions of the processing unit in the camera device 100. More specifically, the structure of these various processors is a circuit composed of combined semiconductor elements and other circuit components. Furthermore, the processor can be a combination of multiple processors of the same or different types.

[0073] Memory is implemented using RAM, ROM (Read Only Memory), flash memory, etc. Memory stores programs executed by the processor or data used by the processor. Furthermore, the memory can be a combination of multiple memory types, either the same or different.

[0074] <Processing performed by the camera device 100 according to the first embodiment>

[0075] Figure 2 This is a flowchart illustrating an example of the processing performed by the camera device 100 according to the first embodiment. For example, when capturing moving images, the camera device 100 performs processing for each frame captured. Figure 2 The processing shown. Figure 2 The processing shown is performed, for example, by the output control unit 109.

[0076] First, the output control unit 109 determines whether the current mode is the first recording mode (step S21). In the case of the first recording mode (step S21: yes), the output control unit 109 controls the camera unit 119 via the camera control unit 104, thereby capturing one frame at the first exposure value corresponding to BT.709 (step S22).

[0077] Next, the output control unit 109 controls the developing processing unit 107 to perform developing processing, including gamma correction using the gamma characteristics of BT.709, based on the RAW pixel data obtained in step S22, and generates dynamic image data of BT.709 (step S23).

[0078] Next, the output control unit 109 records the motion image data generated in step S23 (step S24), ending a series of processes related to that frame. In step S24, the output control unit 109 outputs the motion image data from the external output I / F 110 to the external recording device 120, recording the motion image data into the recording medium 123 of the external recording device 120.

[0079] In step S21, if it is not the first recording mode (step S21: No), that is, in the second recording mode, the output control unit 109 controls the camera unit 119 via the camera control unit 104, thereby capturing one frame with a second exposure value lower than the first exposure value (step S25).

[0080] Next, the output control unit 109 records the RAW pixel data obtained in step S25 (step S26). In step S26, the output control unit 109 outputs the RAW pixel data to the external recording device 120 from the external output I / F 110, and records the RAW pixel data into the recording medium 123 of the external recording device 120.

[0081] Furthermore, the output control unit 109 performs gain correction to increase brightness on the RAW pixel data obtained in step S25 (step S27). Next, the output control unit 109 controls the development processing unit 107 to perform development processing, including gamma correction using the gamma characteristics of BT.709, based on the RAW pixel data that has undergone gain correction in step S27, generating BT.709 motion image data (step S28). Then, the output control unit 109 proceeds to step S24, records the motion image data generated in step S28, and ends the series of processes related to that frame.

[0082] <Gamma characteristics of BT.709>

[0083] Figure 3 This is a diagram illustrating an example of the gamma characteristics of BT.709. In Figure 3 In the diagram, the horizontal axis represents the signal values ​​of the RAW pixel data, and the vertical axis represents the signal values ​​of the dynamic image data after gamma correction using BT.709. Gamma characteristic 31 represents the gamma characteristic in BT.709 gamma correction. Figure 2 In the development process shown in steps S23 and S28, the imaging device 100 performs gamma correction using gamma characteristic 31, for example.

[0084] Figure 4 It means Figure 3 The graph shows the relationship between the subject reflectance and the gamma-corrected signal value in gamma characteristic 31. Figure 4 In the figure, the horizontal axis represents the reflectance of the subject [%) (i.e., the brightness of the subject being photographed), and the vertical axis represents the signal value of the dynamic image data after gamma correction of BT.709.

[0085] Signal value characteristic 41 refers to the signal value characteristic of the gamma-corrected moving image data of BT.709 when shooting at the first exposure value corresponding to BT.709. Signal value characteristic 41 has a signal value around 0.5 at, for example, a subject reflectance 43 of 18% gray, which is a characteristic suitable for brightness.

[0086] For reference, signal value characteristic 42 represents the signal value characteristic of the BT.709 gamma-corrected moving image data when shooting at a second exposure value lower than the first exposure value described above. The exposure value of the shot with signal value characteristic 42 is low, and correspondingly has a darker characteristic than signal value characteristic 41.

[0087] In the first recording mode, the image is taken at the first exposure value, thus obtaining dynamic image data with appropriate brightness, such as signal value characteristic 41.

[0088] In the second recording mode, an image is taken at a lower exposure value than the first exposure value, but gain correction to increase brightness is performed separately from gamma correction based on gamma characteristic 31. Therefore, similar to the first recording mode, dynamic image data with appropriate brightness can be recorded. Furthermore, regarding RAW pixel data, RAW pixel data with a wide dynamic range, such as signal value characteristic 42, can be recorded. As a result, the processing freedom in the development process after the data is stored in the recording medium 123 is increased.

[0089] Thus, according to the camera device 100 of the first embodiment, the camera unit 119 takes pictures according to different exposure values ​​in a first recording mode that records only the moving image data and the RAW pixel data.

[0090] Therefore, it is possible to take a picture at an appropriate first exposure value in a first recording mode that does not store RAW pixel data, and to take a picture at a lower second exposure value in a second recording mode that stores RAW pixel data, allowing for greater processing freedom in the development process. Thus, it is possible to take a picture at an appropriate exposure value corresponding to the presence or absence of RAW pixel data recording.

[0091] By obtaining RAW pixel data with a wide dynamic range and high degree of processing freedom, it is possible to generate dynamic image data such as HLG (Hybrid Log Gamma) that can exhibit a wide brightness range through subsequent development processing.

[0092] Furthermore, the imaging device 100 can use different development processes for generating de-mosaiced motion image data in the first recording mode and the second recording mode. For example, the imaging device 100 can make the brightness-related image processing included in the development process different in the first recording mode and the second recording mode.

[0093] Therefore, the increase in brightness of the motion image data generated by brightness-related image processing in the second recording mode is greater than the increase in brightness of the motion image data generated by brightness-related image processing in the first recording mode, thus suppressing the decrease in brightness of the motion image data obtained in the second recording mode. In other words, it is possible to obtain RAW pixel data with high processing freedom in the development process in the second recording mode while suppressing the decrease in brightness of the motion image data.

[0094] Additionally, it explains the gain correction performed on the RAW pixel data before demosaicing in a second recording mode (e.g., Figure 2 The structure of step S27) is similar, but the camera device 100 can also perform gain correction on the image data after mosaic processing in a second recording mode.

[0095] (Second Implementation)

[0096] Regarding the second embodiment, the parts that differ from the first embodiment will be described.

[0097] <Processing performed by the camera device 100 according to the second embodiment>

[0098] Figure 5 This is a flowchart illustrating an example of the processing performed by the camera device 100 according to the second embodiment. For example, the camera device 100 according to the second embodiment performs processing for each frame captured when capturing moving images. Figure 5 The processing shown. Figure 5 The processing shown is performed, for example, by the output control unit 109.

[0099] Figure 5 Steps S51 to S57 shown are Figure 2 Steps S21 to S26 and S28 are the same. However, in the development process of step S57, the output control unit 109 performs gamma correction using a gamma characteristic brighter than that of BT.709 based on the RAW pixel data obtained in step S55, and generates dynamic image data of BT.709 (step S57). A gamma characteristic brighter than that of BT.709 refers, for example, a gamma characteristic whose average ratio of output value to input value is higher than that of BT.709 (for example, refer to...). Figure 6 ).

[0100] That is, in Figure 5 In the processing shown, instead of performing gain correction on the RAW pixel data (e.g., Figure 2 In step S27), the gamma characteristics of the development process used for the second recording mode are brighter than the gamma characteristics of the development process used for the first recording mode.

[0101] <Gamma characteristics brighter than BT.709>

[0102] Figure 6 This is a graph showing an example of a gamma characteristic that is brighter than that of BT.709. Figure 6 In the middle, to and Figure 3 The same parts shown are marked with the same symbols, and the descriptions are omitted. Gamma characteristic 61 is a gamma characteristic that is brighter than gamma characteristic 31 of BT.709. Specifically, gamma characteristic 61 is formed by compressing gamma characteristic 31 in the horizontal direction.

[0103] For example, the output control unit 109 in Figure 5 The correction of the development process (i.e., the development process of the first recording mode) in step S53 shown uses gamma characteristic 31. Figure 5 The gamma characteristic 61 is used in the correction of the development process (i.e., the development process of the second recording mode) in step S57 shown.

[0104] Figure 7 It means Figure 6 The graph shows the relationship between the subject reflectance and the gamma-corrected signal value in gamma characteristic 61. Figure 7 In the middle, to and Figure 4 The same parts are marked with the same symbol and the explanation is omitted.

[0105] Signal value characteristic 71 is the characteristic of the signal value of the moving image data after shooting at a second exposure value lower than the first exposure value and using gamma correction with gamma characteristic 61. Signal value characteristic 71 becomes the same as the signal value characteristic 41 when shooting at the first exposure value corresponding to BT.709. Therefore, similar to the first embodiment, moving image data with appropriate brightness can also be recorded in the second recording mode.

[0106] Figure 8 This is another example of a gamma characteristic that is brighter than that of BT.709. In Figure 8 In the middle, to and Figure 3 The same symbols are used for the same parts shown, and the descriptions are omitted. Gamma characteristic 81 is a gamma characteristic that is brighter than gamma characteristic 31 of BT.709. Specifically, gamma characteristic 81 is a gamma characteristic where the rise in dark areas is steeper than that of gamma characteristic 31.

[0107] For example, the output control unit 109 in Figure 5 The correction of the development process (i.e., the development process of the first recording mode) in step S53 shown uses gamma characteristic 31. Figure 5 The gamma characteristic 81 is used in the correction of the development process (i.e., the development process of the second recording mode) in step S57 shown.

[0108] Figure 9 It means Figure 8 The graph shows the relationship between the subject reflectance and the gamma-corrected signal value in gamma characteristic 81. Figure 9 In the middle, to and Figure 4 The same parts are marked with the same symbol and the explanation is omitted.

[0109] Signal value characteristic 91 refers to the signal value characteristics of the moving image data after shooting at a second exposure value lower than the first exposure value and using gamma correction with gamma characteristic 81. Signal value characteristic 91 has a brightness suitable for BT.709 while also having a wide dynamic range. Therefore, similar to the first embodiment, moving image data with appropriate brightness can also be recorded in the second recording mode. Furthermore, this moving image data has a wide dynamic range and a high degree of processing freedom in the development process.

[0110] Thus, according to the camera device 100 of the second embodiment, by performing different gamma corrections in the first recording mode and the second recording mode, similar to the camera device 100 of the first embodiment, it is possible to obtain RAW pixel data with high processing freedom in the development process in the second recording mode, while suppressing the brightness decrease of dynamic image data.

[0111] (Third Implementation)

[0112] Regarding the third embodiment, the parts that differ from the first and second embodiments will be described.

[0113] <Processing performed by the camera device 100 according to the third embodiment>

[0114] Figure 10 This is a flowchart illustrating an example of the processing performed by the camera device 100 according to the third embodiment. For example, the camera device 100 according to the third embodiment performs processing for each frame captured when capturing moving images. Figure 10 The processing shown. Figure 10 The processing shown is performed, for example, by the output control unit 109.

[0115] Figure 10 The steps S101 to S108 shown are... Figure 2 The steps S21 to S28 shown are the same. However, in the development process of steps S103 and S108, the output control unit 109 performs noise reduction and contour enhancement processes.

[0116] Furthermore, the noise reduction effect performed by the output control unit 109 in the development process of step S108 is stronger than that in the development process of step S103. As a result, the noise amplified by the gain correction in step S107 can be appropriately reduced.

[0117] Furthermore, the output control unit 109 performs contour emphasis in the development process of step S108, which has a weaker contour emphasis effect than the contour emphasis in the development process of step S103. As a result, it is possible to suppress the further degradation of image quality caused by the gain correction in step S107 due to the contour emphasis.

[0118] Thus, the camera device 100 according to the third embodiment performs different noise processing in the first recording mode and the second recording mode. As a result, it is possible to appropriately reduce noise in the second recording mode, which has more noise than the first recording mode.

[0119] Furthermore, the imaging device 100 according to the third embodiment performs different contour processing in the first recording mode and the second recording mode. As a result, the image quality of dynamic image data in the second recording mode, which has more noise than the first recording mode, can be suppressed due to contour emphasis.

[0120] (Fourth implementation)

[0121] Regarding the fourth embodiment, the parts that differ from the first to third embodiments will be described.

[0122] <The camera device 100 according to the fourth embodiment>

[0123] Figure 11 This diagram illustrates an example of the camera device 100 according to the fourth embodiment. Figure 11 In the middle, to and Figure 1 The same parts are marked with the same symbol and the explanation is omitted.

[0124] The camera device 100 in the fourth embodiment is a replacement Figure 1 The shown is a moving image recording device that has an external output I / F 110 and includes an internal output I / F 124, an internal input I / F 125, a recording control unit 122, and a recording medium 123.

[0125] Internal output I / F124 is AND Figure 1 The external output I / F110 shown has the same HDMI and other interfaces, but it differs from the external output I / F110 in that it communicates with the internal input I / F125 inside the camera device 100. The internal input I / F125 is for communication with... Figure 1 The external recording device 120 shown has the same HDMI or other interface as the external input I / F121, but it differs from the external input I / F121 in that it is located inside the camera device 100.

[0126] Figure 11 The recording control unit 122 and recording medium 123 shown are for use with Figure 1 The recording control unit 122 and recording medium 123 shown have the same structure, but are located inside the camera device 100. That is, Figure 11 The camera device 100 shown has a built-in high-speed and high-capacity recording medium 123, and uses the internal interface of the camera device 100 to output RAW pixel data and dynamic image data to the recording medium 123.

[0127] In this case, for example, in Figure 2 In step S24, the output control unit 109 outputs dynamic image data from the internal output I / F 124 to the internal input I / F 125, and records the dynamic image data onto the recording medium 123 of the camera device 100. Furthermore, in Figure 2 In step S26 shown, the output control unit 109 outputs RAW pixel data from the internal output I / F 124 to the internal input I / F 125 and records the RAW pixel data into the recording medium 123 of the camera device 100.

[0128] Thus, in the structure of the camera device 100 with a built-in recording medium 123 for recording RAW pixel data and moving image data, the same effect as that of the camera device 100 according to the first embodiment can be obtained.

[0129] (Combinations of various implementation methods)

[0130] The above-described embodiments can also be implemented by combining different methods. For example, in the second embodiment, different gamma corrections can be performed in the first recording mode and the second recording mode, while gain correction is performed as in the first embodiment. Furthermore, noise processing and contour processing as in the third embodiment can be performed in the second embodiment. Moreover, the same processing as in the second and third embodiments can be implemented in the structure of the imaging device 100 according to the fourth embodiment.

[0131] (Modified Example)

[0132] The BT.709 standard for moving image data has been used as an example, but the standard for moving image data is not limited to BT.709 and can be set to various other image standards.

[0133] As stated above, at least the following items are described in this specification.

[0134] (1) A dynamic image control device, comprising:

[0135] The storage unit temporarily stores pixel data output from the camera unit;

[0136] The developing unit outputs dynamic image data obtained by performing developing processing, including demosaic processing, on the pixel data stored in the aforementioned storage unit; and

[0137] The mode switching control unit switches between a first recording mode that records only the aforementioned dynamic image data and the aforementioned dynamic image data stored in the aforementioned storage unit, and a second recording mode that records only the aforementioned dynamic image data and the aforementioned pixel data stored in the aforementioned storage unit, and controls the camera unit to take pictures according to different exposure values ​​in the aforementioned first recording mode and the aforementioned second recording mode.

[0138] (2) The dynamic image control device according to (1), wherein,

[0139] The mode switching control unit controls the exposure value of the shooting in the second recording mode to be lower than the exposure value of the shooting in the first recording mode.

[0140] (3) The dynamic image control device according to (1) or (2), wherein,

[0141] The aforementioned mode switching control unit controls the development process in different ways in the first recording mode and the second recording mode.

[0142] (4) The dynamic image control device according to (3), wherein,

[0143] The aforementioned mode switching control unit controls the brightness-related image processing included in the aforementioned development process in different ways in the aforementioned first recording mode and the aforementioned second recording mode.

[0144] (5) The dynamic image control device according to (4), wherein,

[0145] The mode switching control unit controls the process such that the increase in brightness of the dynamic image data generated by the brightness-related image processing in the second recording mode is greater than the increase in brightness of the dynamic image data generated by the brightness-related image processing in the first recording mode.

[0146] (6) The dynamic image control device according to (4) or (5), wherein,

[0147] The aforementioned brightness-related image processing includes gamma correction.

[0148] (7) The dynamic image control device according to any one of (4) to (6), wherein,

[0149] The aforementioned brightness-related image processing includes gain correction.

[0150] (8) The dynamic image control device according to any one of (4) to (7), wherein,

[0151] The aforementioned brightness-related image processing includes LUT processing.

[0152] (9) The dynamic image control device according to any one of (1) to (8), wherein,

[0153] The mode switching control unit controls the development processing unit to perform different noise processing on the pixel data in the first recording mode and the second recording mode.

[0154] (10) The dynamic image control device according to any one of (1) to (9), wherein,

[0155] The aforementioned mode switching control unit controls the manner in which the aforementioned developing processing unit performs different contour processing on the aforementioned pixel data in the aforementioned first recording mode and the aforementioned second recording mode.

[0156] (11) The dynamic image control device according to any one of (1) to (10) includes an output control unit that outputs the recorded data.

[0157] (12) A dynamic image recording device comprising:

[0158] The dynamic image control device according to any one of (1) to (10); and

[0159] The Records Department records the aforementioned data.

[0160] (13) A dynamic image control method, which is a dynamic image control method of a dynamic image control device having a storage unit for temporarily storing pixel data output from a camera unit, comprising the following steps:

[0161] Outputs dynamic image data obtained by performing a development process, including demosaicing, on the pixel data stored in the aforementioned storage unit; and

[0162] The system switches between a first recording mode that records only the aforementioned dynamic image data and the aforementioned pixel data stored in the aforementioned storage unit, and a second recording mode that records only the aforementioned dynamic image data and the aforementioned pixel data stored in the aforementioned storage unit, and controls the camera unit to take pictures according to different exposure values ​​in the aforementioned first recording mode and the aforementioned second recording mode.

[0163] (14) According to the dynamic image control method described in (13), wherein,

[0164] The control is performed in such a way that the exposure value of the shooting in the second recording mode is lower than the exposure value of the shooting in the first recording mode.

[0165] (15) The dynamic image control method according to (13) or (14), wherein,

[0166] This allows the development process to be controlled in different ways under the first recording mode and the second recording mode.

[0167] (16) According to the dynamic image control method described in (15), wherein,

[0168] This allows the brightness-related image processing in the aforementioned development process to be controlled in different ways in the first recording mode and the second recording mode.

[0169] (17) According to the dynamic image control method described in (16), wherein,

[0170] The control is implemented such that the increase in brightness of the dynamic image data generated by the brightness-related image processing in the second recording mode is greater than the increase in brightness of the dynamic image data generated by the brightness-related image processing in the first recording mode.

[0171] (18) The dynamic image control method according to (16) or (17), wherein,

[0172] The aforementioned brightness-related image processing includes gamma correction.

[0173] (19) The dynamic image control method according to any one of (16) to (18), wherein,

[0174] The aforementioned brightness-related image processing includes gain correction.

[0175] (20) The dynamic image control method according to any one of (16) to (19), wherein,

[0176] The aforementioned brightness-related image processing includes LUT processing.

[0177] (21) The dynamic image control method according to any one of (13) to (20), wherein,

[0178] The noise processing included in the above-described development process differs between the first recording mode and the second recording mode.

[0179] (22) The dynamic image control method according to any one of (13) to (21), wherein,

[0180] The contour processing included in the above-described development process differs between the first recording mode and the second recording mode.

[0181] (23) The dynamic image control method according to any one of (13) to (22), wherein,

[0182] Output the above recorded data.

[0183] (24) A method for recording dynamic images, wherein,

[0184] The above-mentioned recording data is recorded in any one of the dynamic image control methods (13) to (22).

[0185] (25) A computer-readable recording medium storing a motion image control program, wherein the motion image control program is a motion image control device having a storage unit for temporarily storing pixel data output from a camera unit, and is used to cause the processor of the motion image control device to perform the following processing:

[0186] Outputs dynamic image data obtained by performing a development process, including demosaicing, on the pixel data stored in the aforementioned storage unit; and

[0187] The system switches between a first recording mode that records only the aforementioned dynamic image data and the aforementioned pixel data stored in the aforementioned storage unit, and a second recording mode that records only the aforementioned dynamic image data and the aforementioned pixel data stored in the aforementioned storage unit, and controls the camera unit to take pictures according to different exposure values ​​in the aforementioned first recording mode and the aforementioned second recording mode.

[0188] Symbol Explanation

[0189] 31, 61, 81 - Gamma characteristics; 41, 42, 71, 91 - Signal value characteristics; 43 - Subject reflectivity; 100 - Imaging device; 101 - Imaging lens system; 102 - Imaging element; 103 - ADC; 104 - Imaging control unit; 105 - RAW correction unit; 106 - Temporary storage unit; 107 - Development processing unit; 108 - Monitor; 109 - Output control unit; 110 - External output I / F; 119 - Imaging unit; 120 - External recording device; 121 - External input I / F; 122 - Recording control unit; 123 - Recording medium; 124 - Internal output I / F; 125 - Internal input I / F.

Claims

1. A dynamic image control device, comprising: The storage unit temporarily stores pixel data output from the camera unit; The developing unit outputs dynamic image data obtained by performing developing processing, including demosaic processing, on the pixel data stored in the storage unit; and The mode switching control unit switches between a first recording mode that records only the dynamic image data and the dynamic image data stored in the storage unit, and a second recording mode that records both the dynamic image data and the pixel data stored in the storage unit. It also controls the camera unit to capture images according to different exposure values ​​in both the first and second recording modes. The mode switching control unit controls the brightness-related image processing in a manner that makes the increase in brightness of the dynamic image data generated by the brightness-related image processing included in the development process in the second recording mode greater than the increase in brightness of the dynamic image data generated by the brightness-related image processing included in the development process in the first recording mode, depending on whether it is the first recording mode or the second recording mode.

2. The dynamic image control device according to claim 1, wherein, The mode switching control unit controls the shooting in a manner that the exposure value of the shot in the second recording mode is lower than the exposure value of the shot in the first recording mode.

3. The dynamic image control device according to claim 1 or 2, wherein, The brightness-related image processing includes gamma correction.

4. The dynamic image control device according to claim 1 or 2, wherein, The brightness-related image processing includes gain correction.

5. The dynamic image control device according to claim 1 or 2, wherein, The brightness-related image processing includes lookup table processing.

6. The dynamic image control device according to claim 1 or 2, wherein, The mode switching control unit controls the development processing unit to perform different noise processing on the pixel data in the first recording mode and the second recording mode.

7. The dynamic image control device according to claim 1 or 2, wherein, The mode switching control unit controls the development processing unit to perform different contour processing on the pixel data in the first recording mode and the second recording mode.

8. The dynamic image control device according to claim 1 or 2, further comprising an output control unit that outputs the recorded data.

9. A dynamic image recording device, comprising: The dynamic image control device according to claim 1 or 2; and The recording department records the recorded data.

10. A dynamic image control method, comprising a dynamic image control device having a storage unit for temporarily storing pixel data output from a camera unit, comprising the following steps: Output dynamic image data obtained by performing a development process, including demosaicing, on the pixel data stored in the storage unit; Switching between a first recording mode that records only the dynamic image data and the pixel data stored in the storage unit, and a second recording mode that records both the dynamic image data and the pixel data stored in the storage unit, and controlling the camera unit to take pictures according to different exposure values ​​in the first and second recording modes; and The brightness increase of the dynamic image data generated by the brightness-related image processing included in the development process under the second recording mode is greater than the brightness increase of the dynamic image data generated by the brightness-related image processing included in the development process under the first recording mode, wherein the brightness-related image processing is controlled according to whether it is the first recording mode or the second recording mode.

11. The dynamic image control method according to claim 10, wherein, The control is performed in such a way that the exposure value of the shot in the second recording mode is lower than the exposure value of the shot in the first recording mode.

12. The dynamic image control method according to claim 10 or 11, wherein, The brightness-related image processing includes gamma correction.

13. The dynamic image control method according to claim 10 or 11, wherein, The brightness-related image processing includes gain correction.

14. The dynamic image control method according to claim 10 or 11, wherein, The brightness-related image processing includes lookup table processing.

15. The dynamic image control method according to claim 10 or 11, wherein, The noise processing included in the development process differs between the first recording mode and the second recording mode.

16. The dynamic image control method according to claim 10 or 11, wherein, The contour processing included in the development process differs between the first recording mode and the second recording mode.

17. The dynamic image control method according to claim 10 or 11, wherein, Output the recorded data.

18. A method for recording dynamic images, wherein, The recorded data is recorded in the dynamic image control method according to claim 10 or 11.

19. A computer-readable recording medium storing a motion image control program, wherein the motion image control program is a motion image control device having a storage unit for temporarily storing pixel data output from a camera unit, and is used to cause the processor of the motion image control device to perform the following processing: Output dynamic image data obtained by performing a development process, including demosaicing, on the pixel data stored in the storage unit; Switching between a first recording mode that records only the dynamic image data and the pixel data stored in the storage unit, and a second recording mode that records both the dynamic image data and the pixel data stored in the storage unit, and controlling the camera unit to take pictures according to different exposure values ​​in the first and second recording modes; and The brightness increase of the dynamic image data generated by the brightness-related image processing included in the development process under the second recording mode is greater than the brightness increase of the dynamic image data generated by the brightness-related image processing included in the development process under the first recording mode, wherein the brightness-related image processing is controlled according to whether it is the first recording mode or the second recording mode.

Images