Imaging device, imaging method, and imaging program

Abstract

To provide an imaging apparatus, an imaging method, and an imaging program, capable of suppressing an influence of a delay in output of metadata on development processing.SOLUTION: An imaging unit 119 generates image data of a video as a plurality pieces of frame data that are continuous in a temporal order. An imaging control unit 104 generates metadata indicating an imaging condition of the imaging unit in association with the frame data in a case in which the imaging condition of the imaging unit 119 is changed. An output control unit 110 and an external output I / F 111 add the metadata to the frame data and output the frame data to which the metadata is added as video data before demosaicing. In addition, in a case in which a data amount of the metadata associated with the frame data exceeds an addable data amount, the output control unit 110 and the external output I / F111 add, to the frame data, metadata selected based on a priority from among the metadata associated with the frame data.SELECTED DRAWING: Figure 4

Description

Technical Field

[0005]

[0001] The present invention relates to an imaging device, an imaging method, and an imaging program.

Background Art

[0002] Patent Document 1 describes a configuration in which RAW video data and image correction data output from an imaging unit are input to a recording unit or an external recording device for recording. When the data size of the image correction data exceeds the data size that can be transmitted in one frame, the image correction data is divided into an arbitrary size and transmitted in multiple frames.

[0003] Patent Document 2 describes generating development parameters (development parameters for moving images) required when reproducing (developing) RAW moving image data based on the RAW video data on a frame-by-frame basis, and recording the RAW moving image data and the development parameters for the RAW image frames constituting the RAW moving image data on a recording medium.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

[0005] One embodiment according to the technology of the present disclosure provides an imaging device, an imaging method, and an imaging program capable of suppressing the influence on the development process due to the delay in the output of metadata such as development parameters.

Means for Solving the Problems

[0006] An imaging apparatus according to one embodiment of the technology of this disclosure comprises: an imaging unit that generates image data as a plurality of temporally continuous frame data; a metadata generation unit that generates metadata indicating the imaging conditions in association with the frame data; and an output unit that adds the metadata to the frame data and outputs it as image data, wherein the output unit adds metadata that is not added to the first frame data from the metadata associated with the first frame data generated by the imaging unit to the second frame data which is later than the first frame data.

[0007] An imaging method according to one embodiment of the technology of this disclosure is an imaging method using an imaging device equipped with an imaging unit that generates image data as a plurality of temporally continuous frame data, wherein metadata indicating the imaging conditions is generated in association with the frame data, the metadata is added to the frame data and output as image data, and metadata that is not added to the first frame data from the metadata associated with the first frame data generated by the imaging unit is added to the second frame data which is later than the first frame data.

[0008] An imaging program according to one embodiment of the technology of the present disclosure is an imaging program for an imaging device that includes an imaging unit that generates image data as a plurality of temporally consecutive frame data, and causes the processor of the imaging device to perform the following processes: generate metadata indicating the imaging conditions in association with the frame data; add the metadata to the frame data and output it as image data; and add the metadata that is not added to the first frame data from the metadata associated with the first frame data generated by the imaging unit to the second frame data which is later than the first frame data. [Effects of the Invention]

[0009] According to one embodiment of the technology of this disclosure, it is possible to provide an imaging device, an imaging method, and an imaging program that can suppress the impact of delays in metadata output on the development process. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example of an imaging device 100 according to Embodiment 1. [Figure 2] This figure shows specific examples of the types of metadata generated by the imaging control unit 104. [Figure 3] This figure shows an example of metadata generation by the imaging control unit 104 according to Embodiment 1. [Figure 4] This figure shows an example of output control by the output control unit 110 according to Embodiment 1. [Figure 5] This flowchart shows an example of processing performed by the imaging device 100 according to Embodiment 1. [Figure 6] This figure shows an example of an immediate output flag for metadata according to Embodiment 2. [Figure 7] This figure shows an example of metadata generation by the imaging control unit 104 according to Embodiment 2. [Figure 8] This figure shows an example of output control by the output control unit 110 according to Embodiment 2. [Figure 9] This flowchart shows an example of processing performed by the imaging device 100 according to Embodiment 2. [Figure 10] This figure shows an example of metadata generation by the imaging control unit 104 according to Embodiment 3. [Figure 11] This figure shows an example of output control by the output control unit 110 according to Embodiment 3. [Figure 12] This flowchart shows an example of processing performed by the imaging device 100 according to Embodiment 3. [Figure 13] This figure shows an example of output control by the output control unit 110 according to Embodiment 4. [Figure 14]It is a flowchart showing an example of the processing by the imaging device 100 according to Embodiment 4. [Figure 15] It is a diagram showing an example of the generation of metadata by the imaging control unit 104 according to Embodiment 5. [Figure 16] It is a diagram showing an example of the output control by the output control unit 110 according to Embodiment 5. [Figure 17] It is a diagram showing an example of the imaging device 100 according to Embodiment 6.

Modes for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0012] (Embodiment 1) <Imaging device 100 according to Embodiment 1> FIG. 1 is a diagram showing an example of the imaging device 100 according to Embodiment 1. The imaging device 100 is a video imaging device capable of generating a video by performing continuous imaging. However, the imaging device 100 may have a function of generating a still image in addition to the function of generating a video.

[0013] The external recording device 120 is provided outside the imaging device 100 and records RAW video data and metadata described later output from the imaging device 100. The data output terminal of the imaging device 100 is connected to the input terminal of the external recording device 120 via a communication interface such as HDMI (High-Definition Multimedia Interface). Note that HDMI is a registered trademark. The data output from the imaging device 100 is not limited to HDMI, and may be output using wireless communication (for example, UWB (Ultra Wide Band) or wireless HDMI-SDI (High-Definition Multimedia Interface-Serial Digital Interface)).

[0014] The imaging device 100 includes an imaging unit 119, an imaging control unit 104, a RAW correction unit 105, a temporary storage unit 106, a demosaicing processing unit 107, an image correction unit 108, a monitor 109, an output control unit 110, and an external output I / F 111.

[0015] The imaging unit 119 generates video image data as multiple time-sequential frame data by performing continuous imaging. Specifically, the imaging unit 119 includes an imaging lens system 101, an image sensor 102, and an ADC (Analog / Digital Converter) 103. The ADC 103 may be built into the image sensor 102.

[0016] The imaging lens system 101 includes lenses for transmitting light from the subject and forming an image on the image sensor 102. The imaging lens system 101 may also include an aperture, an ND (Neutral Density) filter, a focus lens, a zoom lens, a shift lens, etc. The movable parts of these imaging lens systems 101 are controlled by the imaging control unit 104.

[0017] The image sensor 102 converts the optical image from the imaging lens system 101 into an analog image signal and outputs the converted analog image signal to the ADC 103. The image sensor 102 is composed of an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) type image sensor or a CCD (Charge-Coupled Device) type image sensor.

[0018] Furthermore, the image sensor 102 is equipped with an electronic shutter, which is a shutter for adjusting the exposure time. Image capture by the image sensor 102 is controlled by the image control unit 104. For example, when capturing video, the image sensor 102 continuously captures images in time under the control of the image control unit 104, and sequentially outputs the resulting analog image signals to the ADC 103.

[0019] The ADC103 converts the analog image signal from the image sensor 102 into digital frame data and outputs the converted frame data to the RAW correction unit 105. The frame data output continuously from the ADC103 is the RAW video data before demosaicing.

[0020] The imaging control unit 104 controls imaging by the imaging lens system 101 and the image sensor 102. For example, the imaging control unit 104 controls the focus, exposure, etc. of imaging by the imaging unit 119 in response to instructions from the user. Alternatively, the imaging control unit 104 may automatically control the exposure, etc. of imaging by the imaging unit 119 based on the demosaicing image obtained by the demosaicing processing unit 107, which will be described later.

[0021] Furthermore, the imaging control unit 104 constitutes a metadata generation unit that generates metadata indicating the imaging conditions by the imaging lens system 101 and the image sensor 102. The metadata is used when performing development processing to generate a demosaicing image based on the RAW video data. Specific examples of metadata will be described later (see, for example, Figure 2).

[0022] For example, the imaging control unit 104 generates metadata indicating each imaging condition of the imaging unit 119 at the start of imaging by the imaging unit 119, in association with the first frame data after the start of imaging by the imaging unit 119.

[0023] Generating metadata in association with a frame data means, for example, enabling the output control unit 110 (described later) to identify that the metadata corresponds to that frame data. As an example, the imaging control unit 104 adds the frame number of the corresponding frame data to the metadata.

[0024] Furthermore, if the imaging conditions for imaging by the imaging unit 119 are changed, the imaging control unit 104 generates metadata indicating the changed imaging conditions, associated with the frame data immediately following the change. On the other hand, if the imaging conditions for imaging by the imaging unit 119 have not been changed, the imaging control unit 104 does not generate new metadata. The imaging control unit 104 performs this processing for each type of metadata.

[0025] Therefore, when imaging by the imaging unit 119 begins, metadata indicating the imaging conditions of the imaging unit 119 at that time is generated in association with the first frame data. Subsequently, only when the imaging conditions are changed, metadata indicating the changed imaging conditions is generated in association with the frame data at that time. The imaging control unit 104 outputs the generated metadata to the temporary storage unit 106.

[0026] The RAW correction unit 105 performs corrections on the frame data (RAW video data) output from the ADC 103. The corrections performed by the RAW correction unit 105 are applied to the RAW video data before demosaicing, and include, for example, pixel value correction, defective pixel correction, and shading correction according to the characteristics of the image sensor 102. The RAW correction unit 105 outputs the corrected frame data to the temporary storage unit 106.

[0027] The temporary storage unit 106 temporarily stores the frame data output from the RAW correction unit 105 and the metadata output from the imaging control unit 104. For example, the temporary storage unit 106 can be implemented using memory (any type) such as RAM (Random Access Memory). Alternatively, the temporary storage unit 106 may be implemented using multiple memories. For example, the temporary storage unit 106 may be implemented using a memory for storing frame data and a memory for storing metadata.

[0028] The demosaicing processing unit 107 generates a demosaiced video by performing demosaicing on the frame data stored by the temporary storage unit 106, and outputs the generated demosaiced video to the image correction unit 108. The frame data, after performing defective pixel correction and shading correction, is output from the RAW correction unit 105. At this point, each individual pixel has only one of the three colors: R, G, or B. Therefore, the other two colors are interpolated from surrounding pixels so that each pixel has data for all three colors. For example, if a pixel only has the R color, then there is no G and B data, so the G color data for that pixel is interpolated from the data of surrounding G pixels or predicted from surrounding G pixels. As a result, every pixel has all three colors: R, G, and B. This is called demosaicing.

[0029] The image correction unit 108 performs various image corrections on the demosaicing video output from the demosaicing processing unit 107 and outputs the corrected demosaicing video to the monitor 109. The image corrections performed by the image correction unit 108 are performed on the demosaicing video after demosaicing, and include processes such as peripheral light falloff correction, color correction, edge enhancement, noise reduction, gamma correction, debayering, and compression.

[0030] The monitor 109 displays the demosaiced video output from the image correction unit 108 to the user. This allows the user to view the video being captured as a live image while video is being captured.

[0031] The output control unit 110 and the external output I / F 111 constitute the output unit of one embodiment of the technology of this disclosure. The output control unit 110 reads the frame data and metadata stored in the temporary storage unit 106 and adds the metadata to the frame data. The output control unit 110 then outputs the frame data with the metadata added as RAW video data before demosaicing to the external recording device 120 via the external output I / F 111.

[0032] For example, the output control unit 110 continuously outputs frame data as a video signal from the external output I / F 111, while outputting metadata from the external output I / F 111 during the blanking period of the video signal. In other words, the output control unit 110 controls the external output I / F 111 to interrupt the output of frame data to output metadata. This makes it possible to output frame data with metadata added.

[0033] The external output interface 111 is a communication interface for communicating with the external recording device 120. For example, the external output interface 111 communicates via HDMI. The external output interface 111 outputs frame data and metadata to the external recording device 120 according to the control from the output control unit 110.

[0034] The imaging device 100 may also be equipped with internal memory for storing demosaicing motion images and the like. Furthermore, the imaging device 100 may be equipped with a user interface for receiving various operations from the user and outputting various data to the user.

[0035] The imaging device 100 may also be equipped with a microphone that converts ambient sound into electrical signals. In this case, the electrical signals obtained by the microphone may be converted into digital acoustic data and output together with frame data and metadata from the external output I / F 111 to the external recording device 120, where they may be recorded.

[0036] The external recording device 120 comprises an external input interface 121, a recording control unit 122, and a recording medium 123. The external input interface 121 receives frame data and metadata output from the external output interface 111 of the imaging device 100, and outputs the received frame data and metadata to the recording control unit 122.

[0037] The recording control unit 122 controls the recording of frame data and metadata output from the external input I / F 121 onto the recording medium 123. The recording medium 123 is a high-capacity recording medium capable of high-speed writing so that large amounts of frame data continuously output from the imaging device 100 can be recorded in real time. For example, the recording medium 123 can be realized by a memory card or an SSD (Solid State Drive).

[0038] Although not shown in the diagram, the external recording device 120 has an external output interface that outputs frame data and metadata stored in the recording medium 123 to an external data processing device (e.g., a personal computer) that is different from the imaging device 100 and the external recording device 120. This allows the data processing device to perform development processing based on the frame data and metadata stored in the recording medium 123.

[0039] Alternatively, the external recording device 120 may be a data processing device (e.g., a personal computer) having a processor and memory for developing the image. In this case, the external recording device 120 can perform developing the image based on the frame data and metadata stored in the recording medium 123.

[0040] <Hardware configuration of imaging device 100> The imaging control unit 104, RAW correction unit 105, demosaicing processing unit 107, image correction unit 108, and output control unit 110 in the imaging device 100 are implemented by a processor that operates in coordination with the memory of the imaging device 100.

[0041] This processor is, for example, a CPU (Central Processing Unit), MPU (Micro Processing Unit), FPGA (Field Programmable Gate Array), PLD (Programmable Logic Device), or ASIC (Application Specific Integrated Circuit). This processor functions as the processing unit in the imaging device 100 by reading and executing a program stored in memory. More specifically, the structure of the above-mentioned various processors is an electrical circuit combining circuit elements such as semiconductor elements. This processor may be a combination of multiple processors of the same or different types.

[0042] Memory is implemented using RAM, ROM (Read Only Memory), flash memory, etc. Memory stores programs executed by the processor, or data used by the processor. This memory may be a combination of multiple memory types, either identical or different.

[0043] <Specific examples of metadata types generated by the imaging control unit 104> Figure 2 shows a specific example of the types of metadata generated by the imaging control unit 104. The imaging control unit 104 generates metadata that indicates imaging conditions, such as black offset level, coefficient for converting RAW values ​​to a color system, white balance parameters, lens correction parameters, color conversion parameters, gamma correction parameters, noise correction parameters, time code, imaging date and time, and product name, as shown in Figure 2.

[0044] Furthermore, these metadata items are assigned a priority level for each type and stored in the memory of the imaging device 100. The priority level indicates the degree to which the data should be output to the external recording device 120 with priority.

[0045] For example, the priority of each metadata type is set so that metadata with a greater impact on the development process result has a higher priority; in other words, metadata with a greater impact on the development process result is output preferentially. Outputting metadata with higher priority means that for each metadata with the same conditions other than priority, the metadata with the higher priority is output first.

[0046] For example, metadata used in earlier stages of the development process has a greater impact on the image quality of the demosaiced video obtained through the development process, and therefore is assigned a higher priority. In the example shown in Figure 2, the black offset level used in the development process is assigned the highest priority. On the other hand, the product name, which is not directly used in the development process, is assigned the lowest priority.

[0047] The type of metadata generated by the imaging control unit 104 is not limited to the example shown in Figure 2 and can be set arbitrarily. Similarly, the priority can be set arbitrarily and is not limited to the example shown in Figure 2. For example, at least some of the corrections performed by the RAW correction unit 105, such as pixel value correction, defective pixel correction, and shading correction, may be performed in the development process instead of being performed by the RAW correction unit 105. In this case, the imaging control unit 104 generates metadata for performing corrections such as pixel value correction, defective pixel correction, and shading correction and outputs it to the temporary storage unit 106.

[0048] <Generation of metadata by the imaging control unit 104 according to Embodiment 1> Figure 3 shows an example of metadata generation by the imaging control unit 104 according to Embodiment 1. The horizontal axis in Figure 3 represents time. Frame data F1 to F5 are frame data obtained from five consecutive imagings by the imaging unit 119, and are RAW video data before demosaicing.

[0049] Metadata M1 to M13 are metadata generated by the imaging control unit 104. Metadata M1 to M7 are data indicating each imaging condition set at the start of imaging, and are generated in association with the first frame data F1 after the start of imaging.

[0050] Metadata M1 is metadata indicating imaging conditions related to the number of pixels (resolution) used in the image sensor 102. Metadata M2 is metadata indicating imaging conditions related to the black offset level (black level). Metadata M3 and M4 are metadata indicating imaging conditions related to shading. Metadata M5 and M6 are metadata indicating imaging conditions related to distortion. Metadata M7 is metadata indicating imaging conditions related to white balance (WB).

[0051] In the example shown in Figure 3, the imaging conditions were not changed between the imaging of frame data F1 and the imaging of frame data F2. In this case, the imaging control unit 104 does not generate metadata associated with frame data F2.

[0052] Furthermore, in the example shown in Figure 3, assume that the imaging conditions related to white balance were changed between the imaging of frame data F2 and the imaging of frame data F3. In this case, the imaging control unit 104 generates metadata M8(WB) indicating the changed white balance imaging conditions, associated with frame data F3.

[0053] Furthermore, in the example shown in Figure 3, assume that the imaging conditions related to shading, distortion, and white balance are changed between the imaging of frame data F3 and the imaging of frame data F4. In this case, the imaging control unit 104 generates metadata M9 and M10 indicating the changed imaging conditions related to shading, metadata M11 and M12 indicating the changed imaging conditions related to distortion, and metadata M13 indicating the changed imaging conditions related to white balance, in association with frame data F4.

[0054] Furthermore, in the example shown in Figure 3, the imaging conditions were not changed between the imaging of frame data F4 and the imaging of frame data F5. In this case, the imaging control unit 104 does not generate metadata associated with frame data F5.

[0055] <Output control by the output control unit 110 according to Embodiment 1> Figure 4 shows an example of output control by the output control unit 110 according to Embodiment 1. In the example of metadata generation shown in Figure 3, the output control unit 110 outputs frame data and metadata, for example, as shown in Figure 4.

[0056] In this example, the data size of one metadata item is assumed to be 28 bytes, and the capacity of metadata that can be attached to one frame data and output from the external output I / F111 is assumed to be 28 × 4 bytes. In other words, it is assumed that four metadata items can be attached to one frame data item and output. Note that, for example, in HDMI, the packet container that can store metadata is inserted by the hardware during the blanking period, so there is a capacity limit, and there is a limit of 28 × 4 bytes / frame, similar to this example.

[0057] The number of metadata elements associated with frame data F1 is seven (metadata elements M1-M7), which exceeds the number of metadata elements that can be attached to frame data F1 (four). In other words, the amount of data of the metadata elements associated with frame data F1 is 28 × 7 [bytes], which exceeds the amount of data that can be attached to frame data F1 (28 × 4 [bytes]).

[0058] In this case, the output control unit 110 adds metadata selected from the metadata M1 to M7 associated with the frame data F1 based on the priority of metadata M1 to M7, and outputs it to the external recording device 120 via the external output I / F 111. In this example, the priority is set in the order of resolution, black level, shading, distortion, and white balance. In this case, the output control unit 110 adds four metadata M1 to M4 selected from metadata M1 to M7 in order of priority to the frame data F1, and outputs it to the external recording device 120 via the external output I / F 111.

[0059] Furthermore, the output control unit 110 adds the metadata M5 to M7 that were not added to the frame data F1 to the next frame data F2, and outputs it to the external recording device 120 via the external output I / F 111.

[0060] Furthermore, the output control unit 110 adds metadata M8, which is associated with the frame data F3, to the frame data F3 and outputs it to the external recording device 120 via the external output I / F 111.

[0061] The number of metadata associated with frame data F4 is five, consisting of metadata M9 to M13, which exceeds the number of metadata that can be added to frame data F4 (four). In this case, the output control unit 110 adds four metadata M9 to M12, selected from metadata M9 to M13 in order of priority, to frame data F4 and outputs them to the external recording device 120 via the external output I / F 111.

[0062] Furthermore, the output control unit 110 adds the metadata M13 that was not added to the frame data F4 to the next frame data F5, and outputs it to the external recording device 120 via the external output I / F 111.

[0063] <Specific examples of development processes> The development process in the example shown in Figure 4 will now be explained. In the example shown in Figure 4, the development process for the resolution, black level, and shading of frame data F1 is performed using metadata M1 to M4 attached to frame data F1. Since there is no metadata for distortion and white balance that can be applied to frame data F1, the development process for frame data F1 is performed using predetermined metadata for distortion and white balance, for example.

[0064] The distortion and white balance processing of frame data F2 is performed using metadata M5-M7 attached to frame data F2. Furthermore, the resolution, black level, and shading processing of frame data F2 are performed using metadata M1-M4.

[0065] The white balance processing for frame data F3 is performed using metadata M8 attached to frame data F3. Furthermore, the resolution, black level, shading, and distortion processing for frame data F3 are performed using metadata M1-M6.

[0066] The shading and distortion processing of frame data F4 is performed using metadata M9-M12 attached to frame data F4. Furthermore, the resolution, black level, and white balance processing of frame data F4 are performed using metadata M1, M2, and M8.

[0067] The white balance processing for frame data F5 is performed using metadata M13 attached to frame data F5. Furthermore, the resolution, black level, shading, and distortion processing for frame data F5 are performed using metadata M1, M2, M9-M12.

[0068] <Processing by the imaging device 100 according to Embodiment 1> Figure 5 is a flowchart showing an example of processing performed by the imaging device 100 according to Embodiment 1. The imaging device 100 according to Embodiment 1 performs, for example, the processing shown in Figure 5. The processing shown in Figure 5 is performed, for example, by a processor that implements the imaging control unit 104 and the output control unit 110.

[0069] First, the imaging device 100 controls the imaging unit 119 to start capturing video (step S51). Next, the imaging device 100 sets n to 1 and waits until the nth image is captured by the imaging unit 119 (step S52). n indicates the number of the frame currently being processed.

[0070] Next, the imaging device 100 generates metadata indicating the changes in imaging conditions between the (n-1)th imaging and the (n)th imaging, in association with the (n)th frame data (step S53). However, if n=1 in step S53, the imaging device 100 generates metadata for all imaging conditions, in association with the (n)th frame data. Also, in step S53, if none of the imaging conditions have changed between the (n-1)th imaging and the (n)th imaging, the imaging device 100 does not generate metadata.

[0071] Next, the imaging device 100 selects metadata to be added to and output from the nth frame data, within the range that can be added to a single frame data (step S54). In step S54, the imaging device 100 selects metadata that has not been output and is associated with the (n-1)th or earlier frame data (unoutput metadata from the previous frame), and metadata with higher priority that is associated with the nth frame data.

[0072] In other words, the imaging device 100 selects any unoutput metadata associated with the (n-1)th or earlier frame data, and if there is still space available, it selects the metadata with the highest priority from among the metadata associated with the nth frame data. If the amount of unoutput metadata associated with the (n-1)th or earlier frame data exceeds the range that can be attached to a single frame data, the imaging device 100 selects the metadata with the highest priority from among that unoutput metadata. In cases where there are no metadata to select in step S54, such as when the imaging conditions remain unchanged, the imaging device 100 does not select any metadata.

[0073] Next, the imaging device 100 adds the metadata selected in step S54 to the nth frame data (step S55). If no metadata to be selected exists in step S54, the imaging device 100 does not add any metadata.

[0074] Next, the imaging device 100 outputs the nth frame data, to which metadata has been added in step S55, to the external recording device 120 via the external output I / F 111 (step S56). If metadata was not added in step S55, the imaging device 100 outputs the nth frame data without metadata to the external recording device 120 via the external output I / F 111.

[0075] Next, the imaging device 100 increments n (n=n+1) and waits until the nth image is captured by the imaging unit 119 (step S57), then returns to step S53.

[0076] As described above, the imaging device 100 of Embodiment 1 outputs frame data (RAW video data) before demosaicing to an external recording device 120. This makes it possible to record high-data-rate RAW video data even if the imaging device 100 does not have a high-speed, high-capacity storage medium. In addition, by outputting metadata along with the frame data to the external recording device 120, the imaging device 100 can easily perform development processing based on the frame data and metadata stored in the external recording device 120.

[0077] Furthermore, the imaging device 100 generates metadata indicating the imaging conditions when the imaging conditions of the imaging unit 119 are changed, associates it with the frame data, and adds the generated metadata to the frame data to output it as video data before demosaicing. This reduces the amount of metadata data to be output compared to a configuration that outputs all metadata for each frame data even if the imaging conditions are not changed.

[0078] Therefore, metadata can be output with limited communication capacity, such as during the blanking period for frame data. Furthermore, since it is not necessary to increase the communication capacity during the blanking period, problems such as changes in frame rate or inability to maintain connection compatibility specifications due to increased communication capacity during the blanking period can be suppressed.

[0079] Furthermore, if the amount of metadata associated with a frame data exceeds the amount of data that can be added to that frame data, the imaging device 100 adds selected metadata to the frame data based on the priority given to the metadata type. This allows metadata of the type used in the development process to be output preferentially. Therefore, the impact on the development process results can be suppressed without outputting all metadata for each frame data.

[0080] For example, metadata for each imaging condition is generated at the start of imaging. Also, for example, if the focus of a zoom lens is changed, a lot of new metadata for lens correction is generated. In these cases, it may not be possible to output all the generated metadata in a single frame data, but according to the imaging device 100, the impact on the result of the development process (for example, the image quality of demosaicing video) can be suppressed by prioritizing the output of metadata of the type used in the development process.

[0081] Furthermore, the imaging device 100 prioritizes adding metadata that is not attached to the first frame data, rather than metadata associated with the output second frame data and metadata that is not attached to the first frame data preceding the second frame data, to the second frame data. This allows for the priority output of metadata that is delayed, thereby suppressing the impact of delays in metadata output on the development process results.

[0082] (Embodiment 2) The differences between Embodiment 2 and Embodiment 1 will now be described.

[0083] <Immediate output flag for metadata related to Embodiment 2> Figure 6 shows an example of an immediate output flag for metadata according to Embodiment 2. As shown in Figure 6, an immediate output flag may be set for each type of metadata. The immediate output flag indicates that the metadata should be output (i.e., immediately output) by being added to the frame data associated with that metadata.

[0084] The number of metadata types for which the immediate output flag is set is determined, for example, so that all metadata with the immediate output flag set can be attached to a single frame data. This ensures that at least the metadata types with the immediate output flag set can be attached to the frame data associated with that metadata and output without delay.

[0085] In the example shown in Figure 6, the immediate output flag is set to only three types of metadata indicating imaging conditions related to the black offset level, the coefficient for converting RAW values ​​to a color system, and the white balance parameter.

[0086] <Generation of metadata by the imaging control unit 104 according to Embodiment 2> Figure 7 shows an example of metadata generation by the imaging control unit 104 according to Embodiment 2. In Figure 7, parts that are the same as those shown in Figure 3 are denoted by the same reference numerals and their descriptions are omitted.

[0087] Metadata M1 to M15 are metadata generated by the imaging control unit 104. Metadata M1 to M7 are data indicating each imaging condition set at the start of imaging, and are generated in association with the first frame data F1 after the start of imaging.

[0088] In the example shown in Figure 7, assume that the imaging conditions related to the black offset level and white balance were changed between the imaging of frame data F1 and the imaging of frame data F2. In this case, the imaging control unit 104 generates metadata M8 (black level) indicating the changed imaging conditions related to the black offset level and metadata M9 (WB) indicating the changed imaging conditions related to the white balance, in association with frame data F2.

[0089] Furthermore, in the example shown in Figure 7, assume that the imaging conditions related to white balance were changed between the imaging of frame data F2 and the imaging of frame data F3. In this case, the imaging control unit 104 generates metadata M10(WB) indicating the changed white balance imaging conditions, associating it with frame data F3.

[0090] Furthermore, in the example shown in Figure 7, assume that the imaging conditions related to shading and distortion were changed between the imaging of frame data F3 and the imaging of frame data F4. In this case, the imaging control unit 104 generates metadata M11 and M12 indicating the changed imaging conditions related to shading, and metadata M13 and M14 indicating the changed imaging conditions related to distortion, in association with frame data F4.

[0091] Furthermore, in the example shown in Figure 7, the imaging conditions were not changed between the imaging of frame data F4 and the imaging of frame data F5. In this case, the imaging control unit 104 does not generate metadata associated with frame data F5.

[0092] <Output control by the output control unit 110 according to Embodiment 2> Figure 8 shows an example of output control by the output control unit 110 according to Embodiment 2. In the example of metadata generation shown in Figure 7, the output control unit 110 outputs frame data and metadata, for example, as shown in Figure 8. In Figure 8, the metadata enclosed in thick lines is metadata of the type for which the immediate output flag described above is set.

[0093] The number of metadata associated with frame data F1 is seven, consisting of metadata M1 to M7, which exceeds the number of metadata that can be attached to frame data F1 (four). In this example, the imaging device 100 first selects metadata M2 and M7, which have the immediate output flag set, and then selects metadata M1 and M3 in order of priority. That is, the imaging device 100 attaches metadata M1 to M3 and M7 to frame data F1 and outputs it to the external recording device 120 via the external output I / F111.

[0094] Furthermore, the output control unit 110 selects metadata to be added to frame data F2 and output from among metadata M4 to M6 that are associated with frame data F1 but were not added to frame data F1, and metadata M8 and M9 that are associated with frame data F2. At this time, the output control unit 110 first selects metadata M8 and M9, for which the immediate output flag has been set, and then selects metadata M4 and M5 in order of priority. In other words, the imaging device 100 adds metadata M4, M5, M8, and M9 to frame data F2 and outputs it to the external recording device 120 from the external output I / F 111.

[0095] Furthermore, the output control unit 110 adds metadata M6, which is associated with frame data F2 but was not added to frame data F2, and metadata M10, which is associated with frame data F3, to frame data F3, and outputs them to the external recording device 120 via the external output I / F 111.

[0096] Furthermore, the output control unit 110 adds metadata M11 to M14, which is associated with frame data F4, to frame data F4 and outputs it to the external recording device 120 via the external output I / F 111. Alternatively, the output control unit 110 outputs frame data F5 to the external recording device 120 via the external output I / F 111 without adding metadata.

[0097] <Processing by the imaging device 100 according to Embodiment 2> Figure 9 is a flowchart showing an example of processing performed by the imaging device 100 according to Embodiment 2. The imaging device 100 according to Embodiment 2 performs, for example, the processing shown in Figure 9. The processing shown in Figure 9 is performed, for example, by a processor that implements the imaging control unit 104 and the output control unit 110.

[0098] Steps S91 to S97 shown in Figure 9 are the same as steps S51 to S57 shown in Figure 5. However, in step S94, the imaging device 100 selects metadata with an immediate output flag, unoutput metadata associated with the (n-1)th or earlier frame data (unoutput metadata from the previous frame), and metadata associated with the nth frame data with the highest priority. That is, if metadata associated with the nth frame data and with an immediate output flag set exists, the imaging device 100 selects that metadata with the highest priority.

[0099] Thus, the output control unit 110 of the second embodiment outputs metadata of the type with the immediate output flag set before metadata whose output is delayed, thereby preventing delays in the output of metadata that is important in the development process.

[0100] (Embodiment 3) The differences between Embodiment 3 and Embodiments 1 and 2 will be described below.

[0101] <Generation of metadata by the imaging control unit 104 according to Embodiment 3> Figure 10 shows an example of metadata generation by the imaging control unit 104 according to Embodiment 3. In Figure 10, parts that are the same as those shown in Figure 3 are denoted by the same reference numerals and their descriptions are omitted.

[0102] Metadata M1 to M14 are metadata generated by the imaging control unit 104. Metadata M1 to M7 are data indicating each imaging condition set at the start of imaging, and are generated in association with the first frame data F1 after the start of imaging.

[0103] In the example shown in Figure 10, assume that the imaging conditions related to white balance were changed between the imaging of frame data F1 and the imaging of frame data F2. In this case, the imaging control unit 104 generates metadata M8(WB) indicating the changed white balance imaging conditions, associated with frame data F2.

[0104] Furthermore, in the example shown in Figure 10, assume that the imaging conditions related to white balance were changed between the imaging of frame data F2 and the imaging of frame data F3. In this case, the imaging control unit 104 generates metadata M9(WB) indicating the changed white balance imaging conditions, associated with frame data F3.

[0105] Furthermore, in the example shown in Figure 10, assume that the imaging conditions related to shading, distortion, and white balance are changed between the imaging of frame data F3 and the imaging of frame data F4. In this case, the imaging control unit 104 generates metadata M10 and M11 indicating the changed imaging conditions related to shading, metadata M12 and M13 indicating the changed imaging conditions related to distortion, and metadata M14 indicating the changed imaging conditions related to white balance, in association with frame data F4.

[0106] Furthermore, in the example shown in Figure 10, the imaging conditions were not changed between the imaging of frame data F4 and the imaging of frame data F5. In this case, the imaging control unit 104 does not generate metadata associated with frame data F5.

[0107] <Output control by the output control unit 110 according to Embodiment 3> Figure 11 shows an example of output control by the output control unit 110 according to Embodiment 3. In the example of metadata generation shown in Figure 10, the output control unit 110 outputs frame data and metadata, for example, as shown in Figure 11.

[0108] The number of metadata associated with frame data F1 is seven, consisting of metadata M1 to M7, which exceeds the number of metadata that can be added to frame data F1 (four). In this case, the output control unit 110, similar to the example shown in Figure 4, adds four metadata M1 to M4, selected in order of priority from metadata M1 to M7, to frame data F1 and outputs them to the external recording device 120 via the external output I / F 111.

[0109] Furthermore, the output control unit 110 discards the metadata M7 that was not added to the frame data F1 because metadata M8, which indicates the same white balance imaging conditions as metadata M7, is associated with the next frame data F2 following frame data F1. The output control unit 110 also adds the metadata M5 and M6 that were not added to the frame data F1, along with metadata M8 associated with frame data F2, to the frame data F2 and outputs it to the external recording device 120 via the external output I / F 111.

[0110] In other words, metadata M7, which is associated with frame data F1 but was not attached to frame data F1, becomes a candidate for metadata to be attached to frame data F2 and output. However, if there is similar metadata M8 associated with frame data F2, metadata M8 will overwrite metadata M7 and be attached to frame data F2 and output.

[0111] Furthermore, the output control unit 110 adds metadata M9, ​​which is associated with the frame data F3, to the frame data F3 and outputs it to the external recording device 120 via the external output I / F 111.

[0112] Furthermore, the output control unit 110 adds four metadata M10 to M13, selected in order of priority from the metadata M10 to M14 associated with the frame data F4, to the frame data F4, and outputs them to the external recording device 120 via the external output I / F 111.

[0113] Furthermore, the output control unit 110 adds the metadata M14 that was not added to the frame data F4 to the next frame data F5, and outputs it to the external recording device 120 via the external output I / F 111.

[0114] <Processing by the imaging device 100 according to Embodiment 3> Figure 12 is a flowchart showing an example of processing performed by the imaging device 100 according to Embodiment 3. The imaging device 100 according to Embodiment 3 performs, for example, the processing shown in Figure 12. The processing shown in Figure 12 is performed, for example, by a processor that implements the imaging control unit 104 and the output control unit 110.

[0115] Steps S121 to S123 shown in Figure 12 are the same as steps S51 to S53 shown in Figure 5. After step S123, if the imaging device 100 has any unoutput metadata of the same type as the metadata generated in step S123 that is associated with the n-1th frame data or earlier (unoutput metadata from the previous frame), it discards the unoutput metadata from the previous frame (step S124) and proceeds to step S125.

[0116] Steps S125 to S128 shown in Figure 12 are the same as steps S54 to S57 shown in Figure 5. In step S125, metadata discarded in step S124 is not included in the selection.

[0117] As described above, in the output control unit 110 of the embodiment 3, if the imaging conditions indicated by metadata associated with the first frame data that is not added to the first frame data have changed during the imaging of the second frame data after the first frame data, the output control unit 110 adds metadata indicating the imaging conditions after the change to the second frame data. This prevents the output of metadata that has been delayed and is no longer needed for development due to changes in imaging conditions from being output, thereby improving output efficiency.

[0118] (Embodiment 4) The differences between Embodiment 4 and Embodiments 1 to 3 will be described below.

[0119] <Output control by the output control unit 110 according to Embodiment 4> Figure 13 shows an example of output control by the output control unit 110 according to Embodiment 4. In the example of metadata generation shown in Figure 3, the output control unit 110 according to Embodiment 4 outputs frame data and metadata, for example, as shown in Figure 13. The example of output control shown in Figure 13 is the same as the example of output control shown in Figure 4, except that delayed data is added to the metadata that is output with a delay.

[0120] Specifically, the output control unit 110 adds delayed data to metadata M5 and M6, which originally correspond to frame data F1 but are output after being added to frame data F2 (i.e., with a delay), so that the original frame data F1 to which metadata M5 and M6 correspond can be identified. This allows the development process to identify that metadata M5 and M6 are data that should be applied from frame data F1, and therefore, even if the output of metadata M5 and M6 is delayed, the impact on the development process results can be suppressed.

[0121] In the example shown in Figure 13, the delay data attached to metadata M5 and M6 is data (delay 1) indicating that the frame data F2 to which metadata M5 and M6 are attached is delayed by one frame relative to the frame data F1 to which metadata M5 and M6 originally correspond. This helps to suppress the large amount of data in the delay data. However, the delay data attached to metadata M5 and M6 may also be the frame number of the frame data F1 to which metadata M5 and M6 originally correspond.

[0122] Similarly, the output control unit 110 adds delayed data to the metadata M13, which originally corresponds to the frame data F4 but is output with a delay (i.e., appended to the frame data F5), that allows the metadata M13 to identify the frame data F4 it originally corresponds to.

[0123] This allows the image processing to identify that metadata M13 is data that should be applied from frame data F4, thus minimizing the impact on the image processing results even if the output of metadata M13 is delayed.

[0124] <Example of image processing using delayed data> The distortion and white balance processing for frame data F1 is performed using metadata M5-M7 attached to frame data F2. Similarly, the white balance processing for frame data F4 is performed using metadata M13 attached to frame data F5. Other processing steps are the same as those in the output control example shown in Figure 4.

[0125] <Processing by the imaging device 100 according to Embodiment 4> Figure 14 is a flowchart showing an example of processing performed by the imaging device 100 according to Embodiment 4. The imaging device 100 according to Embodiment 4 performs, for example, the processing shown in Figure 14. The processing shown in Figure 14 is performed, for example, by a processor that implements the imaging control unit 104 and the output control unit 110.

[0126] Steps S141 to S144 shown in Figure 14 are the same as steps S51 to S54 shown in Figure 5. Following step S144, if the imaging device 100 has delayed metadata (unoutput metadata from the previous frame) in the metadata selected in step S144, it adds delayed data to that metadata that allows it to identify the frame data to which that metadata corresponds (step S145). Steps S146 to S148 shown in Figure 14 are the same as steps S55 to S57 shown in Figure 5.

[0127] As described above, in Embodiment 4, when the output control unit 110 adds metadata that is not attached to the first frame data from the metadata associated with the first frame data to the second frame data that follows the first frame data, it adds data that can identify the first frame data to the second frame data (for example, metadata attached to the second frame data). As a result, even if the output of metadata is delayed, the metadata can be applied retrospectively to the previous frame data during the development process, thereby suppressing the impact on the results of the development process. In Embodiment 4, it is not necessary to use priority when selecting metadata to be attached to the frame data.

[0128] (Embodiment 5) The differences between Embodiment 5 and Embodiments 1 to 4 will be described below.

[0129] In Embodiment 5, the imaging unit 119 can change the imaging conditions for imaging only for each set of frames.

[0130] <Generation of metadata by the imaging control unit 104 according to Embodiment 5> Figure 15 shows an example of metadata generation by the imaging control unit 104 according to Embodiment 5. In Figure 15, parts that are the same as those shown in Figure 3 are denoted by the same reference numerals and their descriptions are omitted.

[0131] Metadata M1 to M13 are metadata generated by the imaging control unit 104. The imaging control unit 104 changes the imaging conditions of the imaging unit 119 according to the demosaicing image obtained by the demosaicing processing unit 107 and instructions from the user. In the example shown in Figure 15, the imaging conditions can be changed for odd-numbered imaging, but not for even-numbered imaging. For example, even if the imaging control unit 104 receives an operation to change the imaging conditions between frame data F1 and frame data F2, the change will not be applied to the imaging of frame data F2, but will be applied from the imaging of frame data F3.

[0132] Therefore, new metadata is generated and associated only with odd-numbered frame data. In the example shown in Figure 15, metadata is generated and associated with frame data F1, F3, and F5, but metadata is not generated and associated with frame data F2 and F4.

[0133] Metadata M1 to M7 are data that indicates each imaging condition set at the start of imaging, and are generated in association with the first frame data F1 after the start of imaging.

[0134] Furthermore, in the example shown in Figure 15, assume that the imaging conditions related to shading, distortion, and white balance are changed between the imaging of frame data F2 and the imaging of frame data F3. In this case, the imaging control unit 104 generates metadata M8 and M9 indicating the changed imaging conditions related to shading, metadata M10 and M11 indicating the changed imaging conditions related to distortion, and metadata M12 indicating the changed imaging conditions related to white balance, in association with frame data F3.

[0135] Furthermore, in the example shown in Figure 15, assume that the imaging conditions related to white balance were changed between the imaging of frame data F4 and the imaging of frame data F5. In this case, the imaging control unit 104 generates metadata M13 indicating the changed imaging conditions related to white balance, associating it with frame data F5.

[0136] <Generation of metadata by the imaging control unit 104 according to Embodiment 5> Figure 16 shows an example of output control by the output control unit 110 according to Embodiment 5. In Figure 16, parts that are the same as those shown in Figure 4 are denoted by the same reference numerals and their descriptions are omitted.

[0137] The number of metadata associated with frame data F1 is seven, consisting of metadata M1 to M7, which exceeds the number of metadata that can be added to frame data F1 (four). In this case, the output control unit 110, similar to the example shown in Figure 4, adds four metadata M1 to M4, selected in order of priority from metadata M1 to M7, to frame data F1 and outputs them to the external recording device 120 via the external output I / F 111.

[0138] Furthermore, the output control unit 110 adds the metadata M5 to M7 that were not added to the frame data F1 to the next frame data F2, and outputs it to the external recording device 120 via the external output I / F 111.

[0139] Furthermore, the number of metadata associated with frame data F3 is five, consisting of metadata M8 to M12, which exceeds the number of metadata that can be added to frame data F3 (four). In this case, the output control unit 110 adds four metadata M8 to M11, selected from metadata M8 to M12 in order of priority, to frame data F3 and outputs them to the external recording device 120 via the external output I / F 111.

[0140] Furthermore, the output control unit 110 adds the metadata M12 that was not added to the frame data F3 to the next frame data F4, and outputs it to the external recording device 120 via the external output I / F 111.

[0141] Furthermore, the output control unit 110 adds metadata M13, which is associated with the frame data F5, to the frame data F5 and outputs it to the external recording device 120 via the external output I / F 111.

[0142] As described above, the imaging device 100 according to Embodiment 5 allows the imaging conditions of the imaging unit 119 to be changed only for every multiple frames. This reduces the frequency of metadata generation, and enables metadata output even with a small communication capacity, such as during the blanking period for frame data. Furthermore, development processing can be performed considering that the imaging conditions are not changed for specific frame data. Therefore, the impact of delays in metadata output on the results of development processing can be suppressed.

[0143] For example, in the example shown in Figure 16, the metadata M5-M7 attached to frame data F2, which has no changes in imaging conditions, indicates the imaging conditions applied to the preceding frame data F1, and is therefore applied starting from the development process of frame data F1. Similarly, the metadata M12 attached to frame data F4, which has no changes in imaging conditions, indicates the imaging conditions applied to the preceding frame data F3, and is therefore applied starting from the development process of frame data F3.

[0144] (Embodiment 6) The differences between Embodiment 6 and Embodiments 1 to 5 will now be described. The configuration described above involves outputting frame data (RAW video data) and metadata stored in the temporary storage unit 106 of the imaging device 100 to an external recording device 120 outside the imaging device 100, but the configuration is not limited to this.

[0145] <Imaging device 100 according to Embodiment 6> Figure 17 shows an example of an imaging device 100 according to Embodiment 6. In Figure 17, parts that are the same as those shown in Figure 1 are denoted by the same reference numerals and their descriptions are omitted.

[0146] The imaging device 100 according to Embodiment 6 includes an internal output I / F 124, an internal input I / F 125, a recording control unit 122, and a recording medium 123, instead of the external output I / F 111 shown in Figure 1.

[0147] The internal output I / F 124 is an interface such as HDMI, similar to the external output I / F 111 shown in Figure 1, but differs from the external output I / F 111 in that it communicates with the internal input I / F 125 located inside the imaging device 100. The internal input I / F 125 is an interface such as HDMI, similar to the external input I / F 121 of the external recording device 120 shown in Figure 1, but differs from the external input I / F 121 in that it is located inside the imaging device 100.

[0148] The recording control unit 122 and recording medium 123 shown in Figure 17 have the same configuration as the recording control unit 122 and recording medium 123 shown in Figure 1, but are located inside the imaging device 100. In other words, the imaging device 100 shown in Figure 17 has a built-in high-speed, high-capacity recording medium 123, and outputs frame data (RAW video data) and metadata to the recording medium 123 using an interface such as HDMI inside the imaging device 100.

[0149] Even in this configuration, similar to the imaging device 100 shown in Figure 1, metadata indicating the imaging conditions is generated in association with the frame data when the imaging conditions of the imaging unit 119 are changed, and the generated metadata is added to the frame data and output as video data before demosaicing. This reduces the amount of metadata data to be output.

[0150] Therefore, metadata can be output with limited communication capacity, such as during the blanking period for frame data. Furthermore, since it is not necessary to increase the communication capacity during the blanking period, problems such as changes in frame rate or inability to maintain connection compatibility specifications due to increased communication capacity during the blanking period can be suppressed.

[0151] Furthermore, if the amount of metadata associated with a frame data exceeds the amount of data that can be added to that frame data, the imaging device 100 adds selected metadata to the frame data based on the priority given to the metadata type. This allows metadata of the type used in the development process to be output preferentially. Therefore, the impact on the development process results can be suppressed without outputting all metadata for each frame data.

[0152] (Combinations of each embodiment) The embodiments described above can also be implemented in combination. For example, the same processing as in embodiments 2 to 5 can be implemented in the configuration of the imaging device 100 according to embodiment 6.

[0153] (modified version) In the embodiments described above, the process of adding metadata to each frame data as much as possible at that time has been explained, but the process is not limited to this. For example, in the example shown in Figure 4, the imaging device 100 may add only three metadata M1 to M3 to frame data F1 and metadata M4 to M7 to frame data F2.

[0154] As explained above, this specification contains at least the following information:

[0155] (1) An imaging unit that generates video image data as multiple time-sequential frame data, A metadata generation unit generates metadata indicating the imaging conditions in association with the frame data when the imaging conditions of the above imaging unit are changed, The system includes an output unit that adds the above metadata to the above frame data and outputs it as video data before demosaicing, The output unit, when the amount of metadata associated with the frame data exceeds the amount of data that can be added, adds metadata selected from the metadata associated with the frame data based on priority to the frame data. Video recording device.

[0156] (2) (1) The video imaging device described above, The above priority is based on the type of metadata mentioned above. Video recording device.

[0157] (3) (1) or (2) The video imaging device described above, The metadata generation unit generates metadata indicating the imaging conditions of the imaging unit at the start of imaging by the imaging unit, in association with the first frame data after the start of imaging. Video recording device.

[0158] (4) A video imaging device according to any one of (1) to (3), The output unit adds metadata associated with the first frame data generated by the imaging unit that is not added to the first frame data to the second frame data which is later than the first frame data. Video recording device.

[0159] (5) (4) The video imaging device described above, The output unit above prioritizes adding metadata that is not added to the first frame data to the second frame data, out of the metadata associated with the second frame data and the metadata that is not added to the first frame data. Video recording device.

[0160] (6) (4) or (5) a video imaging device, The output unit, if the imaging conditions of the imaging unit indicated by the metadata associated with the first frame data that is not added to the first frame data have been changed during the imaging of the second frame data, adds the metadata indicating the imaging conditions after the change to the second frame data. Video recording device.

[0161] (7) A video imaging device according to any one of (4) to (6), The output unit, when adding metadata to the second frame data that is not attached to the first frame data from the metadata associated with the first frame data, adds data that can identify the first frame data to the second frame data. Video recording device.

[0162] (8) A video imaging device according to any one of (1) to (7), The above imaging unit can change the above imaging conditions for every multiple frames. Video recording device.

[0163] (9) A video imaging device comprising an imaging unit that generates video image data as data of multiple temporally consecutive frames, and a video imaging method, When the imaging conditions of the above imaging unit are changed, metadata indicating the above imaging conditions is generated in association with the above frame data. The above metadata is added to the above frame data and output as video data before demosaicing. If the amount of metadata associated with the above frame data exceeds the amount of data that can be added, the metadata selected from the metadata associated with the above frame data based on priority is added to the above frame data. Video recording method.

[0164] (10) (9) The video capture method described above, The above priority is based on the type of metadata mentioned above. Video recording method.

[0165] (11) (9) or (10) The video imaging method described above, Metadata indicating the imaging conditions of the imaging unit at the start of imaging by the imaging unit is generated in association with the first frame data after the start of imaging. Video recording method.

[0166] (12) A video imaging method described in any one of (9) to (11), The metadata associated with the first frame data generated by the imaging unit, which is not attached to the first frame data, is attached to the second frame data which is later than the first frame data. Video recording method.

[0167] (13) (12) The video imaging method described above, Of the metadata associated with the second frame data and the metadata not to be added to the first frame data, the metadata not to be added to the first frame data is preferentially added to the second frame data. Video recording method.

[0168] (14) (12) or (13) a video imaging method, If the imaging conditions of the imaging unit, indicated by the metadata associated with the first frame data that is not added to the first frame data, are changed during the imaging of the second frame data, the metadata indicating the imaging conditions after the change is added to the second frame data. Video recording method.

[0169] (15) A video imaging method described in any one of (12) to (14), When adding metadata associated with the first frame data that is not attached to the first frame data to the second frame data, data that can identify the first frame data is added to the second frame data. Video recording method.

[0170] (16) A video capture method described in any one of (9) to (15), The above imaging unit can change the above imaging conditions for every multiple frames. Video recording method.

[0171] (17) A video imaging program for a video imaging device, which includes an imaging unit that generates video image data as multiple temporally consecutive frame data, The processor of the above video imaging device, When the imaging conditions of the above imaging unit are changed, metadata indicating the above imaging conditions is generated in association with the above frame data. The above metadata is added to the above frame data and output as video data before demosaicing. If the amount of metadata associated with the above frame data exceeds the amount of data that can be added, the metadata selected from the metadata associated with the above frame data based on priority is added to the above frame data. A video capture program for executing processing. [Explanation of Symbols]

[0172] 100 Imaging device 101 Imaging lens system 102 Image sensor 103 ADC 104 Imaging control unit 105 RAW Correction Section 106 Temporary storage 107 Demosaicing Processing Unit 108 Image Correction Unit 109 Monitors 110 Output control unit 111 External Output Interface 119 Imaging Unit 120 External recording device 121 External Input I / F 122 Recording Control Unit 123 Recording media 124 Internal Output Interface 125 Internal Input Interface F1~F5 Frame Data M1-M15 Metadata

Claims

1. An imaging unit that generates image data as multiple time-sequential frame data, A metadata generation unit generates metadata indicating imaging conditions in association with the frame data, The system includes an output unit that adds the metadata to the frame data and outputs it as image data, The output unit adds metadata associated with the first frame data generated by the imaging unit that is not added to the first frame data to the second frame data which is later than the first frame data. Imaging device.

2. The imaging apparatus according to claim 1, The output unit, when the amount of metadata associated with the frame data exceeds the amount of data that can be added, adds the metadata selected from the metadata associated with the frame data based on selection criteria to the frame data. Imaging device.

3. The imaging apparatus according to claim 2, The selection criteria are priority based on the type of metadata. Imaging device.

4. An imaging method using an imaging device that includes an imaging unit that generates image data as multiple temporally consecutive frame data, Metadata indicating the imaging conditions is generated in association with the aforementioned frame data. The metadata is added to the frame data and output as image data. The metadata associated with the first frame data generated by the imaging unit, which is not attached to the first frame data, is attached to the second frame data which is later than the first frame data. Imaging method.

5. An imaging program for an imaging device comprising an imaging unit that generates image data as multiple temporally consecutive frame data, The processor of the aforementioned imaging device: Metadata indicating the imaging conditions is generated in association with the aforementioned frame data. The metadata is added to the frame data and output as image data. The metadata associated with the first frame data generated by the imaging unit, which is not attached to the first frame data, is attached to the second frame data which is later than the first frame data. An imaging program to execute the processing.

Images