Dynamic image pickup device, dynamic image pickup method, and recording medium

Abstract

The present application provides a kind of dynamic image camera device, dynamic image camera method and recording medium capable of inhibiting the influence of the delay of the output of metadata on development processing.It generates image data of dynamic image in the form of multiple frame data continuously in time by imaging unit (119).Imaging control unit (104) generates metadata indicating the shooting condition in a manner corresponding to frame data when the shooting condition of imaging unit (119) is changed.The output control unit (110) and external output I / F (111) attach metadata to frame data and output as dynamic image data before demosaicing.In the case where the data volume of metadata corresponding to frame data exceeds the attachable data volume, the output control unit (110) and external output I / F (111) attach metadata selected according to priority from metadata corresponding to the frame data to the frame data.

Description

Technical Field

[0001] This invention relates to a motion image capturing device, a motion image capturing method, and a computer-readable recording medium storing a motion image capturing program. Background Technology

[0002] Patent Document 1 describes the following: In a structure that inputs RAW motion image data and image correction data output from a camera unit into a recording unit or an external recording device for recording, if the data size of the image correction data exceeds the data size that can be transmitted in one frame, the image correction data is divided into multiple frames of arbitrary size and transmitted.

[0003] Patent document 2 describes the following: The development parameters (development parameters for dynamic images) required for playing (developing) RAW dynamic image data are generated in frame units based on RAW dynamic image data, and the RAW dynamic image data and the development parameters for the RAW image frames constituting the RAW dynamic image data are recorded in the recording medium.

[0004] Previous technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2017-163307

[0007] Patent Document 2: Japanese Patent Application Publication No. 2011-244423 Summary of the Invention

[0008] One embodiment of the present invention provides a dynamic image capturing device, a dynamic image capturing method, and a recording medium readable by a computer that can suppress the influence of the delay in the output of metadata such as developing parameters on the developing process.

[0009] means for solving technical problems

[0010] A motion picture camera apparatus according to one embodiment of the present invention includes: a camera unit that generates image data of a motion picture in the form of a plurality of frames of data that are consecutive in time; a metadata generation unit that generates metadata representing the shooting conditions in a manner that establishes a correspondence with the frame data when the shooting conditions of the camera unit are changed; and an output unit that appends the metadata to the frame data and outputs it as motion picture data before de-mosaicing. If the amount of metadata corresponding to the frame data exceeds the amount of data that can be appended, the output unit appends metadata selected from the metadata corresponding to the frame data according to priority to the frame data.

[0011] One embodiment of the present invention relates to a dynamic image capturing method performed by a dynamic image capturing apparatus having an image data unit that generates dynamic images in the form of multiple frames of data that are consecutive in time. The dynamic image capturing method includes the following steps: generating metadata representing the shooting conditions in a manner that establishes a correspondence with the frame data when the shooting conditions of the camera unit are changed; and appending the metadata to the frame data and outputting it as dynamic image data before de-mosaicing; if the amount of metadata that has a correspondence with the frame data exceeds the amount of data that can be appended, appending metadata selected from the metadata that has a correspondence with the frame data according to priority to the frame data.

[0012] The dynamic image camera program of one embodiment of the present invention is a dynamic image camera program for a dynamic image camera device having a camera unit that generates dynamic image data in the form of multiple frames of data that are consecutive in time. The program is used to cause the processor of the dynamic image camera device to perform the following processing: generating metadata representing the shooting conditions in a manner that establishes a correspondence with the frame data when the shooting conditions of the camera unit are changed; and appending the metadata to the frame data and outputting it as dynamic image data before de-mosaicing; and when the amount of metadata that has a correspondence with the frame data exceeds the amount of data that can be appended, appending metadata selected from the metadata that has a correspondence with the frame data according to priority to the frame data.

[0013] Invention Effects

[0014] According to one embodiment of the present invention, a motion picture camera device, a motion picture camera method, and a recording medium readable by a computer that can suppress the effect of the delay in metadata output on the developing process can be provided. Attached Figure Description

[0015] Figure 1 This is a diagram illustrating an example of the camera device 100 according to the first embodiment.

[0016] Figure 2 This is a diagram showing a specific example of the category of metadata generated by the camera control unit 104.

[0017] Figure 3 This diagram illustrates an example of metadata generation by the camera control unit 104 according to the first embodiment.

[0018] Figure 4 This diagram illustrates an example of output control performed by the output control unit 110 according to the first embodiment.

[0019] Figure 5 This is a flowchart illustrating an example of the processing performed by the camera device 100 according to the first embodiment.

[0020] Figure 6 This is a diagram illustrating an example of an instant output tag for metadata involved in the second embodiment.

[0021] Figure 7 This diagram illustrates an example of metadata generation by the camera control unit 104 according to the second embodiment.

[0022] Figure 8 This diagram illustrates an example of output control performed by the output control unit 110 according to the second embodiment.

[0023] Figure 9 This is a flowchart illustrating an example of the processing performed by the camera device 100 according to the second embodiment.

[0024] Figure 10 This diagram illustrates an example of metadata generation by the camera control unit 104 according to the third embodiment.

[0025] Figure 11 This diagram illustrates an example of output control performed by the output control unit 110 according to the third embodiment.

[0026] Figure 12 This is a flowchart illustrating an example of the processing performed by the camera device 100 according to the third embodiment.

[0027] Figure 13 This diagram illustrates an example of output control performed by the output control unit 110 according to the fourth embodiment.

[0028] Figure 14 This is a flowchart illustrating an example of the processing performed by the camera device 100 according to the fourth embodiment.

[0029] Figure 15 This diagram illustrates an example of metadata generation by the camera control unit 104 according to the fifth embodiment.

[0030] Figure 16 This diagram illustrates an example of output control performed by the output control unit 110 according to the fifth embodiment.

[0031] Figure 17 This is a diagram illustrating an example of the camera device 100 according to the sixth embodiment. Detailed Implementation

[0032] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0033] (First Embodiment)

[0034] <The camera device 100 according to the first embodiment>

[0035] Figure 1 This diagram illustrates an example of the camera device 100 according to the first embodiment. The camera device 100 is a motion picture camera device capable of generating motion pictures by performing continuous shooting. However, in addition to the function of generating motion pictures, the camera device 100 may also have the function of generating still images.

[0036] An external recording device 120 is disposed outside the camera device 100 and records the RAW motion picture data and metadata output from the camera device 100 (described later). The data output terminal of the camera device 100 is connected to the input terminal of the external recording device 120, for example, via a communication interface such as HDMI (High-Definition Multimedia Interface). HDMI is a registered trademark. Data output from the camera device 100 is not limited to HDMI; it can also be output wirelessly (e.g., UWB (Ultra Wide Band)) or wireless HDMI-SDI (High-Definition Multimedia Interface-Serial Digital Interface).

[0037] The camera device 100 includes a camera unit 119, a camera control unit 104, a RAW correction unit 105, a temporary storage unit 106, a de-mosaic processing unit 107, an image correction unit 108, a monitor 109, an output control unit 110, and an external output I / F 111.

[0038] The camera unit 119 generates image data of a moving image in the form of multiple frames of data that are consecutive in time by performing continuous shooting. Specifically, the camera unit 119 includes an imaging lens system 101, an imaging element 102, and an ADC (Analog / Digital Converter) 103. The ADC 103 may also be built into the imaging element 102.

[0039] The imaging lens system 101 includes a lens for transmitting light from the subject and imaging it onto the imaging element 102. Furthermore, the imaging lens system 101 may include an aperture, an ND (Neutral Density) filter, a focusing lens, a zoom lens, a shifting lens, etc. These movable parts of the imaging lens system 101 are controlled by the camera control unit 104.

[0040] Imaging element 102 converts the optical image formed by imaging lens system 101 into an analog image signal and outputs the converted analog image signal to ADC 103. Imaging element 102 is composed of imaging elements such as CMOS (Complementary Metal-Oxide-Semiconductor) type image sensor or CCD (Charge-Coupled Device) type image sensor.

[0041] Furthermore, the imaging element 102 is equipped with an electronic shutter that serves as a shutter for adjusting the exposure time. The shooting performed by the imaging element 102 is controlled by the camera control unit 104. For example, when shooting moving images, the imaging element 102 continuously shoots in time under the control of the camera control unit 104, and sequentially outputs the resulting analog image signals to the ADC 103.

[0042] The ADC103 converts the analog image signal from the imaging element 102 into digital frame data and outputs the converted frame data to the RAW correction unit 105. The frame data continuously output from the ADC103 is the RAW motion image data before de-mosaicing.

[0043] The camera control unit 104 controls the shooting performed by the camera unit 119 by controlling the imaging lens system 101 and the imaging element 102. For example, the camera control unit 104 controls the focus or exposure of the shooting performed by the camera unit 119 according to instructions from the user. Furthermore, the camera control unit 104 can also automatically control the exposure of the shooting performed by the camera unit 119 based on the de-mosaic image obtained by the de-mosaic processing unit 107 described later.

[0044] Furthermore, the camera control unit 104 constitutes a metadata generation unit that generates metadata representing the shooting conditions performed by the imaging lens system 101 and the imaging element 102. This metadata is used during the development process, in which a de-mosaic image is generated based on RAW video data. Specific examples of the metadata will be described later (for example, see reference...). Figure 2 ).

[0045] For example, the camera control unit 104 generates metadata representing the shooting conditions of the camera unit 119 when shooting begins, in a manner that establishes a correspondence with the initial frame data after shooting begins by the camera unit 119.

[0046] Generating metadata in a way that establishes a correspondence with a certain frame of data means, for example, enabling the output control unit 110 (described later) to determine that the metadata corresponds to that frame of data. As an example, the camera control unit 104 appends the frame number of the frame data corresponding to the metadata to the metadata.

[0047] Furthermore, if the shooting conditions of the camera unit 119 are changed, the camera control unit 104 generates metadata representing the changed shooting conditions in a manner that establishes a correspondence with the frame data after the change. On the other hand, if the shooting conditions of the camera unit 119 are not changed, the camera control unit 104 does not generate new metadata. The camera control unit 104 performs this processing according to the category of metadata.

[0048] Therefore, when shooting begins by the camera unit 119, metadata representing the shooting conditions of the camera unit 119 at that point in time is generated in a manner that establishes a correspondence with the initial frame data. Furthermore, thereafter, metadata representing the changed shooting conditions is generated only when the shooting conditions are changed, in a manner that establishes a correspondence with the frame data at that point in time. The camera control unit 104 outputs the generated metadata to the temporary storage unit 106.

[0049] The RAW correction unit 105 corrects the frame data (RAW dynamic image data) output from the ADC 103. The correction performed by the RAW correction unit 105 is a correction of the RAW dynamic image data before demosaicing, such as pixel value correction, defect pixel correction, shadow correction, etc., corresponding to the characteristics of the imaging element 102. The RAW correction unit 105 outputs the corrected frame data to the temporary storage unit 106.

[0050] Temporary storage unit 106 temporarily stores frame data output from RAW correction unit 105 and metadata output from camera control unit 104. For example, temporary storage unit 106 may be implemented using memory (any type) such as RAM (Random Access Memory). Furthermore, temporary storage unit 106 may be implemented using multiple memories. For example, temporary storage unit 106 may be implemented using memory that stores frame data and memory that stores metadata.

[0051] The demosaic processing unit 107 generates a demosaic motion image by performing demosaic processing on the frame data stored in the temporary storage unit 106, and outputs the generated demosaic motion image to the image correction unit 108. Frame data that has undergone defect pixel correction or shadow correction, etc., is output from the RAW correction unit 105. At this point in time, each pixel is only one of the colors R, G, and B. Therefore, the other two colors are supplemented from the surrounding pixels, so that each pixel has data of all three colors. For example, if a pixel only has the color R, then there is no data for G and B, so the data for the G color of that pixel is supplemented by data from the surrounding G pixels or predicted based on the surrounding G pixels. Thus, all pixels can have the three colors R, G, and B respectively. This is called demosaic processing.

[0052] The image correction unit 108 performs various image corrections on the demosaic motion image output from the demosaic processing unit 107, and outputs the image-corrected demosaic motion image to the monitor 109. The image corrections performed by the image correction unit 108 are processing of the demosaic motion image after demosaic processing, such as peripheral light attenuation correction, color correction, contour enhancement, noise removal, gamma correction, de-Bayer processing, compression, etc.

[0053] The monitor 109 displays the de-pixelated motion image output from the image correction unit 108 to the user. This allows the user to view the motion image being captured in real-time while it is being captured.

[0054] The output control unit 110 and the external output I / F 111 constitute the output unit of one embodiment of the present invention. The output control unit 110 reads the frame data and metadata stored in the temporary storage unit 106 and appends the metadata to the frame data. Then, the output control unit 110 outputs the frame data with appended metadata from the external output I / F 111 to the external recording device 120 as RAW motion picture data before de-mosaicing.

[0055] For example, the output control unit 110 continuously outputs frame data in the form of an image signal from the external output I / F 111, and simultaneously outputs metadata from the external output I / F 111 during the blanking period of the image signal. That is, the output control unit 110 controls the external output I / F 111 to insert the output of metadata into the output of frame data. Thus, metadata can be appended to the frame data for output.

[0056] External Output I / F111 is a communication interface used for communicating with the external recording device 120. As an example, External Output I / F111 communicates via HDMI. Under the control of the output control unit 110, External Output I / F111 outputs frame data and metadata to the external recording device 120.

[0057] In addition, the camera device 100 may have an internal memory for storing de-pixelated moving images, etc. Furthermore, the camera device 100 may have a user interface for accepting various operations from the user or outputting various data to the user.

[0058] Furthermore, the camera device 100 may include a microphone that converts ambient sound into electrical signals. In this case, the electrical signals obtained through the microphone can be converted into digital sound data, which, along with frame data and metadata, is output from the external output I / F 111 to the external recording device 120 and recorded by the external recording device 120.

[0059] The external recording device 120 includes an external input I / F 121, a recording control unit 122, and a recording medium 123. The external input I / F 121 reads frame data and metadata output from the external output I / F 111 of the camera device 100, and outputs the read frame data and metadata to the recording control unit 122.

[0060] The recording control unit 122 controls the recording of frame data and metadata output from the external input I / F 121 into the recording medium 123. The recording medium 123 is a high-capacity recording medium capable of high-speed writing to record large amounts of frame data continuously output from the camera device 100 in real time. For example, the recording medium 123 can be implemented as a memory card or an SSD (Solid State Drive).

[0061] Furthermore, although not illustrated, the external recording device 120 has an external output I / F that outputs frame data and metadata stored in the recording medium 123 to an external data processing device (e.g., a personal computer) that is different from the camera device 100 and the external recording device 120. Thus, development processing based on the frame data and metadata stored in the recording medium 123 can be performed in the data processing device.

[0062] Alternatively, the external recording device 120 may be a data processing device (e.g., a personal computer) having a processor and memory for performing development processing. In this case, the external recording device 120 is capable of performing development processing based on frame data and metadata stored in the recording medium 123.

[0063] <Hardware Structure of Camera Device 100>

[0064] The camera control unit 104, RAW correction unit 105, de-mosaic processing unit 107, image correction unit 108, and output control unit 110 in the camera device 100 are implemented by a processor that operates in cooperation with the memory of the camera device 100.

[0065] The processor can be, for example, a CPU (Central Processing Unit), an MPU (Microprocessor Unit), a FPGA (Field Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit). This processor executes a program stored in memory to perform the functions of the processing unit in the camera device 100. More specifically, the structure of these various processors is a circuit composed of combined semiconductor elements and other circuit components. Furthermore, the processor can be a combination of multiple processors of the same or different types.

[0066] Memory is implemented using RAM, ROM (Read Only Memory), flash memory, etc. Memory stores programs executed by the processor or data used by the processor. Furthermore, the memory can be a combination of multiple memory types, either the same or different.

[0067] <Specific examples of the categories of metadata generated by camera control unit 104>

[0068] Figure 2 This is a diagram illustrating specific examples of categories of metadata generated by the camera control unit 104. For example, the camera control unit 104 generates representations of... Figure 2 The metadata shown includes the black offset level, the coefficient for converting RAW values ​​to color scales, white balance parameters, lens correction parameters, color conversion parameters, gamma correction parameters, noise correction parameters, timecode, shooting date and time, product name, and other relevant shooting conditions.

[0069] Furthermore, these metadata items are pre-defined with priorities according to their categories and stored in the memory of the camera device 100. The priority indicates the degree to which they should be output to the external recording device 120.

[0070] For example, the priority of each category of metadata is set so that the greater the impact of the metadata on the development process, the higher the priority. In other words, the greater the impact of the metadata on the development process, the higher the priority of output. Higher priority metadata is output first, meaning that for metadata with the same conditions other than priority, the metadata with the higher priority is output first.

[0071] For example, metadata processed earlier in the development process has a greater impact on the image quality of the de-mosaic motion picture obtained through development, and therefore is given higher priority. Figure 2 In the example shown, the highest priority is set for the black offset level used in the developing process. On the other hand, the lowest priority is set for product names that are not directly used in the developing process.

[0072] Furthermore, the types of metadata generated by the camera control unit 104 are not limited to... Figure 2 The examples shown can be set arbitrarily. Furthermore, the priority is not limited to any particular type. Figure 2 The example shown can be arbitrarily set. For example, at least some of the corrections such as pixel value correction, defect pixel correction, and shadow correction performed by the RAW correction unit 105 may not be performed in the RAW correction unit 105, but may be performed during the development process. In this case, the camera control unit 104 generates metadata for performing corrections such as pixel value correction, defect pixel correction, and shadow correction, and outputs it to the temporary storage unit 106.

[0073] <Generate metadata through the camera control unit 104 according to the first embodiment>

[0074] Figure 3 This diagram illustrates an example of metadata generation by the camera control unit 104 according to the first embodiment. Figure 3 The horizontal axis represents time. Frame data F1 to F5 are frame data obtained by the camera unit 119 taking 5 consecutive shots, and are RAW motion image data before de-mosaic processing.

[0075] Metadata M1 to M13 are metadata generated by the camera control unit 104. Metadata M1 to M7 are data representing the shooting conditions set at the start of shooting, generated in a way that establishes a correspondence with the initial frame data F1 after the start of shooting.

[0076] Metadata M1 represents shooting conditions related to the number of pixels (resolution) used in the imaging element 102. Metadata M2 represents shooting conditions related to black offset level (black level). Metadata M3 and M4 represent shooting conditions related to shadows. Metadata M5 and M6 represent shooting conditions related to distortion. Metadata M7 represents shooting conditions related to white balance (WB).

[0077] exist Figure 3 In the example shown, the shooting conditions were not changed between the capture of frame data F1 and the capture of frame data F2. In this case, the camera control unit 104 does not generate metadata that corresponds to frame data F2.

[0078] Furthermore, in Figure 3In the example shown, it is assumed that the shooting conditions related to white balance were changed between the shooting of frame data F2 and the shooting of frame data F3. In this case, the camera control unit 104 generates metadata M8(WB) representing the shooting conditions related to the changed white balance in a manner that establishes a correspondence with frame data F3.

[0079] Furthermore, in Figure 3 In the example shown, assume that the shooting conditions related to shadows, distortion, and white balance are changed between the shooting of frame data F3 and the shooting of frame data F4. In this case, the camera control unit 104 generates metadata M9 and M10 representing the shooting conditions related to the changed shadows, metadata M11 and M12 representing the shooting conditions related to the changed distortion, and metadata M13 representing the shooting conditions related to the changed white balance, in a manner that establishes a correspondence with frame data F4.

[0080] Furthermore, in Figure 3 In the example shown, the shooting conditions were not changed between the capture of frame data F4 and the capture of frame data F5. In this case, the camera control unit 104 does not generate metadata that corresponds to frame data F5.

[0081] <Output control performed by the output control unit 110 according to the first embodiment>

[0082] Figure 4 This diagram illustrates an example of output control performed by the output control unit 110 according to the first embodiment. Figure 3 In the example of generating metadata shown, the output control unit 110 is, for example, as follows: Figure 4 The output frame data and metadata are shown.

[0083] In this example, the data size of one metadata element is set to 28 bytes, and the capacity of the metadata that can be appended to a frame of data and output from the external I / F111 is set to 28 × 4 bytes. That is, it is set to be able to append 4 metadata elements to a frame of data for output. Additionally, for example, the packet container that can store metadata in HDMI is inserted by hardware during blanking, so the upper limit of its capacity is determined, and similarly, there is a limit of 28 × 4 bytes / frame.

[0084] The number of metadata corresponding to frame data F1 is 7, namely metadata M1 to M7, which exceeds the number of metadata that can be attached to frame data F1 (4). That is, the data size of the metadata corresponding to frame data F1 is 28 × 7 [bytes], which exceeds the data size that can be attached to frame data F1 (28 × 4 [bytes]).

[0085] In this case, the output control unit 110 appends metadata selected from metadata M1 to M7, which correspond to the frame data F1, according to the priority of metadata M1 to M7, to the frame data F1, and outputs it to the external recording device 120 from the external output I / F111. In this example, the priority is high in the order of resolution, black level, shadow, distortion, and white balance. In this case, the output control unit 110 appends four metadata M1 to M4, selected from metadata M1 to M7 in high priority order, to the frame data F1, and outputs them to the external recording device 120 from the external output I / F111.

[0086] Furthermore, the output control unit 110 appends the metadata M5 to M7, which are not attached to the frame data F1, to the next frame data F2 of the frame data F1, and outputs it from the external output I / F111 to the external recording device 120.

[0087] Furthermore, the output control unit 110 appends the metadata M8, which corresponds to the frame data F3, to the frame data F3 and outputs it from the external output I / F111 to the external recording device 120.

[0088] The number of metadata corresponding to frame data F4 is 5, namely metadata M9 to M13, which exceeds the number of metadata that can be attached to frame data F4 (4). In this case, the output control unit 110 attaches 4 metadata M9 to M12 selected from metadata M9 to M13 in order of high priority to frame data F4, and outputs them from external output I / F111 to external recording device 120.

[0089] Furthermore, the output control unit 110 appends the metadata M13, which is not attached to the frame data F4, to the next frame data F5 of the frame data F4, and outputs it from the external output I / F111 to the external recording device 120.

[0090] <Specific examples of development processing>

[0091] right Figure 4 The development process in the example shown will be explained. Figure 4 In the example shown, the development processing of frame data F1 related to resolution, black level, and shadow is performed using metadata M1 to M4 attached to frame data F1. Furthermore, since there is no metadata applicable to frame data F1 related to distortion and white balance, for example, regarding distortion and white balance, pre-defined metadata is used for the development processing of frame data F1.

[0092] The development processing of frame data F2 related to distortion and white balance is performed using metadata M5 to M7 attached to frame data F2. Furthermore, the development processing of frame data F2 related to resolution, black level, and shadows is performed using metadata M1 to M4.

[0093] The white balance-related development processing of frame data F3 is performed using metadata M8 attached to frame data F3. Furthermore, the resolution, black level, shadows, and distortion-related development processing of frame data F3 is performed using metadata M1 to M6.

[0094] The development processing related to shadows and distortion of frame data F4 is performed using metadata M9 to M12 attached to frame data F4. Furthermore, the development processing related to resolution, black level, and white balance of frame data F4 is performed using metadata M1, M2, and M8.

[0095] The white balance-related development processing of frame data F5 is performed using metadata M13 attached to frame data F5. Furthermore, the resolution, black level, shadow, and distortion-related development processing of frame data F5 is performed using metadata M1, M2, and M9 to M12.

[0096] <Processing performed by the camera device 100 according to the first embodiment>

[0097] Figure 5 This is a flowchart illustrating an example of the processing performed by the camera device 100 according to the first embodiment. The camera device 100 according to the first embodiment, for example, performs... Figure 5 The processing is shown. Figure 5 The processing shown is performed, for example, by a processor that implements the camera control unit 104 or the output control unit 110.

[0098] First, the camera device 100 controls the camera unit 119 to start capturing moving images (step S51). Next, the camera device 100 sets n to 1 and waits until the camera unit 119 performs the nth capture (step S52). n represents the number of the frame to be processed.

[0099] Next, the camera device 100 generates metadata representing the changing shooting conditions from the (n-1)th to the nth shooting, in a manner that establishes a correspondence with the nth frame data (step S53). However, when n=1 in step S53, the camera device 100 generates metadata for all shooting conditions in a manner that establishes a correspondence with the nth frame data. Furthermore, in step S53, if the shooting conditions have not changed from the (n-1)th to the nth shooting, the camera device 100 does not generate metadata.

[0100] Next, the camera device 100 selects metadata to be output for the nth frame data from the range that can be attached to a frame data (step S54). In step S54, the camera device 100 selects metadata in the order of priority: metadata that has not been output and corresponds to the (n-1)th previous frame data (data that has not been output in the previous frame) and metadata that corresponds to the nth frame data, with higher priority.

[0101] That is, if there is unoutputted metadata that corresponds to the (n-1)th or earlier frame data, the camera device 100 selects that metadata. Furthermore, if there is a gap, the camera device 100 selects the metadata with the highest priority that corresponds to the nth frame data. Additionally, if there is unoutputted metadata corresponding to the (n-1)th or earlier frame data that exceeds the range that can be appended to a single frame data, the camera device 100 selects the metadata with the highest priority from this unoutputted metadata. Furthermore, in cases where the shooting conditions remain unchanged, or where no metadata needs to be selected in step S54, the camera device 100 does not select metadata.

[0102] Next, the camera device 100 appends the metadata selected in step S54 to the nth frame data (step S55). Alternatively, if no metadata is available to be selected in step S54, the camera device 100 does not append metadata.

[0103] Next, the camera device 100 outputs the nth frame data with metadata appended in step S55 to the external recording device 120 from the external output I / F 111 (step S56). Alternatively, if metadata is not appended in step S55, the camera device 100 outputs the nth frame data without metadata appended to the external recording device 120 from the external output I / F 111.

[0104] Next, the camera device 100 increments n by 1 (n = n + 1), and waits until the nth shot is taken by the camera unit 119 (step S57), and then returns to step S53.

[0105] Thus, the camera device 100 of the first embodiment outputs frame data (RAW motion picture data) before de-mosaicing to the external recording device 120. Therefore, even if the camera device 100 does not have a high-speed, high-capacity recording medium, it can still record high-data-rate RAW motion picture data. Furthermore, by outputting metadata along with the frame data to the external recording device 120, the camera device 100 can easily perform development processing based on the frame data and metadata stored in the external recording device 120.

[0106] Furthermore, the camera device 100 generates metadata representing the shooting conditions in a manner that establishes a correspondence with the frame data when the shooting conditions of the camera unit 119 are changed, and appends the generated metadata to the frame data and outputs it as motion image data before de-mosaicing. Therefore, compared to a structure that outputs all metadata for each frame data even if the shooting conditions have not changed, the amount of metadata output can be reduced.

[0107] Therefore, metadata can be output with limited communication capacity, such as the blanking period of frame data. Furthermore, without increasing the communication capacity during the blanking period, issues such as frame rate fluctuations due to increased communication capacity during the blanking period or failure to meet connection compatibility specifications can be suppressed.

[0108] Furthermore, if the amount of metadata corresponding to the frame data exceeds the amount of data that can be appended to that frame data, the imaging device 100 appends metadata selected from the metadata corresponding to the frame data according to the priority of the metadata category to the frame data. This allows for the priority output of metadata of the category used for the development process. Therefore, even if not all metadata is output for each frame data, the impact on the development process result can be suppressed.

[0109] For example, metadata for each shooting condition is generated at the start of shooting. Furthermore, for example, if the focus of the zoom lens is changed, a large amount of new metadata for lens correction is generated. In these cases, it may sometimes be impossible to output the generated metadata as a single frame of data, but according to the imaging device 100, by prioritizing the output of metadata for the category used in the development process, the impact on the result of the development process (e.g., the image quality of the de-mosaic motion picture) can be suppressed.

[0110] Furthermore, the camera device 100 preferentially appends metadata that is not attached to the first frame data, from the metadata that corresponds to the output second frame data and the metadata of the first frame data that is not attached to the second frame data, to the second frame data. This allows for the preferential output of delayed metadata, thereby suppressing the impact of metadata output delay on the development process.

[0111] (Second Implementation)

[0112] Regarding the second embodiment, the parts that differ from the first embodiment will be described.

[0113] <Instant output tags of metadata involved in the second embodiment>

[0114] Figure 6 This is a diagram illustrating an example of an instantaneous output tag for metadata involved in the second embodiment. For example... Figure 6As shown, each metadata item can be set with an instant output flag according to its category. The instant output flag indicates that the metadata should be appended to the frame data that has a corresponding relationship with that metadata and output (i.e., instant output).

[0115] The number of metadata categories with instant output flags is determined, for example, to be such that all metadata with instant output flags can be appended to a single frame of data. Thus, at least the metadata of the categories with instant output flags can be appended to the frame of data that corresponds to that metadata without delay.

[0116] exist Figure 6 In the example shown, the instant output marker is set only for three metadata related to shooting conditions: black offset level, coefficient for converting RAW values ​​to color scale, and white balance parameters.

[0117] <Generate metadata through the camera control unit 104 according to the second embodiment>

[0118] Figure 7 This diagram illustrates an example of metadata generation via the camera control unit 104 according to the second embodiment. Figure 7 In the middle, to and Figure 3 The same parts are marked with the same symbol and the explanation is omitted.

[0119] Metadata M1 to M15 are metadata generated by the camera control unit 104. Metadata M1 to M7 are data representing the shooting conditions set at the start of shooting, generated in a way that establishes a correspondence with the initial frame data F1 after the start of shooting.

[0120] exist Figure 7 In the example shown, it is assumed that the shooting conditions related to the black offset level and white balance are changed between the shooting of frame data F1 and the shooting of frame data F2. In this case, the camera control unit 104 generates metadata M8 (black level) representing the shooting conditions related to the changed black offset level and metadata M9 (WB) representing the shooting conditions related to the changed white balance, in a manner that establishes a correspondence with frame data F2.

[0121] Furthermore, in Figure 7 In the example shown, it is assumed that the shooting conditions related to white balance were changed between the shooting of frame data F2 and the shooting of frame data F3. In this case, the camera control unit 104 generates metadata M10(WB) representing the shooting conditions related to the changed white balance in a manner that establishes a correspondence with frame data F3.

[0122] Furthermore, in Figure 7In the example shown, it is assumed that the shooting conditions related to shadows and distortion were changed between the shooting of frame data F3 and the shooting of frame data F4. In this case, the camera control unit 104 generates metadata M11 and M12 representing the shooting conditions related to the changed shadows and metadata M13 and M14 representing the shooting conditions related to the changed distortion, in a manner that establishes a correspondence with frame data F4.

[0123] Furthermore, in Figure 7 In the example shown, the shooting conditions were not changed between the capture of frame data F4 and the capture of frame data F5. In this case, the camera control unit 104 does not generate metadata that corresponds to frame data F5.

[0124] <Output control performed by the output control unit 110 according to the second embodiment>

[0125] Figure 8 This diagram illustrates an example of output control performed by the output control unit 110 according to the second embodiment. Figure 7 In the example of generating metadata shown, the output control unit 110 is, for example, as follows: Figure 8 The output frame data and metadata are shown. Figure 8 In the text, the metadata enclosed in thick circles represents the metadata of the categories that have the aforementioned instant output tags set.

[0126] The number of metadata corresponding to frame data F1 is seven, namely metadata M1 to M7, which exceeds the number of metadata that can be attached to frame data F1 (four). In this example, the camera device 100 first selects metadata M2 and M7, which are set with instant output flags, and then selects metadata M1 and M3 in order of higher priority. That is, the camera device 100 attaches metadata M1 to M3 and M7 to frame data F1 and outputs it from external output I / F111 to external recording device 120.

[0127] Furthermore, the output control unit 110 selects metadata to be appended to frame data F2 and output from metadata M4 to M6 that correspond to frame data F1 but are not attached to frame data F1, and metadata M8 and M9 that correspond to frame data F2. At this time, the output control unit 110 first selects metadata M8 and M9 that have set an instant output flag, and then selects metadata M4 and M5 in order of higher priority. That is, the camera device 100 appends metadata M4, M5, M8, and M9 to frame data F2 and outputs it from external output I / F111 to external recording device 120.

[0128] Furthermore, the output control unit 110 attaches metadata M6, which is associated with frame data F2 but not attached to frame data F2, and metadata M10, which is associated with frame data F3, to frame data F3, and outputs them from external output I / F111 to external recording device 120.

[0129] Furthermore, the output control unit 110 appends metadata M11 to M14, which correspond to the frame data F4, to the frame data F4, and outputs it to the external recording device 120 from the external output I / F111. Conversely, the output control unit 110 outputs frame data F5 to the external recording device 120 from the external output I / F111 without appending metadata.

[0130] <Processing performed by the camera device 100 according to the second embodiment>

[0131] Figure 9 This is a flowchart illustrating an example of the processing performed by the camera device 100 according to the second embodiment. The camera device 100 according to the second embodiment, for example, performs... Figure 9 The processing is shown. Figure 9 The processing shown is performed, for example, by a processor that implements the camera control unit 104 or the output control unit 110.

[0132] Figure 9 The steps S91 to S97 shown are Figure 5 Steps S51 to S57 are the same. However, in step S94, the camera device 100 selects metadata with the highest priority among metadata with an instant output flag, metadata that has a correspondence with the (n-1)th previous frame data (unoutput metadata from the previous frame), and metadata that has a correspondence with the nth frame data. That is, if there is metadata that has a correspondence with the nth frame data and has an instant output flag, the camera device 100 selects that metadata with the highest priority.

[0133] Thus, the output control unit 110 of the second embodiment can prevent the output of important metadata from being delayed during the development process by outputting metadata of categories with instant output flags before the output of delayed metadata.

[0134] (Third Implementation)

[0135] Regarding the third embodiment, the parts that differ from the first and second embodiments will be described.

[0136] <Generate metadata through the camera control unit 104 according to the third embodiment>

[0137] Figure 10This diagram illustrates an example of metadata generation via the camera control unit 104 according to the third embodiment. Figure 10 In the middle, to and Figure 3 The same parts are marked with the same symbol and the explanation is omitted.

[0138] Metadata M1 to M14 are metadata generated by the camera control unit 104. Metadata M1 to M7 are data representing the shooting conditions set at the start of shooting, generated in a way that establishes a correspondence with the initial frame data F1 after the start of shooting.

[0139] exist Figure 10 In the example shown, it is assumed that the shooting conditions related to white balance were changed between the shooting of frame data F1 and the shooting of frame data F2. In this case, the camera control unit 104 generates metadata M8(WB) representing the shooting conditions related to the changed white balance in a manner that establishes a correspondence with frame data F2.

[0140] Furthermore, in Figure 10 In the example shown, it is assumed that the shooting conditions related to white balance were changed between the shooting of frame data F2 and the shooting of frame data F3. In this case, the camera control unit 104 generates metadata M9(WB) representing the shooting conditions related to the changed white balance in a manner that establishes a correspondence with frame data F3.

[0141] Furthermore, in Figure 10 In the example shown, assume that the shooting conditions related to shadows, distortion, and white balance are changed between the shooting of frame data F3 and the shooting of frame data F4. In this case, the camera control unit 104 generates metadata M10 and M11 representing the shooting conditions related to the changed shadows, metadata M12 and M13 representing the shooting conditions related to the changed distortion, and metadata M14 representing the shooting conditions related to the changed white balance, in a manner that establishes a correspondence with frame data F4.

[0142] Furthermore, in Figure 10 In the example shown, the shooting conditions were not changed between the capture of frame data F4 and the capture of frame data F5. In this case, the camera control unit 104 does not generate metadata that corresponds to frame data F5.

[0143] <Output control performed by the output control unit 110 according to the third embodiment>

[0144] Figure 11 This diagram illustrates an example of output control performed by the output control unit 110 according to the third embodiment. Figure 10 In the example of generating metadata shown, the output control unit 110 is, for example, as follows: Figure 11 The output frame data and metadata are shown.

[0145] The number of metadata items corresponding to frame data F1 is 7 (M1 to M7), which exceeds the number of metadata items that can be attached to frame data F1 (4 items). In this case, with Figure 4 Similarly, in the example shown, the output control unit 110 will select four metadata M1 to M4 from the metadata M1 to M7 in order of high priority and attach them to the frame data F1, and output them from the external output I / F111 to the external recording device 120.

[0146] Furthermore, the output control unit 110 establishes a correspondence between metadata M8, which represents shooting conditions related to white balance and is identical to metadata M7, and the next frame data F2 of frame data F1, thus discarding metadata M7 that was not attached to frame data F1. The output control unit 110 then attaches metadata M5 and M6, which were not attached to frame data F1, and metadata M8, which is associated with frame data F2, to frame data F2, and outputs them from external output I / F111 to external recording device 120.

[0147] In other words, metadata M7, which has a corresponding relationship with frame data F1 but is not attached to frame data F1, becomes a candidate for metadata to be attached to frame data F2 and output. However, if there is metadata M8 of the same category that has a corresponding relationship with frame data F2, metadata M8 is overwritten onto metadata M7 and attached to frame data F2 and output.

[0148] Furthermore, the output control unit 110 appends metadata M9, ​​which corresponds to the frame data F3, to the frame data F3 and outputs it from the external output I / F111 to the external recording device 120.

[0149] Furthermore, the output control unit 110 will select four metadata M10 to M13 from the metadata M10 to M14 that have a corresponding relationship with the frame data F4 in order of high priority and attach them to the frame data F4, and output them from the external output I / F111 to the external recording device 120.

[0150] Furthermore, the output control unit 110 appends the metadata M14, which is not attached to the frame data F4, to the next frame data F5 of the frame data F4, and outputs it from the external output I / F111 to the external recording device 120.

[0151] <Processing performed by the camera device 100 according to the third embodiment>

[0152] Figure 12 This is a flowchart illustrating an example of the processing performed by the camera device 100 according to the third embodiment. The camera device 100 according to the third embodiment, for example, performs... Figure 12 The processing is shown. Figure 12The processing shown is performed, for example, by a processor that implements the camera control unit 104 or the output control unit 110.

[0153] Figure 12 The steps S121 to S123 shown are... Figure 5 The steps S51 to S53 shown are the same. Immediately following step S123, if there is unoutputted metadata (unoutputted metadata of the previous frame) of the same category as the metadata generated in step S123 that corresponds to the frame data before the (n-1)th time, the camera device 100 discards the unoutputted metadata of the previous frame (step S124) and proceeds to step S125.

[0154] Figure 12 Steps S125 to S128 shown are Figure 5 Steps S54 to S57 are the same. In step S125, the metadata discarded in step S124 is not selected.

[0155] Thus, if the shooting conditions indicated by metadata not attached to the first frame data in the metadata that corresponds to the first frame data are changed when the second frame data is captured after the first frame data, the output control unit 110 of the third embodiment attaches the metadata indicating the changed shooting conditions from the metadata before and after the change to the second frame data. This prevents the output of metadata whose shooting conditions have changed and are no longer used for development from being delayed, thereby improving output efficiency.

[0156] (Fourth implementation)

[0157] Regarding the fourth embodiment, the parts that differ from the first to third embodiments will be described.

[0158] <Output control performed by the output control unit 110 according to the fourth embodiment>

[0159] Figure 13 This diagram illustrates an example of output control performed by the output control unit 110 according to the fourth embodiment. Figure 3 In the example of generating metadata shown, the output control unit 110 involved in the fourth embodiment, for example... Figure 13 The output frame data and metadata are shown. Aside from appending the delay data to the metadata of the delayed output, Figure 13 The example of output control shown is similar to Figure 4 The example shown is the same for output control.

[0160] Specifically, the output control unit 110 adds delayed data to the metadata M5 and M6, which originally corresponded to frame data F1 but were appended to frame data F2 and output (i.e., delayed output), to determine the original frame data F1 to which the metadata M5 and M6 correspond. Therefore, it is possible to determine during the development process that the metadata M5 and M6 are data applicable from frame data F1, and thus, even if the output of the metadata M5 and M6 is delayed, the impact on the development process result can be suppressed.

[0161] exist Figure 13 In the example shown, the delayed data appended to metadata M5 and M6 represents frame data F2 of appended metadata M5 and M6 that is delayed by one frame relative to the original frame data F1 corresponding to metadata M5 and M6 (delay 1). This helps to suppress the increase in the amount of delayed data. However, the delayed data appended to metadata M5 and M6 could also be the frame number of the original frame data F1 corresponding to metadata M5 and M6, etc.

[0162] Similarly, the output control unit 110 adds delayed data to the metadata M13, which originally corresponded to frame data F4 but was appended to frame data F5 and output (i.e., delayed output), to determine the original frame data F4 to which the metadata M13 originally corresponded.

[0163] Therefore, it is possible to determine the metadata M13 as the data that should be applied from the frame data F4 during the development process, so that even if the output of the metadata M13 is delayed, the impact on the result of the development process can be suppressed.

[0164] <Example of developing using delayed data>

[0165] The distortion and white balance related development processing of frame data F1 is performed using metadata M5 to M7 appended to frame data F2. Furthermore, the white balance related development processing of frame data F4 is performed using metadata M13 appended to frame data F5. Other development processing is similar to... Figure 4 The development process is the same in the example of output control shown.

[0166] <Processing performed by the camera device 100 according to the fourth embodiment>

[0167] Figure 14 This is a flowchart illustrating an example of the processing performed by the camera device 100 according to the fourth embodiment. The camera device 100 according to the fourth embodiment, for example, performs... Figure 14 The processing is shown. Figure 14 The processing shown is performed, for example, by a processor that implements the camera control unit 104 or the output control unit 110.

[0168] Figure 14 The steps S141 to S144 shown are... Figure 5 Steps S51 to S54 are the same. Immediately following step S144, if there is delayed metadata (unoutputted metadata from the previous frame) in the metadata selected in step S144, the camera device 100 appends the delayed data of the frame data corresponding to the metadata to the metadata (step S145). Figure 14 Steps S146 to S148 shown are Figure 5 The steps S55 to S57 shown are the same.

[0169] Thus, when metadata that is not attached to the first frame data is appended to the second frame data following the first frame data, the output control unit 110 of the fourth embodiment can determine whether the data of the first frame data is appended to the second frame data (e.g., the metadata appended to the second frame data). Therefore, even if the output of metadata is delayed, it is possible to trace back to the metadata applicable to the previous frame data during the development process, thereby suppressing any impact on the development process results. Furthermore, in the fourth embodiment, priority may not be used when selecting metadata to be appended to the frame data.

[0170] (Fifth Embodiment)

[0171] Regarding the fifth embodiment, the parts that differ from the first to fourth embodiments will be described.

[0172] In the fifth embodiment, the camera unit 119 is able to change the shooting conditions only by shooting multiple frames.

[0173] <Generate metadata through the camera control unit 104 according to the fifth embodiment>

[0174] Figure 15 This diagram illustrates an example of metadata generation via the camera control unit 104 according to the fifth embodiment. Figure 15 In the middle, to and Figure 3 The same parts are marked with the same symbol and the explanation is omitted.

[0175] Metadata M1 to M13 are metadata generated by the camera control unit 104. The camera control unit 104 changes the shooting conditions performed by the camera unit 119 based on the de-mosaic image obtained by the de-mosaic processing unit 107 or instructions from the user. Figure 15 In the example shown, the shooting conditions can be changed in odd-numbered shots, but not in even-numbered shots. For example, even if an operation to change the shooting conditions is received between frame data F1 and frame data F2, the camera control unit 104 will not apply the change during the shooting of frame data F2, but will apply it starting from the shooting of frame data F3.

[0176] Therefore, the new metadata is generated in a way that establishes a correspondence only with the data of the odd-numbered frames. Figure 15 In the example shown, metadata was generated in a way that establishes a correspondence with frame data F1, F3, and F5, but metadata that establishes a correspondence with frame data F2 and F4 was not generated.

[0177] Metadata M1 to M7 represent the shooting conditions that were set at the start of shooting, and are generated in a way that establishes a correspondence with the initial frame data F1 after shooting begins.

[0178] Furthermore, in Figure 15 In the example shown, assume that the shooting conditions related to shadows, distortion, and white balance are changed between the shooting of frame data F2 and the shooting of frame data F3. In this case, the camera control unit 104 generates metadata M8 and M9 representing the shooting conditions related to the changed shadows, metadata M10 and M11 representing the shooting conditions related to the changed distortion, and metadata M12 representing the shooting conditions related to the changed white balance, in a manner that establishes a correspondence with frame data F3.

[0179] Furthermore, in Figure 15 In the example shown, it is assumed that the shooting conditions related to white balance were changed between the shooting of frame data F4 and the shooting of frame data F5. In this case, the camera control unit 104 generates metadata M13 representing the shooting conditions related to the changed white balance in a manner that establishes a correspondence with frame data F5.

[0180] <Generate metadata through the camera control unit 104 according to the fifth embodiment>

[0181] Figure 16 This diagram illustrates an example of output control performed by the output control unit 110 according to the fifth embodiment. Figure 16 In the middle, to and Figure 4 The same parts are marked with the same symbol and the explanation is omitted.

[0182] The number of metadata items corresponding to frame data F1 is 7 (M1 to M7), which exceeds the number of metadata items that can be attached to frame data F1 (4 items). In this case, with Figure 4 Similarly, in the example shown, the output control unit 110 will select four metadata M1 to M4 from the metadata M1 to M7 in order of high priority and attach them to the frame data F1, and output them from the external output I / F111 to the external recording device 120.

[0183] Furthermore, the output control unit 110 appends the metadata M5 to M7, which are not attached to the frame data F1, to the next frame data F2 of the frame data F1, and outputs it from the external output I / F111 to the external recording device 120.

[0184] Furthermore, the number of metadata corresponding to frame data F3 is 5, namely metadata M8 to M12, which exceeds the number of metadata that can be attached to frame data F3 (4). In this case, the output control unit 110 attaches 4 metadata M8 to M11 selected from metadata M8 to M12 in order of high priority to frame data F3, and outputs them from external output I / F111 to external recording device 120.

[0185] Furthermore, the output control unit 110 appends the metadata M12, which is not attached to the frame data F3, to the next frame data F4 of the frame data F3, and outputs it from the external output I / F111 to the external recording device 120.

[0186] Furthermore, the output control unit 110 appends the metadata M13, which corresponds to the frame data F5, to the frame data F5 and outputs it from the external output I / F111 to the external recording device 120.

[0187] Thus, the imaging device 100 according to the fifth embodiment can change the shooting conditions of the imaging unit 119 only on a multi-frame basis. As a result, the frequency of metadata generation is reduced, and metadata can be output even if the communication capacity, such as the blanking period of the frame data, is small. Furthermore, development processing can be performed considering the case where the shooting conditions do not change in a specific frame data. Therefore, the influence of the delay in metadata output on the result of development processing can be suppressed.

[0188] For example, in Figure 16 In the example shown, it can be seen that the metadata M5 to M7 attached to frame data F2, where the shooting conditions remain unchanged, represent the shooting conditions applicable to the previous frame data F1, and therefore are applicable to the development process of frame data F1. Furthermore, it can be seen that the metadata M12 attached to frame data F4, where the shooting conditions remain unchanged, represents the shooting conditions applicable to the previous frame data F3, and therefore are applicable to the development process of frame data F3.

[0189] (Sixth Embodiment)

[0190] Regarding the sixth embodiment, the parts that differ from the first to fifth embodiments will be described. The structure for outputting frame data (RAW moving image data) and metadata stored in the temporary storage unit 106 of the camera device 100 to an external recording device 120 outside the camera device 100 will be described, but the structure is not limited to this.

[0191] <The camera device 100 according to the sixth embodiment>

[0192] Figure 17 This diagram illustrates an example of the camera device 100 according to the sixth embodiment. Figure 17 In the middle, to and Figure 1 The same parts are marked with the same symbol and the explanation is omitted.

[0193] The camera device 100 according to the sixth embodiment replaces... Figure 1 The external output I / F111 shown includes an internal output I / F124, an internal input I / F125, a recording control unit 122, and a recording medium 123.

[0194] Internal output I / F124 is AND Figure 1 The external output I / F111 shown has the same HDMI or other interfaces, but it differs from the external output I / F111 in that it communicates with the internal input I / F125 inside the camera device 100. The internal input I / F125 is for communication with... Figure 1 The external recording device 120 shown has the same HDMI or other interface as the external input I / F121, but it differs from the external input I / F121 in that it is located inside the camera device 100.

[0195] Figure 17 The recording control unit 122 and recording medium 123 shown are for use with Figure 1 The recording control unit 122 and recording medium 123 shown have the same structure, but are located inside the camera device 100. That is, Figure 17 The camera device 100 shown has a built-in high-speed and high-capacity recording medium 123, and uses the internal HDMI or other interface of the camera device 100 to output frame data (RAW dynamic image data) and metadata to the recording medium 123.

[0196] In this structure, also with Figure 1 Similar to the camera device 100 shown, metadata representing the shooting conditions is generated in a manner that establishes a correspondence with the frame data when the shooting conditions of the camera unit 119 are changed, and the generated metadata is appended to the frame data and output as motion image data before de-mosaicing. As a result, the amount of metadata output can be reduced.

[0197] Therefore, metadata can be output with limited communication capacity, such as the blanking period of frame data. Furthermore, without increasing the communication capacity during the blanking period, issues such as frame rate fluctuations due to increased communication capacity during the blanking period or failure to meet connection compatibility specifications can be suppressed.

[0198] Furthermore, if the amount of metadata corresponding to the frame data exceeds the amount of data that can be appended to that frame data, the imaging device 100 appends metadata selected from the metadata corresponding to the frame data according to the priority of the metadata category to the frame data. This allows for the priority output of metadata of the category used for the development process. Therefore, even if not all metadata is output for each frame data, the impact on the development process result can be suppressed.

[0199] (Combinations of various implementation methods)

[0200] The above-described embodiments can also be implemented by combining them. For example, the same processing as in embodiments 2 to 5 can be implemented in the structure of the camera device 100 according to embodiment 6.

[0201] (Modified Example)

[0202] In the above embodiments, the process of appending metadata to each frame of data as close to the current time as possible has been described, but it is not limited to this process. For example, in Figure 4 In the example shown, the camera device 100 may also append only three metadata M1 to M3 to the frame data F1, and append metadata M4 to M7 to the frame data F2.

[0203] As stated above, at least the following items are described in this specification.

[0204] (1) A dynamic image camera device, comprising:

[0205] The camera unit generates image data for moving images in the form of multiple frames of data that are consecutive in time;

[0206] The metadata generation unit generates metadata representing the shooting conditions in a manner that establishes a correspondence with the frame data when the shooting conditions of the aforementioned camera unit are changed; and

[0207] The output unit appends the aforementioned metadata to the aforementioned frame data and outputs it as the motion image data before de-mosaicing.

[0208] If the amount of metadata corresponding to the frame data exceeds the amount of data that can be appended, the output unit will append metadata selected from the metadata corresponding to the frame data according to priority to the frame data.

[0209] (2) The dynamic image camera device according to (1), wherein,

[0210] The above priorities are based on the categories of the above metadata.

[0211] (3) The dynamic image camera device according to (1) or (2), wherein,

[0212] The aforementioned metadata generation unit generates metadata representing the shooting conditions of the camera unit at the start of the shooting process in a manner that establishes a correspondence with the initial frame data after the shooting begins by the aforementioned camera unit.

[0213] (4) The motion image capturing device according to any one of (1) to (3), wherein,

[0214] The output unit appends metadata that is not attached to the first frame data from the metadata that corresponds to the first frame data generated by the camera unit to the second frame data following the first frame data.

[0215] (5) The dynamic image camera device according to (4), wherein,

[0216] The output unit will preferentially append metadata that is not attached to the first frame of data to the second frame of data, which has a corresponding relationship with the second frame of data and metadata that is not attached to the first frame of data.

[0217] (6) The dynamic image capturing device according to (4) or (5), wherein,

[0218] If the shooting conditions of the camera unit, which are not attached to the metadata of the first frame data that correspond to the first frame data, are changed when the second frame data is captured, the output unit will attach the metadata of the changed shooting conditions from the metadata of the shooting conditions before and after the change to the second frame data.

[0219] (7) The motion image capturing device according to any one of (4) to (6), wherein,

[0220] When metadata that is not attached to the first frame data that corresponds to the first frame data is attached to the second frame data, the output unit will be able to determine that the data of the first frame data is attached to the second frame data.

[0221] (8) The motion image capturing device according to any one of (1) to (7), wherein,

[0222] The aforementioned camera unit can change the shooting conditions on a multi-frame basis.

[0223] (9) A method for capturing dynamic images, performed by a dynamic image capturing device having an image data unit that generates dynamic images in the form of multiple frames of data that are consecutive in time, the method comprising the following steps:

[0224] Metadata representing the aforementioned shooting conditions is generated in a manner that establishes a correspondence with the frame data when the shooting conditions of the aforementioned camera unit are changed; and

[0225] The aforementioned metadata is appended to the aforementioned frame data and output as dynamic image data before de-mosaicing. If the amount of the aforementioned metadata that corresponds to the aforementioned frame data exceeds the amount of data that can be appended, metadata selected from the aforementioned metadata that corresponds to the aforementioned frame data according to priority is appended to the aforementioned frame data.

[0226] (10) According to the dynamic image capturing method described in (9), wherein,

[0227] The above priorities are based on the categories of the above metadata.

[0228] (11) The dynamic image capturing method according to (9) or (10), wherein,

[0229] Metadata representing the shooting conditions of the camera unit at the start of the shooting is generated in a manner that establishes a correspondence with the initial frame data after the shooting begins by the aforementioned camera unit.

[0230] (12) The dynamic image capturing method according to any one of (9) to (11), wherein,

[0231] The metadata that is not attached to the first frame data and is related to the first frame data generated by the camera unit is attached to the second frame data after the first frame data.

[0232] (13) According to the dynamic image capturing method described in (12), wherein,

[0233] Metadata that corresponds to the data in the second frame and metadata that was not attached to the data in the first frame are preferentially attached to the data in the second frame.

[0234] (14) The dynamic image capturing method according to (12) or (13), wherein,

[0235] If the shooting conditions of the camera unit, which are not attached to the metadata of the first frame data, are changed when the second frame data is captured, the metadata representing the changed shooting conditions in the metadata representing the shooting conditions before and after the change is attached to the second frame data.

[0236] (15) The dynamic image capturing method according to any one of (12) to (14), wherein,

[0237] When metadata that is not attached to the first frame data that corresponds to the first frame data is attached to the second frame data, the data that can determine the first frame data will be attached to the second frame data.

[0238] (16) The dynamic image capturing method according to any one of (9) to (15), wherein,

[0239] The aforementioned camera unit can change the shooting conditions on a multi-frame basis.

[0240] (17) A motion picture camera program, which is a motion picture camera program for a motion picture camera device having a camera unit that generates motion picture data in the form of multiple frames of data that are consecutive in time, for causing the processor of the motion picture camera device to perform the following processing:

[0241] Metadata representing the aforementioned shooting conditions is generated in a manner that establishes a correspondence with the frame data when the shooting conditions of the aforementioned camera unit are changed; and

[0242] The aforementioned metadata is appended to the aforementioned frame data and output as dynamic image data before de-mosaicing. If the amount of the aforementioned metadata that corresponds to the aforementioned frame data exceeds the amount of data that can be appended, metadata selected from the aforementioned metadata that corresponds to the aforementioned frame data according to priority is appended to the aforementioned frame data.

[0243] Symbol Explanation

[0244] 100-Camera device, 101-Imaging lens system, 102-Imaging element, 103-ADC, 104-Camera control unit, 105-RAW correction unit, 106-Temporary storage unit, 107-De-mosaic processing unit, 108-Image correction unit, 109-Monitor, 110-Output control unit, 111-External output I / F, 119-Camera unit, 120-External recording device, 121-External input I / F, 122-Recording control unit, 123-Recording medium, 124-Internal output I / F, 125-Internal input I / F, F1~F5-Frame data, M1~M15-Metadata.

Claims

1. A dynamic image camera device, comprising: The camera unit generates image data for moving images in the form of multiple frames of data that are consecutive in time; The metadata generation unit generates metadata representing the shooting conditions in a manner that establishes a correspondence with the frame data when the shooting conditions of the camera unit are changed; and The output unit appends the metadata to the frame data and outputs it as motion image data before de-mosaicing. If the amount of metadata corresponding to the frame data exceeds the amount of data that can be appended, the output unit appends metadata selected from the metadata corresponding to the frame data according to priority to the frame data. The output unit appends metadata that is not attached to the first frame data from the metadata that corresponds to the first frame data generated by the camera unit to the second frame data following the first frame data.

2. The dynamic image camera device according to claim 1, wherein, The priority is based on the category of the metadata.

3. The dynamic image capturing device according to claim 1 or 2, wherein, The metadata generation unit generates metadata representing the shooting conditions of the camera unit at the start of the shooting in a manner that establishes a correspondence with the initial frame data after the shooting begins by the camera unit.

4. The dynamic image capturing device according to claim 1 or 2, wherein, The output unit will preferentially append metadata that is not attached to the first frame data to the second frame data, including metadata that has a corresponding relationship with the second frame data and metadata that is not attached to the first frame data.

5. The dynamic image camera device according to claim 1 or 2, wherein, If the shooting conditions of the camera unit, which are not represented by the metadata of the first frame data that correspond to the first frame data, are changed when the second frame data is captured, the output unit will append the metadata representing the changed shooting conditions from the metadata representing the shooting conditions before and after the change to the second frame data.

6. The dynamic image capturing device according to claim 1 or 2, wherein, When metadata that is not attached to the first frame data is attached to the second frame data from the metadata that corresponds to the first frame data, the output unit will be able to determine that the data of the first frame data is attached to the second frame data.

7. The dynamic image capturing device according to claim 1 or 2, wherein, The camera unit can change the shooting conditions on multiple frames.

8. A method for capturing dynamic images, performed by a dynamic image capturing apparatus equipped with an image capturing unit that generates dynamic images in the form of multiple frames of data that are sequential in time, the method comprising the following steps: Metadata representing the shooting conditions is generated in a manner that establishes a correspondence with the frame data when the shooting conditions of the camera unit are changed; and The metadata is appended to the frame data and output as the motion image data before de-mosaicing. If the amount of metadata corresponding to the frame data exceeds the amount of data that can be appended, metadata selected from the metadata corresponding to the frame data according to priority is appended to the frame data. The metadata that is not attached to the first frame data from the metadata that corresponds to the first frame data generated by the camera is appended to the second frame data after the first frame data.

9. The dynamic image capturing method according to claim 8, wherein, The priority is based on the category of the metadata.

10. The dynamic image capturing method according to claim 8 or 9, wherein, Metadata representing the shooting conditions of the camera unit at the start of the shooting is generated in a manner that establishes a correspondence with the initial frame data after the start of shooting by the camera unit.

11. The dynamic image capturing method according to claim 8 or 9, wherein, Metadata that corresponds to the data in the second frame and metadata that was not attached to the data in the first frame are preferentially attached to the data in the second frame.

12. The dynamic image capturing method according to claim 8 or 9, wherein, If the shooting conditions of the camera unit, which are not represented by the metadata of the first frame data that correspond to the first frame data, are changed when the second frame data is captured, the metadata representing the changed shooting conditions in the metadata representing the shooting conditions before and after the change is appended to the second frame data.

13. The dynamic image capturing method according to claim 8 or 9, wherein, When metadata that is not attached to the first frame data is attached to the second frame data from the metadata that corresponds to the first frame data, the data that can determine the first frame data will be attached to the second frame data.

14. The dynamic image capturing method according to claim 8 or 9, wherein, The camera unit can change the shooting conditions on multiple frames.

15. A computer-readable recording medium storing a moving image recording program, the moving image recording program being a moving image recording program for a moving image recording device, the moving image recording device comprising a camera unit that generates image data in the form of a plurality of frames of data that are sequentially continuous in time, the moving image recording program being configured to cause the processor of the moving image recording device to perform the following processing: Metadata representing the shooting conditions is generated in a manner that establishes a correspondence with the frame data when the shooting conditions of the camera unit are changed; and The metadata is appended to the frame data and output as the motion image data before de-mosaicing. If the amount of metadata corresponding to the frame data exceeds the amount of data that can be appended, metadata selected from the metadata corresponding to the frame data according to priority is appended to the frame data. The metadata that is not attached to the first frame data from the metadata that corresponds to the first frame data generated by the camera is appended to the second frame data after the first frame data.

Images