Imaging device and imaging method

Abstract

Provided is an imaging device, wherein the device is configured so that processing is carried out to generate, as metadata associated with a captured image, correction information that is based on: a first shake correction value related to a first shake correction function for correcting the positional relationship between an optical image incident via a lens and an output captured image; and a second shake correction value related to a second shake correction function provided to a lens barrel provided with a lens.

Description

[Technical Field]

[0001] The present technology relates to an imaging device and an imaging method, and in particular to processing of shake correction data. [Background technology]

[0002] 2. Description of the Related Art For example, an imaging device such as an interchangeable lens camera is known, which is made up of a camera body and a lens barrel and is capable of recording moving images on a recording medium. Some imaging devices use a vibration reduction mechanism built into the camera body that mechanically adjusts the vibration, while others use a vibration reduction mechanism built into the lens barrel that mechanically adjusts the vibration. Electronic vibration reduction mechanisms are also known, which change the readout range of the image signal from the imaging element in response to vibration, or change the cropping range of the image during image signal processing.

[0003] The following Patent Document 1 discloses a configuration in which shake correction is performed on both the lens barrel side and the camera body side. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] WO2018 / 025639 publication Summary of the Invention [Problem to be solved by the invention]

[0005] Nowadays, users can easily capture a variety of images and adjust the images using mobile devices such as smartphones and tablets, or cameras themselves, personal computers, etc. In these cases, it may be necessary to remove the effects of camera shake during image capture with high precision, or conversely, to actively add shake to the image to achieve a dramatic effect. Therefore, the present disclosure proposes a technique for saving appropriate information in an imaging device, assuming that image shake will be added or removed from a captured video. [Means for solving the problem]

[0006] The imaging device according to the present technology includes a control unit that generates, as metadata associated with a captured image, correction information based on both a first blur correction value associated with a first blur correction function that corrects the positional relationship between an optical image incident through a lens and an output captured image, and a second blur correction value associated with a second blur correction function provided in a lens barrel that includes the lens. The first blur correction function is a function on the camera body side that corrects the positional relationship between the optical image and the output captured image (the image that is ultimately output from the imaging device). The second blur correction function is a function provided on the lens barrel side. In this way, when the lens barrel side and the camera body side each have a mechanical or electronic blur correction function, correction information based on the blur correction values ​​on the lens barrel side and the camera body side is associated with the video being captured as metadata. For example, the metadata is recorded on a recording medium in a state associated with the image data.

[0007] In the imaging device according to the present technology described above, it is conceivable that the control unit generates the correction information with a data amount that is smaller than the data amount obtained by adding together the data amounts of the first shake correction value and the second shake correction value. If both the first and second shake compensation values ​​were used as compensation information, the amount of data recorded as metadata would increase. Therefore, we have devised a method to record information containing the first and second shake compensation values ​​with a smaller amount of data.

[0008] In the imaging device according to the present technology described above, it is conceivable that the control unit sets, as the correction information, one of the first shake correction value and the second shake correction value, and conversion information for obtaining the other shake correction value using the one shake correction value. By recording one of the first and second shake compensation values ​​and conversion information corresponding to, for example, the ratio, difference, etc. between the first and second shake compensation values, the first and second shake compensation values ​​can be determined from the metadata.

[0009] In the imaging device according to the present technology described above, it is conceivable that the control unit sets, as the correction information, the first shake correction value and conversion information for obtaining the second shake correction value using the first shake correction value. By recording the first shake correction value and the conversion information, the first shake correction value and the second shake correction value can be obtained from the metadata.

[0010] In the imaging device according to the present technology described above, it is conceivable that the control unit generates, at each predetermined timing, the correction information that is a set of one of the first shake correction value and the second shake correction value and conversion information for obtaining the other shake correction value using the one shake correction value. For example, when metadata is recorded corresponding to an image frame, a set of one shake correction value and conversion information is recorded corresponding to each frame.

[0011] In the imaging device according to the present technology described above, it is conceivable that the control unit generates the correction information including one of the first shake correction value and the second shake correction value at each timing, and conversion information for determining the other shake correction value using the one shake correction value at a timing when a ratio between the first shake correction value and the second shake correction value changes. In other words, the conversion information is recorded only when the ratio between the first motion compensation value and the second motion compensation value changes.

[0012] In the imaging device according to the present technology described above, it is conceivable that the control unit causes the ratio between the first shake compensation value and the second shake compensation value to be a fixed ratio from the start to the end of video recording, and generates the correction information including one of the first shake compensation value and the second shake compensation value at each timing, and conversion information according to the fixed ratio for using the one shake compensation value to obtain the other shake compensation value. In other words, it is sufficient to record only one piece of conversion information corresponding to the video.

[0013] In the imaging device according to the present technology described above, it is conceivable that the control unit sets a composite correction value of the first shake correction value and the second shake correction value as the correction information. The correction value for the entire image capture device is recorded as metadata as a composite correction value of the first blur correction value and the second blur correction value.

[0014] In the imaging device according to the present technology described above, it is conceivable that the control unit performs processing to record the first shake correction value and the second shake correction value as the correction information, and to record communication speed information between the lens barrel and the camera body as metadata on a recording medium. Both the first and second image stabilization values ​​are recorded as metadata as compensation information, and communication speed information for the lens barrel and the camera body is also recorded as metadata.

[0015] In the imaging device according to the present technology described above, the communication speed information may be a value indicating the communication speed. The communication speed between the lens barrel and the camera body is used as metadata.

[0016] In the imaging device according to the present technology described above, the communication speed information may be information resulting from a comparison of the communication speed with a predetermined value. For example, flag information indicating whether the communication speed between the lens barrel and the camera body is fast or slow is used as metadata.

[0017] In the imaging device according to the present technology described above, the shake correction value is considered to be a correction execution value indicating a position or a displacement amount of a position due to correction by the shake correction function. The first and second blur correction values ​​are positions where blur correction by the first and second blur correction functions is actually performed, or the amount of displacement of the positions.

[0018] In the imaging device according to the present technology described above, the shake correction value is considered to be a correction instruction value that indicates a position or a displacement amount of a position for correcting the shake correction function. The position or the amount of displacement of the position for the first shake correction function or the second shake correction function is specified to execute the displacement for the shake correction. This specified correction value is used as the shake correction value to be reflected in the metadata.

[0019] In the imaging device according to the present technology described above, it is conceivable that the control unit sets information detected by the shake detection unit as metadata. For example, if a sensor such as a gyro is provided, information about the vibrations detected by the sensor is included in the metadata.

[0020] In an imaging method according to the present technology, an imaging device performs a process of generating correction information based on both a first blur correction value associated with a first blur correction function that corrects the positional relationship between an optical image incident through a lens and an output captured image, and a second blur correction value associated with a second blur correction function provided in a lens barrel that includes the lens, as metadata to be associated with the captured image. This allows the image blur correction values ​​on both the lens barrel side and the body side to be referenced from the metadata when capturing an image. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is an explanatory diagram of an imaging device and an image processing device according to an embodiment of the present technology; [Figure 2] FIG. 2 is an explanatory diagram of a data flow between the imaging device and the image processing device according to the embodiment. [Figure 3] 1 is a block diagram of a configuration example of an imaging apparatus according to an embodiment; [Figure 4] 10A and 10B are explanatory diagrams of correction processing of the imaging device according to the embodiment. [Figure 5] FIG. 2 is an explanatory diagram of metadata according to an embodiment. [Figure 6] FIG. 2 is an explanatory diagram of functions of the image processing apparatus according to the embodiment; [Figure 7] 4 is a flowchart of an example of processing related to shake correction according to the first embodiment. [Figure 8] 1 is a flowchart of recording IMU data according to an embodiment. [Figure 9] 10 is a flowchart of an example of processing related to shake correction according to the second embodiment. [Figure 10] 10 is a flowchart of an example of processing related to shake correction according to the third embodiment. [Figure 11] 13 is a flowchart of an example of processing related to shake correction according to the fourth embodiment. [Figure 12] 13 is a flowchart of an example of processing related to shake correction according to the fifth embodiment. [Figure 13] 13 is a flowchart of an example of a process for recording metadata on the lens side according to the fifth embodiment. [Figure 14] 13 is a flowchart of recording communication speed information according to the fifth embodiment. [Figure 15] 13 is a flowchart of recording communication speed information according to the sixth embodiment. [Figure 16] 13 is a flowchart of processing on the lens barrel side in the seventh embodiment. [Figure 17] 13 is a flowchart of processing on the camera body side according to the seventh embodiment. [Figure 18] 13 is a flowchart of processing on the lens barrel side in another example of the seventh embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0022] The embodiments will be described below in the following order. <1. Shake modification using imaging device and image processing device> 2. Imaging device configuration and metadata <3. Functions of image processing device> 4. First Embodiment 5. Second Embodiment 6. Third Embodiment 7. Fourth Embodiment 8. Fifth Embodiment 9. Sixth Embodiment 10. Seventh Embodiment <11. Summary and Variations>

[0023] <1. Shake modification using imaging device and image processing device> FIG. 1 shows an example of an image capture device 1 according to the embodiment and an image processing device (5, 6) that acquires an image file MF captured by the image capture device 1. The figure shows an example in which a mobile terminal 7 and a personal computer 8 function as the image processing devices 5 and 6. Although not shown, various other devices such as a dedicated image editing device, a cloud server, a television device, and a video recording / playback device are also envisioned as the image processing devices 5 and 6. These devices can function as either the image processing devices 5 and 6.

[0024] The image processing device 5 is a device that performs a primary shake modification process on the image data acquired from the imaging device 1. On the other hand, the image processing device 6 is a device that performs secondarily swing change processing on image data that has already been subjected to swing change processing by another image processing device.

[0025] The term "shake" refers to the interframe shake between images that make up a video. It broadly refers to vibration components (image fluctuations between frames) that occur between frames, such as image shake caused by camera shake in images captured by the imaging device 1, or shake intentionally added by image processing. The term "camera shake" is also used to refer to "shake" caused by camera shake or the like when capturing an image with the imaging device 1. Correction performed within the imaging device 1 to reduce image shake caused by camera shake or the like (including vibrations applied when the imaging device 1 is not held in the hand but is fixed in place) is called "camera shake correction," and is distinguished from the "shake modification" processing in the image processing devices 5 and 6.

[0026] "Interframe shake modification" refers to changing the state of shake in an image, such as reducing the shake occurring in the image or adding shake to the image. This "shake modification" includes the following "interframe shake reduction" and "interframe shake production."

[0027] "Shake removal" refers to the process of eliminating (removing all shake) or reducing (removing part of shake) shake that occurs in an image due to camera shake or the like during image capture by the image processing devices 5 and 6.

[0028] "Shake effect" refers to changing the state of shaking of an image by the image processing devices 5 and 6. This shake effect may involve reducing the shake, and in that sense may result in the same as "shake removal," but in this embodiment, it refers to changing the state of shaking of an image in response to an instruction given by the user's operation or automatic control as to the amount of change in shake. For example, reducing or increasing the shaking that occurs during image capture in response to a user instruction or adding new shaking corresponds to "shake effect." One example of the purpose of the shaking effect is to intentionally shake an image to add impact to a video scene.

[0029] The imaging device 1 in FIG. 1 is a so-called digital still camera or digital video camera, and is capable of at least capturing moving images.

[0030] The camera main body of the imaging device 1 is shown as a camera body 2. The lens barrel 3 functions as a so-called interchangeable lens, and is detachable from the camera main body (camera body 2) of the imaging device 1. The user can change the lens barrel 3 depending on the use case. Although the embodiment assumes such an interchangeable lens type imaging device 1, the technology of the present disclosure can also be applied to a type in which the lens barrel 3 cannot be removed from the camera body 2.

[0031] The imaging device 1 can capture moving images and transfer the image file MF obtained by capturing the moving image to a mobile terminal 7 or a personal computer 8 as an image processing device 5 via wired or wireless communication. Alternatively, the imaging device 1 may record the image file MF on a recording medium such as a memory card, and the mobile terminal 7 or the personal computer 8 may read the image file MF from the memory card. The image file MF includes not only image data as a moving image, but also metadata, which is additional information associated with the image data.

[0032] FIG. 2 shows how information is transmitted between the imaging device 1, the image processing device 5, and the image processing device 6. Image data VD1 and metadata MTD1 are transmitted from the imaging device 1 to the image processing device 5 via wired communication, wireless communication, or a recording medium. The image data VD1 and the metadata MTD1 are information that is transmitted as, for example, an image file MF. In this embodiment, the metadata MTD1 includes, for example, information relating to camera shake correction during image capture.

[0033] The image processing device 5 can receive the image data VD1 and the metadata MTD1 and perform various processes. For example, the image processing device 5 can perform shake modification processing on the image data VD1 using information related to camera shake correction included in the metadata MTD1. As described above, shake modification is a process that cancels camera shake correction and returns the image to the original shaky image, performs more advanced shake removal, or adds shake to the image for dramatic effect.

[0034] The image processing device 5 can further transfer the image data VD2 that has undergone the swing change process and the like, and the metadata MTD2 to another image processing device 6. In this case, by adding information about the swing change process and the like as the metadata MTD2, the image processing device 6 can also perform a variety of swing changes.

[0035] In this embodiment, assuming such information transmission, it is possible to appropriately change the shaking at least in the image processing device 5. For this purpose, the following description will focus on metadata recording at the time of image capture in the imaging device 1.

[0036] 2. Imaging device configuration and metadata FIG. 3 shows an example of the configuration of the imaging device 1 and the lens barrel 3.

[0037] A lens system 10 having a plurality of optical components is formed in the lens barrel 3. For example, the lens system 10 includes a zoom lens 10a, an aperture mechanism 10b, an image stabilization lens mechanism 10c, a focus lens 10d, and the like. The image stabilization lens mechanism 10c is a mechanism that reduces shaking that occurs in an image by mechanically driving a lens in response to camera shake.

[0038] Light from the subject (incident light) is collected on the image sensor unit 12 via the lens system 10 and the shutter 11 in the camera body 2 .

[0039] The imaging element unit 12 includes an image sensor (imaging element) such as a CMOS (Complementary Metal Oxide Semiconductor) type or a CCD (Charge Coupled Device) type. The image sensor unit 12 performs processes such as CDS (Correlated Double Sampling) and AGC (Automatic Gain Control) on the electrical signals obtained by photoelectrically converting the light received by the image sensor, and then performs A / D (Analog / Digital) conversion on the electrical signals, and outputs the resulting digital image signals to the downstream camera signal processor 13.

[0040] An image pickup surface image stabilization unit 30 is provided for the image pickup element section 12. This image pickup surface image stabilization unit 30 is a mechanism that corrects image shaking by mechanically moving the image sensor in response to camera shake or the like.

[0041] The camera signal processing unit 13 is configured as an image processor, for example, using a DSP (Digital Signal Processor), etc. The camera signal processing unit 13 performs various types of signal processing on the digital signal (captured image signal) from the imaging element unit 12. For example, as camera processes, the camera signal processing unit 13 performs preprocessing, synchronization processing, YC generation processing, various correction processing, resolution conversion processing, codec processing, etc.

[0042] In the pre-processing, the captured image signal from the image sensor unit 12 is subjected to clamping processing for clamping the R, G, and B black levels to a predetermined level, correction processing between the R, G, and B color channels, and the like. In the synchronization process, a color separation process is performed so that the image data for each pixel contains all the color components R, G, and B. For example, in the case of an image sensor that uses a Bayer color filter, a demosaic process is performed as the color separation process. In the YC generation process, a luminance (Y) signal and a color (C) signal are generated (separated) from R, G, and B image data. In the resolution conversion process, the image data that has been subjected to various signal processes is subjected to the resolution conversion process.

[0043] 4 shows examples of various correction processes performed by lens system 10 through camera signal processor 13. In FIG. 4, optical image stabilization performed by image stabilization lens mechanism 10c and image plane image stabilization unit 30, as well as correction processes performed by camera signal processor 13, are illustrated in the order of execution.

[0044] As the optical image stabilization in process F1, lens stabilization by image stabilization lens mechanism 10c and body stabilization by image pickup surface image stabilization unit 30 are performed. For example, image stabilization is performed as lens vibration reduction by shifting the image stabilization lens mechanism 10c in the yaw and pitch directions, or as body vibration reduction by shifting the image sensor in the yaw and pitch directions using the imaging surface image stabilization unit 30, so that the image of the subject is formed on the image sensor with the effects of camera shake physically canceled out. This lens stabilization and body stabilization may be performed either alone or both. In addition to the optical image stabilization described above, electrical image stabilization may also be performed.

[0045] In the camera signal processing unit 13, the processes F2 to F6 are performed by spatial coordinate transformation for each pixel. In process F2, lens distortion correction is performed. In process F3, focal plane distortion correction is performed as one element of electronic image stabilization. This corrects distortion that occurs when, for example, a CMOS image sensor reads images using the rolling shutter method.

[0046] In process F4, roll correction is performed, that is, correction of the roll component is performed as one element of electronic image stabilization. In process F5, keystone distortion caused by electronic image stabilization is corrected. Keystone distortion caused by electronic image stabilization is perspective distortion caused by cutting out a portion of the image away from the center. In process F6, shifting in the pitch direction and yaw direction and cropping are performed as one element of electronic image stabilization. For example, camera shake correction, lens distortion correction, and trapezoidal distortion correction are performed in the above procedure. It is not necessary to perform all of the processes listed here, and the order of the processes may be changed as appropriate.

[0047] 3, the image data that has been subjected to the various processes described above is subjected to coding processing and file generation for recording or communication, for example. For example, an image file MF is generated in the MP4 format used for recording video and audio in accordance with MPEG-4. Still image files may also be generated in formats such as JPEG (Joint Photographic Experts Group), TIFF (Tagged Image File Format), and GIF (Graphics Interchange Format).

[0048] Although the audio processing system is not shown in FIG. 3, in reality, an audio recording system and an audio processing system are included, and the image file MF may contain audio data as well as image data as a moving image.

[0049] The camera control unit 18 is configured by a microcomputer (arithmetic processing device) equipped with a CPU (Central Processing Unit). The memory unit 19 stores information and the like used for processing by the camera control unit 18. The illustrated memory unit 19 comprehensively includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, and the like.

[0050] The RAM in the memory unit 19 is used as a work area for the CPU of the camera control unit 18 to process various data, and is used to temporarily store data, programs, and the like. The ROM and flash memory (non-volatile memory) in memory unit 19 are used to store the OS (Operating System) that the CPU uses to control each part, content files such as image files, application programs for various operations, firmware, etc. The memory unit 19 may be a memory area built into the microcomputer chip that serves as the camera control unit 18, or may be configured as a separate memory chip.

[0051] The camera control unit 18 controls the entire imaging device 1 and the lens barrel 3 by executing a program stored in the ROM or flash memory of the memory unit 19 . For example, the camera control unit 18 controls the operation of each necessary unit, such as controlling the shutter speed of the image sensor unit 12, issuing instructions for various signal processing in the camera signal processing unit 13, imaging and recording operations in response to user operations, playback of recorded image files, operations of the lens system 10 such as zoom, focus, and aperture adjustment in the lens barrel 3, and user interface operations.

[0052] The camera control unit 18 also performs various processes and output controls on the image data processed by the camera signal processing unit 13 . The camera control unit 18 can cause the electronic image stabilization control unit 35 to perform electronic image stabilization processing on the image data. The camera control unit 18 also controls the blur correction metadata processing unit 36 ​​to generate metadata related to blur correction. The camera control unit 18 also generates metadata made up of various information including information related to blur correction, and controls the recording of the metadata as information related to the image file MF. The camera control unit 18 also communicates with the lens control unit 20 on the lens barrel 3 side via the communication control unit 33 .

[0053] 3, the electronic image stabilization control unit 35 and the image stabilization metadata processing unit 36 ​​are shown as separate blocks from the camera control unit 18, but these can be considered to be functions realized by the microcomputer that constitutes the camera control unit 18. For the sake of explanation, the camera control unit 18, the electronic image stabilization control unit 35, and the image stabilization metadata processing unit 36 ​​are collectively referred to as the "control unit 40." However, these may also be configured as separate arithmetic processing units.

[0054] The recording control unit 14 performs recording and reproduction on a recording medium such as a nonvolatile memory, etc. The recording control unit 14 performs processing to record image files MF such as moving image data and still image data, thumbnail images, etc. on the recording medium, for example. There are various possible actual forms for the recording control unit 14. For example, the recording control unit 14 may be configured as a flash memory and its write / read circuit built into the imaging device 1, or may be in the form of a card recording / playback unit that performs recording / playback access to a recording medium that can be attached to or detached from the imaging device 1, such as a memory card (portable flash memory, etc.). The recording control unit 14 may also be realized as an HDD (Hard Disk Drive) built into the imaging device 1.

[0055] The display unit 15 is a display unit that displays various information to the user, and is, for example, a display panel or viewfinder using a display device such as a liquid crystal panel (LCD: Liquid Crystal Display) or an organic EL (Electro-Luminescence) display that is arranged on the housing of the imaging device 1. The display unit 15 executes various displays on the display screen based on instructions from the camera control unit 18 . For example, the display unit 15 displays a reproduced image of image data read from a recording medium by the recording control unit 14. Furthermore, image data of the captured image that has been resolution-converted for display by the camera signal processing unit 13 is supplied to the display unit 15, and the display unit 15 may display based on the image data of the captured image in response to an instruction from the camera control unit 18. This causes a so-called through image (monitoring image of the subject), which is the captured image during composition confirmation, to be displayed. Furthermore, based on instructions from the camera control unit 18, the display unit 15 displays various operation menus, icons, messages, etc., that is, GUI (Graphical User Interface), on the screen.

[0056] The output unit 16 performs data communication and network communication with external devices via wire or wirelessly. For example, the captured image data (still image files and moving image files) is transmitted and output to an external display device, recording device, playback device, or the like. The output unit 16 may also be a network communication unit that communicates via various networks such as the Internet, a home network, or a LAN (Local Area Network), and transmits and receives various data to and from servers, terminals, etc. on the network.

[0057] The operation unit 17 collectively refers to input devices that allow the user to input various operations. Specifically, the operation unit 17 refers to various operators (keys, dials, touch panel, touch pad, etc.) provided on the housing of the imaging device 1. The operation unit 17 detects the user's operation, and a signal corresponding to the input operation is sent to the camera control unit 18 .

[0058] The shutter driving unit 31 drives the shutter 11 based on instructions from the camera control unit 18 .

[0059] The correction unit driving section 32 drives the image pickup surface image stabilization unit 30 based on instructions from the camera control section 18, and displaces the image sensor in the image pickup element section 12 for optical image stabilization.

[0060] The shake detection unit 34 indicates a sensor that detects shaking applied to the camera body 2. The shake detection unit 34 is equipped with, for example, an IMU (inertial measurement unit), and can detect angular velocity with a three-axis angular velocity (gyro) sensor of pitch, yaw, and roll, and can detect acceleration with an acceleration sensor. It should be noted that the shake detection unit 34 need not include both a gyro sensor and an acceleration sensor, as long as it includes a sensor that can detect camera shake during image capture.

[0061] The lens barrel 3 is equipped with a lens control unit 20, which is implemented by, for example, a microcomputer. When the lens barrel 3 is attached to the camera body 2, the camera control unit 18 and the lens control unit 20 can communicate with each other via the communication control units 27 and 33. Communication control units 27 and 33 are connected by wire and communicate with each other when lens barrel 3 is attached to camera body 2. However, they may be configured to communicate wirelessly.

[0062] The lens control unit 20 and the camera control unit 18 constantly exchange data in two directions at a certain communication speed. For example, camera control unit 18 issues drive instructions to lens control unit 20 for zoom lens 10a, focus lens 10d, aperture mechanism 10b, and image stabilization lens mechanism 10c. Lens control unit 20 causes lens system 10 to operate in response to these drive instructions. Furthermore, lens control unit 20 transmits to camera control unit 18 lens distortion correction information, focal length information, the position of the correction lens of image stabilization lens mechanism 10c, and the like.

[0063] The lens barrel 3 is provided with, for example, a zoom drive unit 21 having a motor and a motor driver for driving the zoom lens 10a, an aperture drive unit 22 having a motor and a motor driver for driving the aperture mechanism 10b, a correction lens drive unit 23 having a motor and a motor driver for driving the image stabilization lens mechanism 10c, and a focus drive unit 24 having a motor and a motor driver for driving the focus lens 10d. The zoom driver 21, aperture driver 22, correction lens driver 23, and focus driver 24 apply drive currents to the corresponding motors in response to instructions from the lens controller 20, which are based on instructions from the camera controller 18. This allows the zoom operation, aperture opening / closing operation, optical image stabilization operation, and focus operation to be performed.

[0064] The memory unit 26 stores information and the like used for processing by the lens control unit 20. The memory unit 26 comprehensively refers to, for example, ROM, RAM, flash memory, etc. The memory unit 26 may be used to temporarily store information that the lens control unit 20 transmits to the camera control unit 18.

[0065] Shake detection unit 25 represents a sensor that detects shaking inside lens barrel 3, and is assumed to be equipped with an IMU, for example, similar to shake detection unit 34 on the camera body 2 side. Note that it is also assumed that shake detection unit 34 is not equipped inside lens barrel 3.

[0066] Next, the contents of the image file MF and the contents of the metadata transmitted from the image capture device 1 to the image processing device 5 will be described. 5A shows the data contained in an image file MF. As shown in the figure, an image file MF contains various types of data such as a "header," "sound," "movie," and "metadata."

[0067] The "header" describes information such as the file name, file size, and information indicating whether or not metadata is included. "Sound" is the audio data recorded with the video. For example, two-channel stereo audio data is stored. A "movie" is video data, and is made up of image data for each frame (#1, #2, #3, etc.) that makes up the video. As "metadata," additional information associated with each frame (#1, #2, #3, etc.) that makes up the video is written.

[0068] An example of the metadata content is shown in Figure 5B. For example, for one frame, IMU data, coordinate transformation parameters HP, timing information TM, and camera parameters CP are described. Note that these are only part of the metadata content, and other information may also be included. Also, some of the information shown may not be included. Also, in FIG. 5B, the broken line indicates a case where communication speed information is included in the metadata, and this is the case where the processing of the fifth and sixth embodiments described later is performed.

[0069] IMU data includes gyro (angular velocity data), accelerator (acceleration data), and sampling rate. The IMUs installed in the image capture device 1 as the shake detection units 34 and 25 output angular velocity data and acceleration data at a predetermined sampling rate. Generally, this sampling rate is higher than the frame rate of the captured image, so that many IMU data samples can be obtained in one frame period.

[0070] Therefore, as angular velocity data, n samples are associated with one frame, such as gyro sample #1, gyro sample #2, . . . , gyro sample #n, as shown in FIG. 5C. As for acceleration data, m samples are associated with one frame, such as accelerator sample #1, accelerator sample #2, . . . accelerator sample #m. Sometimes n=m, and sometimes n≠m. Although the metadata is associated with each frame in the example described here, there are cases where the IMU data is not perfectly synchronized with the frames. In such cases, for example, time information related to the time information of each frame is stored as an IMU sample timing offset in the timing information TM.

[0071] The coordinate transformation parameters HP are a general term for parameters used for correction involving coordinate transformation of each pixel in an image, including nonlinear coordinate transformation such as lens distortion. The coordinate transformation parameters HP are a term that can include at least lens distortion correction parameters, trapezoidal distortion correction parameters, focal plane distortion correction parameters, electronic image stabilization parameters, and optical image stabilization parameters.

[0072] The lens distortion correction parameters are information for directly or indirectly determining how distortions such as barrel aberration and pincushion aberration have been corrected, and for restoring the image to the state before lens distortion correction.

[0073] The trapezoidal distortion correction parameter is the amount of correction used to correct trapezoidal distortion that occurs when the cutout region is shifted from the center by electronic camera shake correction, and is a value that corresponds to the amount of correction by electronic camera shake correction.

[0074] The focal plane distortion correction parameter is a value indicating the amount of correction for the focal plane distortion for each line.

[0075] For electronic image stabilization and optical image stabilization, the parameter indicates the amount of correction in each of the yaw, pitch, and roll axial directions.

[0076] 3, the optical image stabilization mechanism in this embodiment is provided with image stabilization lens mechanism 10c and image plane image stabilization unit 30. Therefore, as the correction information indicating the amount of correction for optical image stabilization, for example, a body-side image stabilization value and a lens-side image stabilization value are recorded, as shown in FIG. The body side shake correction value is the shake correction value in the image pickup surface image stabilization unit 30. The lens-side shake correction value is the shake correction value in the camera shake correction lens mechanism 10c.

[0077] These shake correction values ​​are assumed to be the actual correction values ​​achieved by image plane image stabilization unit 30 and image stabilization lens mechanism 10c. These correction effective values ​​are values ​​that represent the actual displacement achieved as optical image stabilization, such as position information that changes due to actual correction, detected by a position sensor provided in image plane image stabilization unit 30 and image stabilization lens mechanism 10c, and the amount of displacement of position information from the previous frame.

[0078] Alternatively, these shake correction values ​​may be correction instruction values ​​output by camera control unit 18 to correction unit drive unit 32, or correction instruction values ​​sent by camera control unit 18 to correction lens drive unit 23 via lens control unit 20. This is because image plane image stabilization unit 30 and image stabilization lens mechanism 10c are driven to positions and displacement amounts of positions according to these correction instruction values.

[0079] Furthermore, as the correction information for optical image stabilization, there is also an example in which the body side image stabilization value and conversion information are recorded, as shown in Fig. 5E. This will be described in the first to third embodiments. 5F, the correction information for optical image stabilization may also record a combined correction value that is the amount of shake correction applied overall by imaging device 1 by combining both the body-side shake compensation value and the lens-side shake compensation value. This will be described in the fourth embodiment.

[0080] The parameters for lens distortion correction, trapezoidal distortion correction, focal plane distortion correction, and electronic image stabilization are collectively referred to as coordinate transformation parameters because these correction processes are performed on the image formed on each pixel of the image sensor of the imaging element unit 12, and are parameters for correction processes that involve coordinate transformation of each pixel. For the sake of explanation, the correction information for optical image stabilization is also considered to be one of the coordinate transformation parameters. This is because, in optical image stabilization, correcting the shaking between frames is a process that involves coordinate transformation of each pixel. In other words, by performing inverse correction using these parameters, image data that has been subjected to lens distortion correction, trapezoidal distortion correction, focal plane distortion correction, electronic image stabilization, and optical image stabilization can be returned to the state before each correction process, i.e., the state when the image was formed on the image sensor of the image sensor unit 12.

[0081] Furthermore, the lens distortion correction, trapezoidal distortion correction, and focal plane distortion correction parameters are distortion correction processes for images captured when the optical image from the subject is itself optically distorted, and since each parameter is intended to correct optical distortion, they are collectively referred to as optical distortion correction parameters. In other words, by performing inverse correction using these parameters, image data that has been subjected to lens distortion correction, trapezoidal distortion correction, and focal plane distortion correction can be returned to the state before optical distortion correction.

[0082] The timing information TM in the metadata includes information on the exposure time (shutter speed), exposure start timing, readout time (curtain speed), number of exposure frames (long-second exposure information), IMU sample offset, and frame rate. These are mainly used to match the lines of each frame with IMU data. However, even if the image sensor 12a is a CCD or a global shutter type CMOS, if the exposure center of gravity is shifted using an electronic shutter or a mechanical shutter, correction to match the exposure center of gravity can be made using the exposure start timing and curtain speed.

[0083] As the camera parameters CP in the metadata, the angle of view (focal length), zoom position, and lens distortion information are described.

[0084] <3. Functions of image processing device> The image processing device 5 can perform a swing change process on the image file MF generated by the imaging device 1 through imaging. The functional configuration of the image processing device 5 for this purpose is shown in Fig. 6. It is assumed that the image processing device 6 also has a similar configuration.

[0085] The image processing device 5 may be an information processing device such as the mobile terminal 7 or personal computer 8 in FIG. 1, in which case the application program provides functions such as those shown in FIG. For example, the image processing device 5 has functions as a correction cancellation unit 51, a shaking effect production unit 52, a processing setting unit 53, and a user interface unit 54. Note that "user interface" is also written as "UI," and the user interface unit 54 will hereinafter be written as "UI unit 54."

[0086] The correction canceling unit 51 and the shaking effect producing unit 52 are functions that apply some shaking change to the image.

[0087] The correction cancellation unit 51 is a function that cancels the optical image stabilization or electronic image stabilization applied by the imaging device 1 and performs shake modification so as to return the image to a state in which no image stabilization has been applied, i.e., a state in which the image is originally shaken by camera shake. Alternatively, the correction cancellation unit 51 can perform processing to cancel only the correction by the image stabilization lens mechanism 10c, processing to cancel only the correction by the image plane image stabilization unit 30, processing to cancel only the electronic image stabilization, etc.

[0088] The vibration production unit 52 is a function that performs processing to change the vibration state of image data in accordance with parameters and instructions input by the user. This shaking effect unit 52 can perform shaking effect processing such as adding or removing shaking to image data VD1 that has not been subjected to cancellation processing by the correction cancellation unit 51, or to image data VD1 that has been subjected to cancellation processing by the correction cancellation unit 51. The shaking effect processing may involve reducing shaking in the image, removing shaking with higher accuracy than the image stabilization of the imaging device 1, or adding shaking to the image.

[0089] The UI unit 54 has a function of presenting operators relating to correction cancellation and fluctuation change to the user and acquiring operation information from the operators.

[0090] The processing setting unit 53 sets processing parameters for correction cancellation based on the metadata MTD1, and executes processing by the correction cancellation unit 51. The processing setting unit 53 also sets processing parameters for shaking change in accordance with the user operation detected by the UI unit 54 and the metadata MTD1, and executes processing by the shaking production unit 52.

[0091] By using such an image processing device 5, the user can make desired changes to the shaking of the image file MF obtained by the imaging device 1. In particular, since the metadata MTD1 includes information about optical image stabilization, it is possible to recognize the correction by the image stabilization lens mechanism 10c and the correction by the image plane image stabilization unit 30 and perform processing accordingly. For example, it is possible to cancel image stabilization or add an appropriate amount of shake.

[0092] 4. First Embodiment Various examples of processing according to the embodiment will be described below, focusing in particular on optical image stabilization and the corresponding metadata recording.

[0093] For example, if the imaging device 1 uses the body side shake compensation value and the lens side shake compensation value as metadata and associates them with the image data as time-series information as shown in FIG. 5D, the image processing device 5 can perform shake modification (correction cancellation or shake effect) using the body side shake compensation value and the lens side shake compensation value.

[0094] However, if the body and lens stabilization values ​​are recorded as metadata for each frame, as shown in Figure 5D, the amount of data in the metadata MTD1 increases, which may not be desirable in some cases. For example, if the body and lens stabilization values ​​are recorded as metadata for each frame of a relatively long video, the amount of data in the image file MF increases significantly. Therefore, in the first embodiment, the amount of data in the metadata MTD1 can be reduced while still allowing the image processing device 5 to detect the body side shake compensation value and the lens side shake compensation value using the metadata MTD1.

[0095] Fig. 7 shows an example of processing by the control unit 40 (camera control unit 18, etc.) in the camera body 2. Fig. 7 shows an example of processing related to shake correction that the control unit 40 executes at the timing of each frame when recording a moving image.

[0096] In step S101, control unit 40 detects the amount of shake. For example, the amount of shaking that occurred between the timing of the previous frame and the current frame is detected from IMU data obtained by shake detection unit 34. In this case, the IMU data obtained by shake detection unit 25 in lens barrel 3 may be referenced.

[0097] In step S102, control unit 40 calculates the amount of body side shake compensation. The amount of body side shake compensation is the amount of compensation performed by image plane image stabilization unit 30. In step S103, control unit 40 calculates the amount of lens-side shake compensation. The amount of lens-side shake compensation is the amount of compensation performed by camera shake compensation lens mechanism 10c. For example, based on the amount of shake applied to the imaging device 1 detected in step S101, the control unit 40 calculates how much the image plane image stabilization unit 30 and the image stabilization lens mechanism 10c should be operated to achieve shake correction, and sets the amount of shake correction for each.

[0098] In this case, the control unit 40 calculates the amount of shake correction for each of the image plane image stabilization unit 30 and the image stabilization lens mechanism 10c, depending on the magnitude of the shake, the frequency component of the shake (vibration), the movable range of the image plane image stabilization unit 30, or the type and performance of the attached lens barrel 3. One reason for taking into consideration the type and performance of the lens barrel 3 in this case is that it is assumed that the lens barrel 3 will be an interchangeable lens. This is because the range of motion of the image stabilization mechanism 10c may differ depending on the lens barrel 3, and some lens barrels may not even be equipped with the image stabilization mechanism 10c.

[0099] When the overall amount of shake correction is set to "1," the control unit 40 determines the amount of shake correction by the image-sensing surface image stabilization unit 30 as "α" and the amount of shake correction by the image stabilization lens mechanism 10c as "1-α" (where 0≦α≦1), and calculates the respective amounts of shake correction.

[0100] In step S104, the control unit 40 transmits a correction instruction value corresponding to the amount of shake correction. That is, a correction instruction value indicating the position and amount of positional displacement for executing correction by the shake correction amount "α" is transmitted to the correction unit drive unit 32, and a correction instruction value indicating the position and amount of positional displacement for executing correction by the shake correction amount "1-α" is transmitted to the lens control unit 20. As a result, the correction unit driver 32 drives the image pickup surface image stabilization unit 30 in an amount corresponding to the shake correction amount "α." Furthermore, the lens control unit 20 transmits a correction instruction value to the correction lens driving unit 23, and the correction lens driving unit 23 drives the image stabilization lens mechanism 10c by an amount corresponding to the shake correction amount "1-α". As a result, optical image stabilization is performed on both the lens barrel 3 side and the camera body 2 side, and functions to reduce image shaking caused by camera shake or the like applied to the imaging device 1.

[0101] In step S105, the control unit 40 calculates a ratio. This ratio is information indicating the ratio between the body-side compensation value and the lens-side compensation value. In other words, it is information indicating the ratio between the shake compensation amount "α" and the shake compensation amount "1-α." The specific information on the ratio may be information that represents the ratio itself or information that allows the ratio to be calculated. In this embodiment, such information on the ratio is defined as conversion information. Conversion information is information that allows one blur compensation value to be found from another blur compensation value, such as information for finding a lens-side blur compensation value from a body-side blur compensation value. The ratio used as conversion information may be in the form of "α:(1-α)", or may be a value of "α" or "1-α". It may also be information showing "α" or "1-α" as a percentage. Furthermore, when the conversion information is considered to be a value that allows the lens-side shake compensation value to be calculated from the body-side shake compensation value, it may be a division value such as (lens-side shake compensation value) / (body-side shake compensation value) or a difference value between the lens-side shake compensation value and the body-side shake compensation value.

[0102] In step S106, the control unit 40 generates metadata related to optical image stabilization and records it as information corresponding to the current frame. In other words, as shown in Figure 5E, metadata is generated that combines the body side image stabilization value and conversion information, and this is recorded in correspondence with the current frame. As described above, the body-side image stabilization value is either an execution value or a command value. When the body-side image stabilization value is used as the execution value, control unit 40 detects actual position information of the image sensor via the position sensor of image plane image stabilization unit 30, and uses that position information itself, or the amount of displacement of the position information from the timing of the previous frame, as the body-side image stabilization value. When the correction instruction value is to be used as the body side image blur correction value, control unit 40 may use the correction instruction value instructed to correction unit driving unit 32 in step S104 as the body side image blur correction value as is. Although the metadata in this example is a set of a body-side image stabilization value and conversion information, it is also possible to use a set of a lens-side image stabilization value and conversion information as metadata. In this case, the conversion information is of course information that enables the body-side image stabilization value to be calculated using the lens-side image stabilization value.

[0103] As described above, if the body side image stabilization value and conversion information are stored as metadata in the form of time-series information for each frame, the lens side image stabilization value can also be calculated from the body side image stabilization value. Furthermore, there is no need to record the lens-side shake correction value as metadata, which contributes to reducing the amount of metadata recording required. For example, if the body stabilization value and lens stabilization value each use 4 bytes for the pitch, yaw, and roll directions, then if the body stabilization value and lens stabilization value are recorded as shown in Figure 5D, the total becomes 24 bytes. On the other hand, the conversion information (for example, the value of "α" above) can also be 4 bytes, but it is possible to use fewer bytes. Even if it is 4 bytes, in the case of Figure 5E, the total number of bytes is 16 bytes, which can reduce the amount of data. This is particularly effective in reducing data volume when recording long videos.

[0104] Recording the body side shake compensation value and conversion information is also suitable for cases where the data communication speed from lens control unit 20 to camera control unit 18 is slow. In other words, even if the actual lens side shake compensation value (actual correction value) cannot be obtained in a timely manner from lens control unit 20, metadata recording is performed and the lens side shake compensation value can be estimated from the conversion information.

[0105] By recording metadata as described above, for example, the image processing device 5 can offset the amount of shake correction performed on the image information for each frame, i.e., the correction amount indicated by the body-side shake correction value and the correction amount indicated by the lens-side shake correction value calculated from the conversion information. Even if distortion correction has been performed in image processing, the information in the metadata MTD1 can be used to create a state in which distortion correction has not been performed, and an image can be created as if no shake correction or distortion correction had been performed.

[0106] Then, by using the IMU data recorded as shown in Fig. 5C, the amount of shake of the camera body 2 can be recalculated from information from the gyro sensor and acceleration sensor, and shake correction and distortion correction can be performed again on images in a state where the above shake correction and distortion correction had not been performed. In other words, shake correction can be performed later on video data captured in the past.

[0107] FIG. 8 shows the processing of IMU data by the control unit 40. At each predetermined sampling timing, in step S161, the control unit 40 acquires the IMU data detected by the shake detection unit 34, and in step S162 performs processing to record the data as metadata. As described in FIG. 5C, the IMU data is not necessarily synchronized with the frame timing of the video, so control is performed so that, for example, multiple IMU data per frame are recorded as metadata. This associates the time series of IMU data with the video frames. It is assumed that such recording of IMU data as metadata will also be performed in the cases of the embodiments described below.

[0108] 5. Second Embodiment An example of processing by the control unit 40 in the second embodiment is shown in Fig. 9. Note that, hereinafter, the same step numbers will be used for processing that has already been described, and detailed overlapping explanations will be avoided.

[0109] The processing example in FIG. 9 is an example in which the ratio between the body side shake compensation amount and the lens side shake compensation amount is kept constant for a certain period of time. The control unit 40 performs the process of FIG. 9 at a timing corresponding to each frame of the moving image. In step S110, the control unit 40 branches the process depending on whether it is time to change the ratio.

[0110] The timing for changing the ratio is, for example, as follows. -Timing to start video recording When a scene change is detected during video recording When recording is paused during video recording -Timing of zoom movement during video recording

[0111] For example, when starting video recording, the ratio between the amount of body-side image stabilization and the amount of lens-side image stabilization is first set. For example, a predetermined initial value may be set, or a ratio determined according to the type or model of lens barrel 3 may be used.

[0112] After video recording begins, the timing for changing the ratio setting is determined to be the point at which the image content is analyzed and a possibility of a scene change is detected based on changes in people or objects recognized as subjects, changes in average brightness, or changes in the average amount of blur detected by the blur detection unit 34. This is to reset the ratio to a more appropriate setting depending on changes in the imaging conditions. Similarly, when recording is paused during video recording, the ratio setting is changed assuming a possible scene change.

[0113] Furthermore, when the zoom lens 10a is operated during video recording and the angle of view changes, the degree to which camera shake affects the image also changes, so this is also the timing to change the ratio setting.

[0114] The above is just one example, and there are other possible times when the ratio setting should be changed, but if it is determined in step S110 that it is time to change the ratio in this way, control unit 40 proceeds to step S111, where it sets the ratio between the body side shake compensation amount and the lens side shake compensation amount. In other words, it updates the setting of the shake compensation amount ratio. Then, in step S112, the control unit 40 generates conversion information corresponding to the newly set ratio and performs processing to record the information in correspondence with the current frame.

[0115] If it is determined in step S110 that it is not time to change the ratio, steps S111 and S112 are not performed.

[0116] In step S101, the control unit 40 detects the amount of shaking. In step S102A, control unit 40 calculates the body side image blur compensation amount. In step S103A, control unit 40 calculates the lens side image blur compensation amount. The calculations in steps S102A and S103A determine the body side image blur compensation amount and lens side image blur compensation amount based on the ratio set in the most recent step S111.

[0117] In step S104, the control unit 40 transmits a correction instruction value corresponding to the amount of body-side shake compensation to the correction unit driving unit 32, and also transmits a correction instruction value corresponding to the amount of lens-side shake compensation to the lens control unit 20, thereby causing both shake compensation operations to be performed. Then, in step S106, the control unit 40 generates metadata related to optical image stabilization and records it as information corresponding to the current frame. At this time, if the ratio has been set at the time of the current frame, the metadata related to the optical image stabilization is the body side image stabilization value (effective correction value or specified correction value) and conversion information. If the ratio has not been set at the time of this frame, the only metadata related to optical image stabilization will be the body stabilization value.

[0118] Therefore, the conversion information is recorded as metadata only when the ratio setting is changed, which can promote reduction in the amount of data as metadata MTD1. It goes without saying that, for example, in the image processing device 5, if conversion information does not exist for a certain frame, it is sufficient to detect conversion information by tracing back from that frame.

[0119] 6. Third Embodiment An example of processing by the control unit 40 in the third embodiment is shown in FIG. The processing example in FIG. 10 is an example in which the ratio between the body side shake compensation amount and the lens side shake compensation amount is kept constant for one video recording. The control unit 40 performs the process of FIG. 10 at a timing corresponding to each frame of the moving image. Only when recording starts, control unit 40 proceeds from step S115 to step S111, and sets the ratio between the amount of body-side shake compensation and the amount of lens-side shake compensation. In step S112, the control unit 40 generates conversion information indicating the newly set ratio and the like, and performs processing to record the conversion information in correspondence with the current frame (in this case, the first frame of the moving image).

[0120] At any time other than the start of recording, the process proceeds from step S115 to step S101, and the control unit 40 does not perform the processes of steps S111 and S112.

[0121] In step S101, the control unit 40 detects the amount of shaking. In step S102B, control unit 40 calculates the body side image blur compensation amount. In step S103B, control unit 40 calculates the lens side image blur compensation amount. The calculations in steps S102B and S103B determine the body side image blur compensation amount and lens side image blur compensation amount based on the ratio set in step S111 when movie recording started.

[0122] In step S104, the control unit 40 transmits a correction instruction value indicating the amount of body-side shake compensation to the correction unit driving unit 32, and also transmits a correction instruction value indicating the amount of lens-side shake compensation to the lens control unit 20, thereby causing both shake compensation operations to be performed. Then, in step S106, the control unit 40 generates metadata related to optical image stabilization and records it as information corresponding to the current frame. At this time, at the time of the first frame of the video, the metadata related to optical image stabilization is the body-side image stabilization value and conversion information. From the second frame onwards, the only metadata related to optical image stabilisation will be the body stabilisation value.

[0123] Therefore, the conversion information is recorded as metadata only for the first frame in the image file MF that constitutes the moving image, which facilitates a reduction in the amount of data as metadata MTD1. For example, image processing device 5 can obtain the ratio between the body-side shake compensation value and the lens-side shake compensation value for each frame by acquiring the conversion information for the first frame.

[0124] 7. Fourth Embodiment An example of processing by the control unit 40 in the fourth embodiment is shown in FIG. The processing example of FIG. 11 is an example in which a combined correction value of the body side shake correction value and the lens side shake correction value is recorded as metadata, as shown in FIG. 5F.

[0125] Steps S101 to S104 are the same as those in FIG. In step S120, control unit 40 calculates a composite correction value of the body side shake correction value and the lens side shake correction value. Then, in step S121, the control unit 40 performs a process to record the combined correction value as metadata related to optical image stabilization in association with the current frame.

[0126] This is effective in further reducing the data amount of the metadata MTD1. Note that, for example, image processing device 5 can detect a composite correction value for each frame, making it possible to change the shake so as to cancel the image stabilization performed by both image stabilization lens mechanism 10c and image plane image stabilization unit 30. In this case, it is not possible to cancel only the correction performed by image stabilization lens mechanism 10c, for example, but this is useful in use cases where such processing is not required. Conversely, if you only want to perform cancellation processing for overall image stabilization in image processing device 5, you also have the advantage that you do not need to calculate the combined image stabilization amount for both image stabilization lens mechanism 10c and image plane image stabilization unit 30.

[0127] 8. Fifth Embodiment The fifth embodiment is premised on the premise that both the body side image stabilization value and the lens side image stabilization value are recorded as metadata, as shown in FIG. 5D. The lens-side vibration compensation value is the actual position information of the image stabilization lens mechanism 10c, or the compensation execution value such as the amount of displacement thereof. Therefore, control unit 40 must wait for notification of the lens-side vibration compensation value from lens control unit 20.

[0128] In the fifth embodiment, as in the first to fourth embodiments described above, the correction amount for image stabilization lens mechanism 10c of lens barrel 3 may also be set by camera control unit 18, and a correction instruction value may be transmitted to lens control unit 20, or on the lens barrel 3 side, lens control unit 20 may set the correction amount for image stabilization lens mechanism 10c. That is, in this example, camera control unit 18 sets the amount of correction for image plane image stabilization unit 30 in accordance with the detection value of shake detection unit 34, thereby performing optical image stabilization on the camera body 2 side, and lens control unit 20 sets the amount of correction for image stabilization lens mechanism 10c in accordance with the detection value of shake detection unit 25, thereby performing optical image stabilization on the lens barrel 3 side.

[0129] 12 shows an example of optical image stabilization processing by control unit 40. This is an example in which the amount of correction on the lens barrel 3 side is also set on the control unit 40 side, and steps S101 to S104 are the same as those in FIG. As described above, there are cases where the camera control unit 18 and the lens control unit 20 perform control on the camera body 2 side and the lens barrel 3 side, respectively. In step S130 of FIG. 12, control unit 40 acquires the body side image stabilization value and sets it so that it is recorded as metadata corresponding to the current frame at a predetermined time point.

[0130] Control unit 40 performs the processing of Fig. 13 asynchronously with the processing of Fig. 12. That is, in step S130, control unit 40 determines whether or not a notification of a lens-side shake compensation value has been received from lens control unit 20, and if so, in step S131, sets the lens-side shake compensation value so that it is recorded corresponding to the current frame.

[0131] For example, both the body side shake compensation value and the lens side shake compensation value set as metadata in step S130 of FIG. 12 and step S131 of FIG. 13 are recorded as metadata corresponding to a certain frame, as shown in FIG. 5D.

[0132] However, there may be cases where the lens-side shake correction value obtained through communication with the lens control unit 20 in step S131 of FIG. 13 lacks precision as information, or where sufficient data cannot be obtained in a time series. For example, if the communication speed between camera control unit 18 and lens control unit 20 drops, it may not be possible to obtain lens-side shake compensation values ​​for all frames. In some cases, lens-side shake compensation values ​​for different frames may be obtained. In this case, depending on the communication speed, there is a possibility that the lens-side shake correction value may be insufficient for subsequent processing in image processing device 5.

[0133] The communication speed between the camera body 2 and the lens barrel 3 varies depending on the model and performance of the lens barrel 3. For example, if an older model of lens barrel 3 is attached, the communication speed may decrease.

[0134] Taking the above circumstances into consideration, the control unit 40 performs the process shown in FIG. 14A, and the image processing device 5 performs the process shown in FIG. 14B.

[0135] FIG. 14A shows an example in which the control unit 40 performs processing to record communication speed information (for example, bps: bits per second) as metadata. In step S201, the control unit 40 detects and stores communication speed information between the lens barrel 3 and the camera body 2, that is, between the lens control unit 20 and the camera control unit 18. In step S202, the control unit 40 records information about the communication speed together with the image data in the metadata (see the dashed line in FIG. 5B).

[0136] If the type of lens barrel 3 is the dominant factor in fluctuations in communication speed, then it is sufficient to record the communication speed once for each video, for example. If there is a factor that causes the communication speed to fluctuate during video recording, it is conceivable to record the communication speed as metadata associated with each frame.

[0137] The image processing device 5 that processes the image file MF performs the process shown in FIG. 14B. In step S301, the image processing device 5 acquires image data VD1 and metadata MTD1 as an image file MF. For example, the image processing device 5 reads the image file MF from a recording medium. Alternatively, the image processing device 5 receives the image file MF transmitted from the imaging device 1.

[0138] In step S302, the image processing device 5 extracts information about the communication speed included in the metadata MTD1 and compares it with a threshold value. This threshold value is used to determine whether the communication speed between the lens control unit 20 and the camera control unit 18 was fast or slow. In other words, this threshold value is used to determine whether the lens-side shake correction value included in the metadata MTD1 is suitable for use in shake modification processing.

[0139] If the communication speed is greater than the threshold, the image processing device 5 determines that high-speed communication was taking place between the lens control unit 20 and the camera control unit 18 when the image file MF was captured by the imaging device 1, and that the reliability of the lens-side blur correction value is maintained, and proceeds to step S303 to generate a compatibility flag. In this case, the image processing device 5 performs processing settings based on the compatibility flag in step S305. Specifically, the settings are made so that shake change processing can be performed using both the body side shake compensation value and the lens side shake compensation value.

[0140] On the other hand, if the communication speed is not greater than the threshold value, the image processing device 5 determines that the communication between the lens control unit 20 and the camera control unit 18 was slow when the image file MF was captured by the imaging device 1, and the reliability of the lens side blur correction value was not maintained, and proceeds to step S304, where it generates an incompatible flag. In this case, the image processing device 5 performs processing settings based on the incompatible flag in step S305. For example, the lens-side shake correction value is set so that it cannot be used in the shake change processing.

[0141] By doing so, it is possible to prevent the image processing device 5 from performing shake change processing using an inappropriate lens-side shake correction value.

[0142] 9. Sixth Embodiment An example of processing by the control unit 40 in the sixth embodiment is shown in Fig. 15A, and an example of processing by the image processing device 5 is shown in Fig. 15B. This is another example of processing having the same purpose as Figs. 14A and 14B.

[0143] FIG. 15A shows an example in which the control unit 40 performs processing to record flag information as information on the communication speed. In step S201, the control unit 40 detects and holds the communication speed (bps) between the lens barrel 3 and the camera body 2, that is, between the lens control unit 20 and the camera control unit 18.

[0144] In step S210, the control unit 40 compares the communication speed with a threshold value. This threshold value is the same as the threshold value described in Fig. 14B, and is a threshold value used to determine whether the communication speed between the lens control unit 20 and the camera control unit 18 was fast or slow.

[0145] If the communication speed is greater than the threshold value, the control unit 40 proceeds to step S212 and generates a support availability flag. If the communication speed is not higher than the threshold value, the control unit 40 proceeds to step S213 and generates an incompatible flag. Then, in step S214, the control unit 40 records the communication speed information together with the image data in the metadata (see the dashed line in FIG. 5B). In this case, a compatible flag or an incompatible flag is recorded as communication speed information.

[0146] The image processing device 5 that processes the image file MF performs the process shown in FIG. 15B. In step S301, the image processing device 5 acquires image data VD1 and metadata MTD1 as an image file MF.

[0147] In step S310, the image processing device 5 performs processing settings based on the supportable flag or the incompatible flag recorded in the metadata MTD1. That is, if the supportable flag is confirmed, the image file MF is set to be able to perform shake change processing using both the body-side shake compensation value and the lens-side shake compensation value. On the other hand, if the incompatible flag is confirmed, the lens shake correction value for this image file MF is set to be unable to be used in the shake change process.

[0148] This also makes it possible to prevent the image processing device 5 from performing shake change processing using an inappropriate lens-side shake correction value. The speed information included in the metadata MTD1 may be, for example, one-bit flag information, which is also advantageous in reducing the data amount of the metadata MTD1. For example, a specific bit in the metadata MTD1 may be set to "0" as a compatible flag and "1" as an incompatible flag.

[0149] 10. Seventh Embodiment In the fifth and sixth embodiments described above, an example has been described in which the lens-side shake correction value received by the control unit 40 from the lens control unit 20 is used as metadata, as shown in FIG. In such a case, the lens control unit 20 may transmit the lens side shake correction value while temporarily storing it in the memory unit 26, which will be described as a seventh embodiment.

[0150] The body side image stabilization value and lens side image stabilization value may ultimately be recorded as metadata for the video. Therefore, it is conceivable that lens control unit 20 does not send the lens side image stabilization value in real time, but instead temporarily stores it in memory unit 26 and sends it when there is sufficient communication between the lens and the body.

[0151] For example, if communication between the lens and body is very slow and sampling the movement of image stabilization lens mechanism 10c at that communication interval results in a large change in movement per unit time being lost, making it impossible to send lens-side image stabilization values ​​in a state where accuracy as data is maintained to camera body 2 in real time, the process of temporarily storing the data in memory unit 26 and transmitting it later is useful.

[0152] From this perspective, the lens control unit 20 performs the process shown in FIG. 16 during video capture and the like. In step S501, the lens control unit 20 determines whether it is time to transmit. This transmission timing is a predetermined timing. For example, it is conceivable that the timing can be set to periodic transmission, irregular transmission, timing according to the storage capacity of the memory unit 26, when video recording ends, etc. Depending on the content of such settings, the lens control unit 20 determines in step S501 whether or not the current timing is right.

[0153] If it is not the transmission timing, lens control unit 20 proceeds to step S502, where it detects a lens-side shake compensation value as the compensation execution value in image stabilization lens mechanism 10c. Then, in step S503, it stores the currently detected lens-side shake compensation value in memory unit 26 in association with a timestamp indicating the current time (the time corresponding to the current frame). This is because in order to associate the lens shake correction value with the video frame, the timestamp must also be sent as a set from the camera body 2.

[0154] When the transmission timing arrives, lens control unit 20 proceeds to step S504 and performs processing to transmit the lens side shake correction value recorded in memory unit 26 to camera control unit 18 together with a time stamp.

[0155] 16, when the set transmission timing arrives, the plurality of samples of lens-side vibration compensation values ​​stored in memory unit 26 are transmitted together to camera control unit 18. For example, a batch transmission can be performed at regular intervals. Alternatively, a batch transmission can be performed in response to some kind of trigger at irregular intervals. By performing a batch transmission on a regular or irregular basis, the number of transmission opportunities is reduced compared to when lens side shake correction values ​​are transmitted one after another in real time, and the communication load can be reduced. Furthermore, a batch transmission may be performed at a timing that depends on the storage capacity of the lens side vibration compensation values ​​in memory unit 26. This makes it possible to avoid a situation where memory unit 26 is unable to store all the lens side vibration compensation values. In addition, the video may be sent all at once after the video has been captured, allowing for transmission at a time when there is sufficient communication capacity. These transmission timings may be used in combination.

[0156] During moving image recording, the control unit 40 performs the process shown in FIG. 17 in addition to the process shown in FIG. 12 described above. The control unit 40 waits for reception of the lens side shake correction value from the lens control unit 20 in step S601 of FIG. When a lens-side blur compensation value is received from lens control unit 20, in step S602, the lens-side blur compensation value is set as metadata. At this time, a timestamp is transmitted corresponding to each lens-side blur compensation value. Therefore, for each received lens-side blur compensation value, control unit 40 uses the timestamp to determine which frame of the video the lens-side blur compensation value corresponds to, and sets the value as metadata corresponding to the determined frame.

[0157] The body side shake compensation value and lens side shake compensation value set as metadata in step S130 in FIG. 12 and step S602 in FIG. 17 are associated with the current frame, either individually or simultaneously, and recorded as metadata on the recording medium by the recording control unit 14.

[0158] The user may be allowed to select whether to transmit the lens side shake correction value in real time or to temporarily store it as described above and transmit it at a set timing. While temporarily storing and later transmitting has the advantage of reducing the communication load as described above, transmitting in real time also has the advantage of reducing the memory section 26 on the lens barrel 3 side, and of reducing the transmission time and metadata processing time that come with transmitting at a later time. Therefore, it is appropriate to let the user make the selection depending on the situation.

[0159] Also, a case where real-time transmission and transmission after temporary storage are used in combination is conceivable. FIG. 18 shows the processing of the lens control unit 20.

[0160] In step S601A, the lens control unit 20 determines whether it is time to transmit all of the lens-side shake correction values ​​stored in the memory unit 26 at once. If it is not the transmission timing, lens control unit 20 proceeds to step S502, where it detects the lens-side vibration compensation value as the compensation execution value in image stabilization lens mechanism 10c.

[0161] In step S530, the lens control unit 20 determines whether there is currently a margin for transmission, for example, based on the current communication speed, the amount of data to be transmitted, and the like. If there is a transmission margin, the lens control unit 20 proceeds to step S531 and transmits the currently detected lens side shake correction value to the camera control unit 18 together with a time stamp. On the other hand, if there is no transmission margin, in step S532, the currently detected lens-side vibration compensation value is stored in memory unit 26 in association with a timestamp indicating the current time (the time corresponding to the current frame).

[0162] When it is time to send the data in step S501A, lens control unit 20 proceeds to step S504 and performs processing to send the lens-side shake correction value recorded in memory unit 26 to camera control unit 18 together with a time stamp.

[0163] 18, the lens-side image stabilization value is transmitted both in real time and all at once after being temporarily stored. By pairing both with a timestamp, the control unit 40 can appropriately associate the value with a video frame and convert it into metadata. By using real-time transmission in combination, there is also the advantage that the capacity of the memory unit 26 can be reduced.

[0164] <11. Summary and Variations> According to the above embodiment, the following effects can be obtained. As described in the first to seventh embodiments, the control unit 40 of the imaging device 1 generates correction information based on both the body-side blur correction value (first blur correction value) associated with the imaging surface image stabilization unit 30 (first blur correction function) of the camera body 2 and the lens-side blur correction value (second blur correction value) associated with the image stabilization lens mechanism 10c (second blur correction function) in the lens barrel 3, and processes the correction information as metadata to be associated with the captured image. That is, when the imaging device 1 is provided with a mechanical shake correction function on both the lens barrel 3 side and the camera body 2 side, correction information based on each shake correction value is associated as metadata with the moving image being captured. This makes it possible to change the state of image shake at a later point in time, for example, in image processing device 5. For example, it becomes possible to cancel both the shake correction performed by image stabilization lens mechanism 10c and the shake correction performed by image plane image stabilization unit 30, or to cancel only one of the shake corrections. Furthermore, if the shake correction performed by imaging device 1 is canceled, it becomes possible to perform more precise shake correction in image processing device 5, or to intentionally add shake to create a shake effect. Although the embodiment has been described focusing on the mechanical shake prevention function provided by image plane image stabilization unit 30 and image stabilization lens mechanism 10c, the technology of the present disclosure can also be applied to cases where an electronic image stabilization function is employed. For example, if image stabilization is performed by image stabilization lens mechanism 10c on the lens barrel 3 side and electronic image stabilization is performed on the camera body 2 side, the electronic image stabilization function can be considered the first image stabilization function, and image stabilization lens mechanism 10c can be considered the second image stabilization function.

[0165] In the examples described in the first, second, third, and fourth embodiments, control unit 40 generates correction information (optical image stabilization information in FIG. 5B, see FIGS. 5E and 5F) with a data volume that is smaller than the data volume obtained by adding together the data volumes of the body-side blur compensation value and the lens-side blur compensation value. This makes it possible to reduce the amount of data recorded as metadata MTD1. In other words, when it is desired to store image blur correction values ​​for both the lens barrel 3 and the camera body 2, it is possible to avoid unnecessarily increasing the amount of data.

[0166] In the examples described in the first, second, and third embodiments, control unit 40 sets, as the correction information for optical image stabilization, one of the body-side blur compensation value and the lens-side blur compensation value, and conversion information indicating the ratio, division value, difference value, etc. between them. This makes it possible to determine the shake correction value by the image stabilization lens mechanism 10c and the shake correction value by the image plane image stabilization unit 30 from the correction information recorded as metadata, which is the same as if these were recorded as metadata. In this case, the amount of data recorded as metadata can be reduced. In other words, when it is desired to store correction values ​​for both the lens barrel 3 and the camera body 2, it is possible to avoid unnecessarily increasing the amount of data. In the first, second, and third embodiments, examples have been described in which the shake compensation value and conversion information of image plane image stabilization unit 30 are recorded as metadata. However, the shake compensation value and conversion information of image stabilization lens mechanism 10c may also be recorded as metadata. In this case, the conversion information may be the ratio "α:(1-α)" described above, or may be a value such as "α" or "1-α." Furthermore, considering that the conversion information is a value that can be used to calculate the body-side shake compensation value from the lens-side shake compensation value, the conversion information may be a division value such as (body-side shake compensation value) / (lens-side shake compensation value) or a difference between the lens-side shake compensation value and the body-side shake compensation value.

[0167] In the examples described in the first, second, and third embodiments, control unit 40 uses the body side shake correction value and conversion information as correction information for optical image stabilization. This reduces the amount of data recorded as metadata, and also allows control unit 40 to generate metadata without receiving the image blur correction value from lens barrel 3. Therefore, metadata can be generated and recorded without being affected by the communication speed between camera control unit 18 and lens control unit 20 or by communication delays.

[0168] In the example described in the first embodiment, control unit 40 generates correction information that is a set of conversion information and one of the body-side and lens-side shake correction values ​​at a predetermined timing, for example, at the timing of each frame. This makes it possible to obtain the shake correction value by the image stabilization lens mechanism 10c and the shake correction value by the image plane image stabilization unit 30 for each frame, for example, from the metadata, and to appropriately cancel correction or perform shake effects in the image processing device 5, for example.

[0169] In the example described in the second embodiment, control unit 40 generates correction information that includes one of the body side blur compensation value and the lens side blur compensation value, and conversion information at the timing when the ratio between the body side blur compensation value and the lens side blur compensation value is converted. This eliminates the need to record conversion information for each frame, for example, and reduces the amount of data in the metadata MTD1.

[0170] In the example described in the third embodiment, control unit 40 keeps the ratio between the body side blur compensation value and the lens side blur compensation value fixed from the start to the end of video recording, and generates correction information that includes one of the body side blur compensation value and the lens side blur compensation value at each timing and conversion information that indicates the fixed ratio. This eliminates the need to record conversion information for each frame, reducing the amount of metadata data. Recording only one piece of conversion information per video is particularly effective in reducing data volume.

[0171] In the example described in the fourth embodiment, control unit 40 sets the composite correction value of the body side shake correction value and the lens side shake correction value as the correction information. This reduces the amount of data recorded as metadata. In addition, since it is a composite correction value, it indicates the overall amount of correction applied to the image, and correction cancellation can be appropriately performed in the image processing device 5, for example.

[0172] In the examples described in the fifth and sixth embodiments, the control unit 40 uses the body side shake compensation value and the lens side shake compensation value as correction information, and also performs processing to record communication speed information between the lens barrel 3 (lens control unit 20) and the camera body 2 (camera control unit 18) on the recording medium as metadata. This makes it possible to check from the metadata whether the lens-side image stabilization value information is being affected by the communication speed. For example, if the communication speed is slow and the lens-side image stabilization value is insufficient, it becomes possible to not cancel the correction using the lens-side image stabilization value.

[0173] In the fifth embodiment, the communication speed information is a value indicating the communication speed (for example, bps). This makes it possible to check from the metadata whether the information on the lens-side shake correction value is delayed, or the degree of delay.

[0174] In the sixth embodiment, the communication speed information is information on the result of comparing the communication speed with a predetermined value (supportable flag / unsupportable flag). This makes it easy to check from the metadata whether the information on the lens-side blur correction value is delayed or not.

[0175] In the embodiment, an example has been given in which the shake correction value as the body side shake correction value or the lens side shake correction value is a correction execution value that indicates the position or the amount of position displacement due to correction by the shake correction function (image plane image stabilization unit 30 or image stabilization lens mechanism 10c). This is a value that indicates the amount of correction that actually affects the image, and is the most accurate value when considering correction cancellation in the image processing device 5, for example.

[0176] In the embodiment, an example has also been described in which the shake correction value is a correction instruction value that indicates a position or a displacement amount of a position for correcting the shake correction function. The correction instruction value is not a value that indicates the actual amount of shake correction applied to the image, but if the operation accuracy of the shake correction function in response to the instruction is high, there is no problem in using it as a value that approximately represents the actual amount of shake correction. Furthermore, because the correction instruction value is generated by camera control unit 18, there is the advantage that metadata can be generated and recorded without being affected by communication with lens barrel 3.

[0177] In the embodiment, an example has been described in which the control unit 40 records information (IMU data) detected by the shake detection unit 34 as metadata. By recording the IMU data as metadata as information on the shaking that actually occurs in the image capture device 1, it is possible to determine the shaking that has affected the actual image from the metadata. This makes it possible to change the shaking in a variety of ways.

[0178] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0179] The present technology can also be configured as follows. (1) and a control unit that generates correction information based on both a first blur correction value associated with a first blur correction function that corrects the positional relationship between an optical image incident through a lens and an output captured image, and a second blur correction value associated with a second blur correction function provided in a lens barrel that includes the lens, as metadata associated with the captured image. Imaging device. (2) The control unit generating the correction information with a data amount smaller than the data amount obtained by adding together the data amounts of the first blur correction value and the second blur correction value; The imaging device according to (1) above. (3) The control unit one of the first blur correction value and the second blur correction value; Conversion information for determining one of the motion compensation values ​​using the other motion compensation value; is the correction information. The imaging device according to (1) or (2) above. (4) The control unit the first blur correction value; Conversion information for obtaining the second motion blur correction value using the first motion blur correction value; is the correction information. The imaging device according to any one of (1) to (3) above. (5) The control unit At each predetermined timing, generating the correction information as a set of one of the first and second blur correction values ​​and conversion information for obtaining the other blur correction value using the one blur correction value; The imaging device according to any one of (1) to (4) above. (6) The control unit One of the first blur correction value and the second blur correction value at each timing; conversion information for determining a ratio of the first blur correction value to the second blur correction value using one of the first and second blur correction values ​​at a timing when the ratio of the other blur correction value changes; Generate the correction information including The imaging device according to any one of (1) to (4) above. (7) The control unit The ratio between the first shake correction value and the second shake correction value is fixed from the start to the end of recording of the moving image, and One of the first blur correction value and the second blur correction value at each timing; Conversion information according to the fixed ratio for obtaining one blur correction value using the other blur correction value; Generate the correction information including The imaging device according to any one of (1) to (4) above. (8) The control unit The correction information is a composite correction value of the first blur correction value and the second blur correction value. The imaging device according to (1) or (2) above. (9) The control unit The first blur correction value and the second blur correction value are set as the correction information, A process for recording communication speed information between the lens barrel and the camera body as metadata on a recording medium is performed. The imaging device according to (1) above. (10) The communication speed information is a value indicating the communication speed. The imaging device according to (9) above. (11) The communication speed information is information on the result of comparing the communication speed with a predetermined value. The imaging device according to (9) above. (12) The shake correction value is a correction execution value that indicates the position or the amount of displacement of the position due to the correction of the shake correction function. The imaging device according to any one of (1) to (11) above. (13) The shake correction value is a correction instruction value that indicates the position or the amount of displacement of the position for correction of the shake correction function. The imaging device according to any one of (1) to (11) above. (14) The control unit sets the vibration information detected by the vibration detection unit as metadata. The imaging device according to any one of (1) to (13) above. (15) a process of generating correction information based on both a first blur correction value associated with a first blur correction function that corrects the positional relationship between an optical image incident through a lens and an output captured image, and a second blur correction value associated with a second blur correction function provided in a lens barrel that includes the lens, as metadata associated with the captured image; An imaging method performed by an imaging device. [Explanation of symbols]

[0180] 1. Imaging device 2 Camera body 3 Lens barrel 5,6 Image processing device 10 Lens system 10a zoom lens 10b Aperture mechanism 10c Image stabilization lens mechanism 10d focus lens 11 Shutter 12 Image sensor section 13 Camera signal processing section 14 Recording control section 15 Display 16 Output section 17 Control section 18 Camera control unit 19,26 Memory section 20 Lens control unit 21 Zoom drive unit 22 Aperture drive unit 23 Correction lens drive unit 24 Focus drive unit 25,34 Shake detection section 27,33 Communication control unit 30 Image-sensing surface image stabilization unit 31 Shutter drive unit 32 Correction unit drive section 35 Electronic image stabilization control unit 36 Image stabilization metadata processing section 40 Control Unit

Claims

1. a control unit that generates, as metadata associated with the captured image, correction information based on both a first blur correction value associated with a first blur correction function that corrects a positional relationship between an optical image incident through a lens and an output captured image, and a second blur correction value associated with a second blur correction function provided in a lens barrel that includes the lens; The control unit generating the correction information with a data amount smaller than the data amount obtained by adding together the data amounts of the first blur correction value and the second blur correction value; Imaging device.

2. a control unit that generates, as metadata associated with the captured image, correction information based on both a first blur correction value associated with a first blur correction function that corrects a positional relationship between an optical image incident through a lens and an output captured image, and a second blur correction value associated with a second blur correction function provided in a lens barrel that includes the lens; The control unit the first blur correction value and the second blur correction value are set as the correction information; A process for recording communication speed information between the lens barrel and the camera body as metadata on a recording medium is performed. Imaging device.

3. The control unit generating the correction information with a data amount smaller than the data amount obtained by adding together the data amounts of the first blur correction value and the second blur correction value; The imaging device according to claim 2 .

4. The control unit one of the first blur correction value and the second blur correction value; Conversion information for determining one of the motion compensation values ​​using the other motion compensation value; is the correction information.

3. The imaging device according to claim 1.

5. The control unit the first blur correction value; conversion information for obtaining the second motion blur correction value using the first motion blur correction value; is the correction information.

3. The imaging device according to claim 1.

6. The control unit At each predetermined timing, generating correction information that is a set of one of the first and second blur correction values ​​and conversion information for obtaining the other blur correction value using the one blur correction value; 3. The imaging device according to claim 1.

7. The control unit one of the first blur correction value and the second blur correction value at each timing; conversion information for determining a ratio of the first blur correction value to the second blur correction value using one of the first and second blur correction values ​​at a timing when the ratio of the other blur correction value changes; Generate the correction information including 3. The imaging device according to claim 1.

8. The control unit The ratio between the first shake correction value and the second shake correction value is fixed from the start to the end of recording of the moving image, and one of the first blur correction value and the second blur correction value at each timing; Conversion information according to the fixed ratio for obtaining one blur correction value using the other blur correction value; Generate the correction information including 3. The imaging device according to claim 1.

9. The control unit A composite correction value of the first blur correction value and the second blur correction value is set as the correction information.

3. The imaging device according to claim 1.

10. The communication speed information is a value indicating the communication speed. The imaging device according to claim 2 .

11. The communication speed information is information on the result of comparing the communication speed with a predetermined value. The imaging device according to claim 2 .

12. The shake correction value is a correction execution value that indicates the position or the amount of displacement of the position due to the correction of the shake correction function.

3. The imaging device according to claim 1.

13. The shake correction value is a correction instruction value that indicates the position or the amount of displacement of the position for correction of the shake correction function.

3. The imaging device according to claim 1.

14. The control unit sets the vibration information detected by the vibration detection unit as metadata.

3. The imaging device according to claim 1.

15. An imaging device comprising: performing a process of generating correction information based on both a first blur correction value associated with a first blur correction function that corrects the positional relationship between an optical image incident through a lens and an output captured image, and a second blur correction value associated with a second blur correction function provided in a lens barrel that includes the lens, as metadata to be associated with the captured image; The correction information is generated with a data amount smaller than the data amount obtained by adding together the data amounts of the first blur correction value and the second blur correction value. Imaging method.

16. An imaging device comprising: performing a process of generating correction information based on both a first blur correction value associated with a first blur correction function that corrects the positional relationship between an optical image incident through a lens and an output captured image, and a second blur correction value associated with a second blur correction function provided in a lens barrel that includes the lens, as metadata to be associated with the captured image; the first blur correction value and the second blur correction value are set as the correction information; A process for recording communication speed information between the lens barrel and the camera body as metadata on a recording medium is performed. Imaging method.

Images