Imaging device, lens barrel device, imaging method, transmission method

Abstract

In this imaging device, processing is performed for generating, as metadata associated with a captured image, correction information including: a first camera shake correction value for a first camera shake correction function by which the positional relationship between an optical image incident via a lens and an output captured image is corrected; and a second camera shake correction value for a second camera shake correction function provided to a lens tube comprising a lens.

Description

Technical Field

[0001] The present technology relates to an imaging device and an imaging method thereof, a lens barrel device and a transmission method thereof, and particularly to the processing of shake correction data.

Background Art

[0002] For example, an imaging device configured by a camera body (camera body) and a lens barrel and capable of recording a moving image on a recording medium is known, such as an interchangeable lens camera. In such an imaging device, there are those that perform shake correction by mechanically moving the shake correction function inside the camera body, or those that perform shake correction by mechanically moving the shake correction function inside the lens barrel. There is also an electronic shake correction function that changes the readout range of the image signal from the imaging element according to shake or changes the cutout range of the image in image signal processing.

[0003] The following Patent Document 1 discloses a configuration in which shake correction is performed on the lens barrel side and the camera body side, respectively.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] These days, users are in an environment where they can easily perform various image shootings and image adjustments using a mobile terminal such as a smartphone or a tablet, or the camera itself or a personal computer. In this case, it may be necessary to remove the influence of hand shake during shooting with high precision, or conversely, to actively add shake to the image to obtain a production effect. Therefore, in the present disclosure, for a video after imaging, it is assumed that image shake is added or removed, and a technique is proposed to ensure that appropriate information is stored in an imaging device for this purpose.

Means for Solving the Problems

[0006] The imaging device according to the present technology includes a control unit that generates, as metadata associated with an imaging image, correction information including a first shake correction value related to a first shake correction function for correcting the positional relationship between an optical image incident through a lens and an output imaging image, and a second shake correction value related to a second shake correction function provided in a lens barrel including the lens. The first shake correction function is a function for correcting the positional relationship between the optical image and the output imaging image (the image finally output from the imaging device) on the body side. The second shake correction function is a function provided on the lens barrel side. In this way, when each of the lens barrel side and the camera body side is provided with a mechanical or electronic shake correction function, correction information including the respective shake correction values of the lens barrel side and the camera body part side is associated with the video being imaged as metadata. For example, the metadata is recorded on a recording medium in a state associated with the image data.

[0007] In the imaging device according to the present technology described above, it is conceivable that the control unit uses the second shake correction value received from the lens barrel as the correction information. The second shake correction value related to the second shake correction function on the lens barrel side is transmitted from the lens barrel to the camera body part. The control unit includes this in the metadata.

[0008] In the imaging device according to the present technology described above, it is conceivable that high-frequency component shake correction is performed by the first shake correction function, and low-frequency component shake correction is performed by the second shake correction function. The shake caused by hand shake or the like applied to the imaging device is divided into high-frequency components and low-frequency components, and shake correction for the high-frequency components is performed on the body side, and shake correction for the low-frequency components is performed on the lens barrel side.

[0009] In the imaging device according to the present technology described above, the lens barrel is configured to be detachable from the camera body, and the control unit performs blur correction for high-frequency components by the first blur correction function and performs blur correction for low-frequency components by the second blur correction function based on the determination of the communication speed with the mounted lens barrel. It can be considered to control as follows. For example, when a lens barrel with a slow communication speed with the main body is mounted, the shake due to hand shake or the like applied to the imaging device is divided into high-frequency components and low-frequency components, and the main body side performs blur correction for the high-frequency components, and the lens barrel side performs blur correction for the low-frequency components.

[0010] In the imaging device according to the present technology described above, in the lens barrel, the second blur correction value at each sample timing is stored in the storage unit together with the time information, and at a predetermined time, the second blur correction value stored in the storage unit and the time information are transmitted to the control unit. It can be considered that the control unit performs a process of using the second blur correction value received from the lens barrel as metadata corresponding to the frame of the moving image based on the time information. Rather than sequentially transmitting the second blur correction value to the main body side, the lens barrel side stores it in the storage unit and transmits it at a certain time.

[0011] In the imaging device according to the present technology described above, it can be considered that the control unit uses the second blur correction value received from the lens barrel and the second blur correction value generated by the interpolation process using the received second blur correction value as the correction information. For example, when the second blur correction value transmitted from the lens barrel side has a low sampling rate, the second blur correction value by the interpolation process can also be generated.

[0012] In the imaging device according to the present technology described above, the control unit controls such that blur correction of high-frequency components is performed by the first blur correction function and blur correction of low-frequency components is performed by the second blur correction function, and it is conceivable to set the frequency range of the low-frequency components based on the communication speed with the lens barrel side. When blur correction of low-frequency components is to be performed on the lens barrel side, the frequency range of the low-frequency components is set according to the communication speed.

[0013] In the imaging device according to the present technology described above, the control unit controls such that blur correction of high-frequency components is performed by the first blur correction function and blur correction of low-frequency components is performed by the second blur correction function, and it is conceivable to set the frequency range of the low-frequency components based on the storage capacity of the storage unit. When blur correction of low-frequency components is to be performed on the lens barrel side, the frequency range of the low-frequency components is set based on the storage capacity of the storage unit, for example, the storage capacity for storing the second blur correction value.

[0014] In the imaging device according to the present technology described above, it is conceivable that the control unit performs a process of recording the communication speed information between the lens barrel and the camera body as metadata on a recording medium. In addition to the first blur correction value and the second blur correction value, the communication speed information between the lens barrel side and the camera body side is also recorded as metadata.

[0015] The lens barrel device according to the present technology includes a blur correction function for displacing a blur correction lens, a storage unit, a process for detecting a blur correction value related to the blur correction function at each sample timing, a process for storing the detected blur correction value in the storage unit together with time information, and a control unit that performs a process of transmitting the blur correction value and the time information stored in the storage unit to a camera body which is a mounting destination at a predetermined time point. When the lens barrel side is provided with a mechanical blur correction function, the blur correction value is temporarily stored together with time information and then transmitted to the camera body side.

[0016] In the lens barrel device according to the present technology described above, it is conceivable that the control unit performs a process of transmitting the shake correction value and the time information stored in the storage unit at intermittent timings during the video imaging period. The control unit on the lens barrel side transmits, for example, regularly or irregularly to the camera body side while storing the shake correction value in the storage unit during the period of video recording.

[0017] In the lens barrel device according to the present technology described above, it is conceivable that the control unit performs a process of transmitting the shake correction value and the time information stored in the storage unit at a point in time after the video imaging is completed. The control unit on the lens barrel side transmits the shake correction value accumulated in the storage unit to the camera body side after the video recording is completed.

[0018] In the lens barrel device according to the present technology described above, the control unit selects whether to transmit the shake correction value detected at each sample timing to the camera body unit or store it in the storage unit, and for the shake correction value to be stored in the storage unit, it is stored in the storage unit together with the time information, and at a predetermined point in time, it is read out from the storage unit and transmitted to the camera body unit together with the time information. It is conceivable to perform such a process. The control unit on the lens barrel side transmits the shake correction value during video recording, but if transmission is inappropriate for some reason, it stores it in the storage unit and transmits it at a later time.

[0019] The imaging method according to the present technology includes a first shake correction value related to a first shake correction function for correcting the positional relationship between the optical image incident through the lens and the output imaging image, and a second shake correction value related to a second shake correction function provided in a lens barrel including the lens. The imaging device performs a process of generating correction information including the correction value as metadata associated with the imaging image. Thereby, at the time of image imaging, the shake correction values on both the lens barrel side and the body side can be referred to from the metadata.

[0020] The transmission method according to the present technology is such that a lens barrel device having a shake correction function for displacing a shake correction lens and a storage unit performs a process of detecting a shake correction value related to the shake correction function at each sample timing, a process of storing the detected shake correction value in the storage unit together with time information, and a process of transmitting the shake correction value and the time information stored in the storage unit to a camera body unit which is the mounting destination at a predetermined time point. Thereby, information transmission is appropriately performed to the camera body side regardless of the communication speed.

Brief Description of the Drawings

[0021]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Embodiments for Carrying Out the Invention

[0022] Hereinafter, the embodiments will be described in the following order. <1. Shake change by the imaging device and the image processing device> <2. Configuration of the imaging device and metadata> <3. Functions of the image processing device> <4. Setting process related to shake correction> <5. First embodiment> <6. Second embodiment> <7. Third embodiment> <8. Fourth embodiment> <9. Fifth embodiment> <10. Sixth embodiment> <11. Summary and modification examples>

[0023] <1. Shake change by the imaging device and the image processing device> FIG. 1 shows an example of the imaging device 1 according to the embodiment and an image processing device (5, 6) that acquires the image file MF captured by the imaging device 1.

[0024] In the figure, an example is shown in which the mobile terminal 7 and the personal computer 8 function as the image processing devices 5 and 6. Although not shown, other various devices such as an image editing dedicated device, a cloud server, a television device, and a video recording and playback device are assumed as the image processing devices 5 and 6. These devices can function as either of the image processing devices 5 and 6.

[0025] The image processing device 5 is a device that primarily performs a shake change process on the image data acquired from the imaging device 1. On the other hand, the image processing device 6 is a device that secondarily performs a shake change process on image data for which a shake change process has already been performed by another image processing device.

[0026] Note that "shake" refers to the interframe shake of the images constituting the video. It broadly refers to the vibration components (the shaking of the images between frames) that occur between frames, such as the shake of the image due to camera shake in the image captured by the imaging device 1 or the shake intentionally added by image processing. When referring to "shake" caused by camera shake during imaging by the imaging device 1, the term "camera shake" is also used. The correction for reducing the shake of the image caused by camera shake, etc. (including vibrations added when the imaging device 1 is fixedly arranged without being held by hand) performed within the imaging device 1 is called "camera shake correction", which is distinguished from the "shake change" process in the image processing devices 5 and 6.

[0027] "Interframe shake modification" refers to changing the state of shake in an image, such as reducing the shake occurring in the image or adding shake to the image. This "interframe shake modification" shall include the following "interframe shake reduction" and "interframe shake production".

[0028] "Interframe shake reduction" refers to eliminating (completely removing the shake) or reducing (partially removing the shake) the shake occurring in the image due to camera shake, etc. by the image processing devices 5 and 6.

[0029] "Interframe shake production" refers to changing the state of shake of the image by the image processing devices 5 and 6. In some cases, this interframe shake production may reduce the shake, and in that sense, the result may be the same as "interframe shake reduction". However, in this embodiment, it refers to changing the amount of change in shake according to an instruction given by a user operation or automatic control and changing the shake state of the image in response to the instruction. For example, reducing or increasing the shake that occurred during imaging according to a user instruction or the like, or adding a new shake, corresponds to "shake effect". As an example of the purpose of the shake effect, it is assumed that the image is deliberately shaken to give a sense of impact to the scene of the video.

[0030] The imaging device 1 in FIG. 1 is a so-called digital still camera or a digital video camera, and is capable of at least video imaging.

[0031] The camera body portion of the imaging device 1 is shown as the camera body 2. The lens barrel 3 functions as a so-called interchangeable lens and is detachable from the camera body portion (camera body 2) of the imaging device 1. The user can replace the lens barrel 3 according to the use case and use it. In the embodiment, such an interchangeable lens type imaging device 1 is assumed, but the technology of the present disclosure can also be applied to a type in which the lens barrel 3 cannot be removed from the camera body 2.

[0032] The imaging device 1 performs video imaging and can transfer the image file MF obtained by the video imaging to a mobile terminal 7 or a personal computer 8 as the image processing device 5 via wired communication or wireless communication. Alternatively, the imaging device 1 may record the image file MF on a recording medium such as a memory card, and the mobile terminal 7 or the personal computer 8 may be able to read the image file MF from the memory card. The image file MF includes not only the image data as a video but also the metadata which is the additional information corresponding to the image data.

[0033] FIG. 2 shows the state of information transmission in the imaging device 1, the image processing device 5, and the image processing device 6. From the imaging device 1 to the image processing device 5, the image data VD1 and the metadata MTD1 are transmitted via wired communication, wireless communication, or a recording medium. The image data VD1 and the metadata MTD1 are information transmitted, for example, as an image file MF. In the case of this embodiment, the metadata MTD1 includes, for example, information regarding shake correction during imaging.

[0034] The image processing apparatus 5 can perform various processes upon receiving the image data VD1 and the metadata MTD1. For example, the image processing apparatus 5 can perform a shake change process on the image data VD1 using information regarding shake correction included in the metadata MTD1 and the like. As described above, the shake change refers to a process of canceling shake correction to return to the original shaky image, performing more advanced shake removal, or adding shake to the image for the purpose of production.

[0035] The image processing apparatus 5 can transfer the image data VD2 on which a shake change process or the like has been performed and the metadata MTD2 to another image processing apparatus 6. In this case, information regarding the shake change process or the like is added as the metadata MTD2, so that the image processing apparatus 6 can also perform various shake changes.

[0036] In this embodiment, assuming such information transmission, at least the shake change in the image processing apparatus 5 can be appropriately performed. The description will focus on the recording of metadata during imaging in the imaging apparatus 1 for this purpose.

[0037] <2. Configuration of Imaging Apparatus and Metadata> FIG. 3 shows a configuration example of the imaging apparatus 1 and the lens barrel 3.

[0038] A lens system 10 having a plurality of optical components is formed in the lens barrel 3. For example, the lens system 10 includes a zoom lens 10a, a diaphragm mechanism 10b, a shake correction lens mechanism 10c, a focus lens 10d, and the like. The hand shake correction lens mechanism 10c is a mechanism that reduces the shake generated in an image by mechanically driving the lens against hand shake.

[0039] The light (incident light) from the subject is condensed onto the imaging element unit 12 through such a lens system 10 and the shutter 11 in the camera body 2.

[0040] The imaging element unit 12 is configured to include an image sensor (imaging element) such as a CMOS (Complementary Metal Oxide Semiconductor) type or a CCD (Charge Coupled Device) type. In this imaging element unit 12, for the electrical signal obtained by photoelectrically converting the light received by the image sensor, for example, CDS (Correlated Double Sampling) processing, AGC (Automatic Gain Control) processing, etc. are executed, and further A / D (Analog / Digital) conversion processing is performed. Then, the imaging signal as digital data is output to the subsequent camera signal processing unit 13.

[0041] An imaging plane hand shake correction unit 30 is provided for the imaging element unit 12. This imaging plane hand shake correction unit 30 is a mechanism that corrects the shake of the image by mechanically moving the image sensor against hand shake or the like.

[0042] The camera signal processing unit 13 is configured as an image processing processor by, for example, a DSP (Digital Signal Processor) or the like. This camera signal processing unit 13 performs various signal processes on the digital signal (imaging image signal) from the imaging element unit 12. For example, as a camera process, the camera signal processing unit 13 performs preprocessing, synchronization processing, YC generation processing, various correction processes, resolution conversion processing, codec processing, etc.

[0043] In the preprocessing, for the imaging image signal from the imaging element unit 12, clamp processing for clamping the black levels of R, G, and B to a predetermined level, correction processing between the color channels of R, G, and B, etc. are performed. In the synchronization process, color separation processing is performed so that the image data for each pixel has all color components of R, G, and B. For example, in the case of an imaging element using a Bayer array color filter, demosaicing processing is performed as the color separation processing. In the YC generation process, a luminance (Y) signal and a color (C) signal are generated (separated) from the R, G, and B image data. In the resolution conversion process, resolution conversion processing is executed on the image data subjected to various signal processes.

[0044] Examples of various correction processes performed in the lens system 10 to the camera signal processing unit 13 are given in FIG. 4. In FIG. 4, the correction processes performed in the camera signal processing unit 13 are illustrated in the order of execution, together with the optical image stabilization performed by the hand shake correction lens mechanism 10c and the imaging plane hand shake correction unit 30.

[0045] As the optical image stabilization of process F1, lens anti - shake by the hand shake correction lens mechanism 10c and body anti - shake by the imaging plane hand shake correction unit 30 are performed. For example, hand shake correction as lens anti - shake by the shift of the hand shake correction lens mechanism 10c in the yaw direction and pitch direction, and hand shake correction as body anti - shake by the shift of the image sensor in the yaw direction and pitch direction by the imaging plane hand shake correction unit 30 are performed, so that the image of the subject is formed on the image sensor in a state where the influence of hand shake is physically canceled. This lens anti - shake and body anti - shake may be executed only on one side or both sides. Note that as hand shake correction, electrical image stabilization may be performed separately from the above optical image stabilization.

[0046] In the camera signal processing unit 13, the processes from process F2 to process F6 are performed by spatial coordinate conversion for each pixel. In process F2, lens distortion correction is performed. In process F3, focal plane distortion correction is performed as one element of electronic shake correction. This corrects the distortion that occurs, for example, when reading in a rolling shutter method using a CMOS type image sensor.

[0047] In process F4, roll correction is performed. That is, correction of the roll component is performed as one element of electronic shake correction. In process F5, trapezoidal distortion correction is performed for the trapezoidal distortion caused by electronic shake correction. The trapezoidal distortion caused by electronic shake correction is the perspective distortion that occurs when cutting out a location away from the center of the image. In process F6, shifting and cropping in the pitch direction and yaw direction are performed as one element of electronic shake correction. For example, shake correction, lens distortion correction, and trapezoidal distortion correction are performed in the above procedure. Note that it is not essential to perform all of the processes listed here, and the order of the processes may be appropriately changed.

[0048] In the codec processing in the camera signal processing unit 13 shown in FIG. 3, for the image data subjected to the above various processes, for example, encoding processing for recording or communication, and file generation are performed. For example, an image file MF is generated in the MP4 format used for recording MPEG-4 compliant moving images and audio. It is also conceivable to generate files in formats such as JPEG (Joint Photographic Experts Group), TIFF (Tagged Image File Format), and GIF (Graphics Interchange Format) as still image files.

[0049] Although the illustration of the audio processing system is omitted in FIG. 3, actually, it may have an audio recording system and an audio processing system, and the image file MF may also include audio data together with the image data as a moving picture.

[0050] The camera control unit 18 is composed of a microcomputer (arithmetic processing unit) equipped with a CPU (Central Processing Unit). The memory unit 19 stores information and the like used by the camera control unit 18 for processing. The illustrated memory unit 19 comprehensively shows, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, etc.

[0051] The RAM in the memory unit 19 is used for temporarily storing data, programs, etc. as a work area during various data processes of the CPU of the camera control unit 18. The ROM and flash memory (non-volatile memory) in the memory unit 19 are used for storing an OS (Operating System) for the CPU to control each part, content files such as image files, application programs for various operations, firmware, etc. The memory unit 19 may be a memory area built into the microcomputer chip as the camera control unit 18, or may be composed of a separate memory chip.

[0052] The camera control unit 18 controls the entire imaging device 1 and the lens barrel 3 by executing the programs stored in the ROM, flash memory, etc. of the memory unit 19. For example, the camera control unit 18 controls the operations of necessary parts regarding the control of the shutter speed of the imaging element unit 12, the instructions for various signal processes in the camera signal processing unit 13, the imaging operation and recording operation according to the user's operation, the playback operation of the recorded image file, the operations of the lens system 10 such as zoom, focus, and aperture adjustment in the lens barrel 3, the user interface operation, etc.

[0053] In addition, the camera control unit 18 also performs various processes and output controls on the image data processed by the camera signal processing unit 13. The camera control unit 18 can cause the electronic shake correction control unit 35 to perform electronic shake correction processing on the image data. In addition, the camera control unit 18 causes the shake correction metadata processing unit 36 to generate metadata related to shake correction. The camera control unit 18 also performs control to generate metadata composed of various types of information including information related to shake correction and record it as information related to the image file MF. In addition, the camera control unit 18 communicates with the lens control unit 20 on the lens barrel 3 side via the communication control unit 33.

[0054] In FIG. 3, the electronic shake correction control unit 35 and the shake correction metadata processing unit 36 are shown as separate blocks from the camera control unit 18, but these can be considered as functions realized by a microcomputer constituting the camera control unit 18. Therefore, for the sake of explanation, the camera control unit 18, the electronic shake correction control unit 35, and the shake correction metadata processing unit 36 are collectively referred to as the "control unit 40". However, these may be configured by separate arithmetic processing devices.

[0055] The recording control unit 14 performs recording and playback on a recording medium such as a non-volatile memory. The recording control unit 14 performs processes such as recording image files MF such as moving image data and still image data, and thumbnail images on the recording medium. The actual form of the recording control unit 14 can be considered in various ways. For example, the recording control unit 14 may be configured as a flash memory built into the imaging device 1 and its write / read circuit, or may be in the form of a card recording / playback unit that performs recording / playback access on a recording medium detachable from the imaging device 1, such as a memory card (portable flash memory, etc.). Also, it may be realized as an HDD (Hard Disk Drive) or the like in a form built into the imaging device 1.

[0056] The display unit 15 is a display unit that performs various displays for the user, and is, for example, a display panel or a viewfinder using a display device such as a liquid crystal panel (LCD: Liquid Crystal Display) or an organic EL (Electro-Luminescence) display arranged on the housing of the imaging device 1. The display unit 15 executes various displays on the display screen based on an instruction from the camera control unit 18. For example, the display unit 15 displays a reproduced image of the image data read from the recording medium in the recording control unit 14. In addition, image data of the captured image resolution-converted for display by the camera signal processing unit 13 is supplied to the display unit 15, and the display unit 15 may perform a display based on the image data of the captured image according to an instruction from the camera control unit 18. As a result, a so-called through image (subject monitoring image), which is a captured image during composition confirmation, is displayed. In addition, the display unit 15 executes displays such as various operation menus, icons, messages, etc., that is, displays as a GUI (Graphical User Interface) on the screen based on an instruction from the camera control unit 18.

[0057] The output unit 16 performs data communication and network communication with external devices by wire or wirelessly. For example, it transmits and outputs captured image data (still image files and moving image files) to an external display device, recording device, playback device, etc. Also, assuming that the output unit 16 is a network communication unit, it may perform communication via various networks such as the Internet, home network, LAN (Local Area Network), etc., and perform various data transmissions and receptions with servers, terminals, etc. on the network.

[0058] The operation unit 17 generally indicates input devices for the user to perform various operation inputs. Specifically, the operation unit 17 indicates various operation elements (keys, dials, touch panels, touch pads, etc.) provided on the housing of the imaging device 1. The operation unit 17 detects the user's operation, and a signal corresponding to the input operation is sent to the camera control unit 18.

[0059] The shutter drive unit 31 drives the shutter 11 based on an instruction from the camera control unit 18.

[0060] The correction unit drive unit 32 drives the imaging plane shake correction unit 30 based on an instruction from the camera control unit 18, and displaces the image sensor in the imaging element unit 12 for optical shake correction.

[0061] The shake detection unit 34 indicates a sensor that detects the shake applied to the camera body 2. As the shake detection unit 34, for example, an IMU (inertial measurement unit) is mounted. For example, the angular velocity can be detected by a three-axis angular velocity (gyro) sensor of pitch, yaw, and roll, and the acceleration can be detected by an acceleration sensor. Note that the shake detection unit 34 only needs to include a sensor that can detect camera shake during imaging, and does not necessarily need to have both a gyro sensor and an acceleration sensor.

[0062] In the lens barrel 3, for example, a lens control unit 20 by a microcomputer is mounted. When the lens barrel 3 is attached to the camera body 2, the camera control unit 18 and the lens control unit 20 can communicate with each other via the communication control units 27 and 33. The communication control units 27 and 33 are connected by wire and communicate when the lens barrel 3 is attached to the camera body 2. However, both may be configured to perform wireless communication.

[0063] The lens control unit 20 and the camera control unit 18 constantly exchange two-way data communication at a certain communication speed. For example, the camera control unit 18 issues drive instructions for the zoom lens 10a, focus lens 10d, aperture mechanism 10b, and shake correction lens mechanism 10c to the lens control unit 20. The lens control unit 20 causes the lens system 10 to execute operations according to these drive instructions. Also, the lens control unit 20 transmits to the camera control unit 18 lens distortion correction information, focal length information, the position of the correction lens of the shake correction lens mechanism 10c, and the like.

[0064] The lens barrel 3 is provided with, for example, a zoom drive unit 21 having a motor and a motor driver for driving the zoom lens 10a, a diaphragm drive unit 22 having a motor and a motor driver for driving the aperture mechanism 10b, a correction lens drive unit 23 having a motor and a motor driver for driving the shake correction lens mechanism 10c, and a focus drive unit 24 having a motor and a motor driver for driving the focus lens 10d. These zoom drive unit 21, diaphragm drive unit 22, correction lens drive unit 23, and focus drive unit 24 apply a drive current to the corresponding motor according to an instruction from the lens control unit 20 based on an instruction from the camera control unit 18. Thereby, a zoom operation, a diaphragm opening / closing operation, an optical shake correction operation, and a focus operation are executed.

[0065] The memory unit 26 stores information and the like used by the lens control unit 20 for processing. The memory unit 26 is generically shown as, for example, a ROM, a RAM, a flash memory, or the like. The memory unit 26 may be used for temporarily storing information that the lens control unit 20 transmits to the camera control unit 18.

[0066] The shake detection unit 25 indicates a sensor that detects shake within the lens barrel 3. For example, it is assumed that an IMU is mounted, similar to the shake detection unit 34 on the camera body 2 side. Note that it is also assumed that the shake detection unit 34 is not mounted within the lens barrel 3.

[0067] Subsequently, the content of the image file MF and the content of the metadata transmitted from the imaging device 1 to the image processing device 5 will be described. FIG. 5A shows the data included in the image file MF. As shown in the figure, the image file MF includes various types of data such as "header", "sound", "movie", and "metadata".

[0068] In the "header", information indicating the presence or absence of metadata, along with information such as the file name and file size, is described. "Sound" is audio data recorded together with the video. For example, 2-channel stereo audio data is stored. "Movie" is video data and is composed of image data for each frame (#1, #2, #3, ···) that makes up the video. As "metadata", additional information associated with each frame (#1, #2, #3, ···) that makes up the video is described.

[0069] An example of the content of the metadata is shown in FIG. 5B. For example, for one frame, IMU data, coordinate transformation parameter HP, timing information TM, and camera parameter CP are described. Note that these are part of the metadata content, and there may be other information. Also, there may be cases where any of the illustrated information is not included. Also, in FIG. 5B, a case where communication speed information is included in the metadata is shown by a dashed line, but this is the case when performing the processing of the fifth and sixth embodiments described later.

[0070] As the IMU data, gyro (angular velocity data), accel (acceleration data), and sampling rate are described. In the IMUs mounted on the imaging device 1 as the shake detection units 34 and 25, angular velocity data and acceleration data are output at a predetermined sampling rate. Generally, this sampling rate is higher than the frame rate of the captured image, and thus a large number of IMU data samples can be obtained in one frame period.

[0071] Therefore, as angular velocity data, n samples are associated with one frame, such as gyro sample #1, gyro sample #2, ···, gyro sample #n shown in FIG. 5C. Also, as acceleration data, m samples are associated with one frame, such as accel sample #1, accel sample #2, ···, accel sample #m. There are cases where n = m, and there are also cases where n ≠ m. Here, the metadata is described as an example associated with each frame. However, for example, IMU data may not be completely synchronized with the frame. In such a case, for example, time information related to the time information of each frame is used as the IMU sample timing offset in the timing information TM.

[0072] The coordinate transformation parameter HP is a general term for parameters used for correction involving coordinate transformation of each pixel in the image. It includes non-linear coordinate transformations such as lens distortion. And the coordinate transformation parameter HP is a term that can include at least lens distortion correction parameters, trapezoidal distortion correction parameters, focal plane distortion correction parameters, electronic shake correction parameters, and optical shake correction parameters.

[0073] The lens distortion correction parameter is information for directly or indirectly grasping how distortions such as barrel aberration and helical aberration are corrected and returning to the image before lens distortion correction.

[0074] The trapezoidal distortion correction parameter is the correction amount when correcting the trapezoidal distortion caused by shifting the cut-out area from the center by electronic shake correction, and it also becomes a value corresponding to the correction amount of the electronic shake correction.

[0075] The focal plane distortion correction parameter is a value indicating the correction amount for each line with respect to the focal plane distortion.

[0076] Regarding electronic shake correction and optical shake correction, they are parameters indicating the correction amounts in the respective axial directions of yaw, pitch, and roll.

[0077] Here, in the present embodiment, as described in FIG. 3, as an optical shake correction mechanism, a shake correction lens mechanism 10c and an imaging surface shake correction unit 30 are provided. Therefore, as correction information indicating the correction amount of optical shake correction, for example, as shown in FIG. 5D, a body-side shake correction value and a lens-side shake correction value are recorded. The body-side shake correction value is the shake correction value in the imaging surface shake correction unit 30. The lens-side shake correction value is the shake correction value in the shake correction lens mechanism 10c.

[0078] These shake correction values are assumed to be the actual correction execution values of the actual imaging surface shake correction unit 30 and the shake correction lens mechanism 10c. This correction effective value is a value representing the actual displacement performed as optical shake correction, such as the position information detected by the position sensors provided in the imaging surface shake correction unit 30 and the shake correction lens mechanism 10c, which changes due to actual correction, and the displacement amount of the position information from the previous frame.

[0079] Alternatively, these shake correction values may be correction instruction values output by the camera control unit 18 to the correction unit driving unit 32, or correction instruction values transmitted by the camera control unit 18 to the correction lens driving unit 23 via the lens control unit 20. This is because the imaging surface shake correction unit 30 and the shake correction lens mechanism 10c are driven so as to be in positions and displacement amounts corresponding to these correction instruction values.

[0080] Note that the parameters of lens distortion correction, trapezoidal distortion correction, focal plane distortion correction, and electronic shake correction are collectively referred to as coordinate conversion parameters. These correction processes are correction processes for the images formed on each pixel of the image sensor of the imaging element unit 12, and are parameters of the correction processes involving coordinate conversion of each pixel. Also, for the sake of explanation, correction information for optical hand shake correction is also regarded as one of the coordinate transformation parameters. This is because, in optical hand shake correction, correction of the shake of the inter-frame component involves processing with coordinate transformation for each pixel. That is, if inverse correction is performed using these parameters, it is possible to restore the image data subjected to lens distortion correction, trapezoidal distortion correction, focal plane distortion correction, electronic hand shake correction, and optical hand shake correction to the state before each correction process, that is, the state when it is imaged on the image sensor of the imaging element unit 12.

[0081] Also, regarding the parameters for lens distortion correction, trapezoidal distortion correction, and focal plane distortion correction, since each of these is a distortion correction process for an image in which the optical image itself from the subject is imaged in an optically distorted state and each is for the purpose of optical distortion correction, they are collectively referred to as optical distortion correction parameters. That is, if inverse correction is performed using these parameters, it is possible to restore the image data subjected to lens distortion correction, trapezoidal distortion correction, and focal plane distortion correction to the state before optical distortion correction.

[0082] As the timing information TM in the metadata, information such as exposure time (shutter speed), exposure start timing, readout time (frame rate), number of exposure frames (long exposure information), IMU sample offset, and frame rate is included. These are mainly used to associate the lines of each frame with the IMU data. However, even in the case where the image sensor 12a is a CCD or in the case of a global shutter type CMOS, when the exposure center of gravity is shifted using an electronic shutter or a mechanical shutter, it is possible to perform correction adjusted to the exposure center of gravity using the exposure start timing and the frame rate.

[0083] As the camera parameters CP in the metadata, the angle of view (focal length), zoom position, and lens distortion information are described.

[0084] <3. Functions of the Image Processing Apparatus> In the image processing device 5, it is possible to perform a shake change process on the image file MF generated by imaging with the imaging device 1. The functional configuration of the image processing device 5 for this purpose is shown in FIG. 6. It is assumed that the image processing device 6 also has a similar configuration.

[0085] The image processing device 5 can be considered as an information processing device such as the portable terminal 7 or the personal computer 8 in FIG. 1. In that case, functions such as those in FIG. 6 are formed by an application program. For example, the image processing device 5 has functions as a correction canceling unit 51, a shake effect unit 52, a process setting unit 53, and a user interface unit 54. Note that "user interface" is also denoted as "UI", and the user interface unit 54 is hereinafter denoted as "UI unit 54".

[0086] The correction canceling unit 51 and the shake effect unit 52 are functions for performing some kind of shake change on the image.

[0087] The correction canceling unit 51 is a function for canceling the optical shake correction or electronic shake correction performed by the imaging device 1 and returning to a state where no shake correction is applied, that is, a state where the original shake due to camera shake is present in the image, and performing a shake change. Alternatively, the correction canceling unit 51 can also perform processes such as canceling only the correction by the shake correction lens mechanism 10c, canceling only the correction by the imaging plane shake correction unit 30, and canceling only the electronic shake correction.

[0088] The shake effect unit 52 is a function for performing a process of changing the shake state of the image data according to parameters or instructions input by the user. This shake effect unit 52 can perform shake effect processes such as adding or removing shake on the image data VD1 on which the canceling process in the correction canceling unit 51 has not been performed, or the image data VD1 after the canceling process by the correction canceling unit 51. As the shake effect processing, processing such as reducing the shake of the image, performing shake removal with higher accuracy than the shake correction of the imaging device 1, or adding shake to the image is assumed.

[0089] The UI unit 54 is a function that presents an operator related to correction cancellation or shake change to the user and performs processing to acquire operation information by the operator.

[0090] The processing setting unit 53 sets processing parameters for correction cancellation based on the metadata MTD1 and causes the correction cancellation unit 51 to execute the processing. Further, the processing setting unit 53 sets processing parameters for shake change according to the user operation detected by the UI unit 54 and the metadata MTD1, and causes the shake effect unit 52 to execute the processing.

[0091] With such an image processing apparatus 5, the user can perform a desired shake change on the image file MF obtained by the imaging device 1. In particular, since the optical shake correction information is included in the metadata MTD1, the correction by the shake correction lens mechanism 10c and the correction by the imaging plane shake correction unit 30 can be recognized respectively, and processing according to each can be performed. For example, cancellation of shake correction, addition of appropriate shake, etc. become possible.

[0092] <4. Setting Process Related to Shake Correction> In the imaging device 1 of the present embodiment, shake correction is performed by each of the shake correction lens mechanism 10c in the lens barrel 3 and the imaging plane shake correction unit 30 in the camera body 2. By making both the shake correction on the lens barrel 3 side and the camera body 2 side function simultaneously, there are merits such as a wider range of movement can be used. Also, when shooting a video, by saving the motion data log as metadata in a state where the time series with the image is aligned with the video, a video that can use the metadata later can be obtained. For example, it is conceivable to save IMU data. In addition, as shown in FIG. 5D for the body-side shake correction value and the lens-side shake correction value, by recording information related to hand shake correction on the lens barrel 3 side and the camera body 2 side in the metadata, it is also possible to cancel the hand shake correction later or take a larger shake that could not be captured by the lens or the body by cropping for the video recorded while applying the hand shake correction.

[0093] Here, when recording a video on the camera body 2 side, the camera control unit 18 can easily save the body-side shake correction value as metadata as it is, but the lens-side shake correction value is received through communication with the lens control unit 20 and used as the metadata.

[0094] In such an imaging device 1, it is conceivable to perform low-frequency component hand shake correction on the lens barrel 3 side and high-frequency component correction on the camera body 2 side. When performing hand shake correction on the lens barrel 3 side and the camera body 2 side together, it is also conceivable to simply move each by half of the movement amount on the lens barrel 3 side and the camera body 2 side, but this will be divided by frequency component. The side that corrects the high-frequency component often has a smaller movement amount, and the side that corrects the low-frequency component often has a larger movement amount. Therefore, simply considering, it is also conceivable to allocate the correction of the high-frequency component to the side with a narrower movable range and the correction of the low-frequency component to the side with a wider movable range. However, in this embodiment, the lens barrel 3 side is made to handle the low-frequency component, regardless of the situation of the movable range. This is also related to the fact that the lens-side shake correction value is sent to the camera control unit 18 through communication (hereinafter also referred to as "lens-body communication") between the lens control unit 20 and the camera control unit 18.

[0095] By correcting the low-frequency component of camera shake on the lens barrel 3 side, the change in the movement of the camera shake correction lens mechanism 10c that moves per unit time becomes smaller compared to the case of correcting the high-frequency component. The fact that the change in movement is small means that the data as the lens-side shake correction value is less likely to be lost even if it is somewhat intermittent. Therefore, the sampling rate required to accurately represent the movement of the camera shake correction lens mechanism 10c when correcting the low-frequency component can also be reduced. Then, as the lens-side shake correction value transmitted from the lens control unit 20 to the camera control unit 18, it becomes possible to send data with sufficient accuracy even at a low sampling rate. This has the advantage of reducing the communication data volume (even if the communication is fast, it can be transmitted with a small amount of data), and the advantage that the lens-side shake correction value can be sent without degrading the accuracy even when the communication is slow.

[0096] In order to perform such an assignment of frequency components, for example, the camera control unit 18 performs setting processing as shown in FIG. 7 for video imaging.

[0097] In step S1 of FIG. 7, the camera control unit 18 branches the process depending on whether to record metadata. For example, either one is selected according to the user's operation or external setting information. When the mode is set not to record metadata, the subsequent processing is not executed.

[0098] When it is the mode to record metadata, the camera control unit 18 sets in step S2 to record metadata along with video imaging. In step S3, the camera control unit 18 branches the process depending on whether to turn on camera shake correction. For example, it is determined according to the user's setting operation. If camera shake correction is not performed, the process of FIG. 7 ends.

[0099] When camera shake correction is to be performed, the camera control unit 18 branches the process in step S4 depending on whether to perform camera shake correction on the camera body 2 side. This is selected, for example, according to the user's setting or the specification of an application program.

[0100] When performing shake correction on both sides of the camera body 2, in step S5, the camera control unit 18 sets to record the body-side shake correction value as metadata.

[0101] In step S6, the camera control unit 18 branches the process depending on whether to perform shake correction on both the camera body 2 side and the lens barrel 3 side. This is also selected according to, for example, user settings or specifications of an application program. Also, there may be a case where the mounted lens barrel 3 is a lens barrel not equipped with the shake correction lens mechanism 10c. In that case, shake correction only on the camera body 2 side is selected. When shake correction is not performed on the lens barrel 3 side, the process of FIG. 7 is ended.

[0102] When performing shake correction on both the camera body 2 side and the lens barrel 3 side, in step S7, the camera control unit 18 determines whether to perform sharing for each frequency component. As this determination, the following examples can be considered.

[0103] First, when performing shake correction on both the camera body 2 side and the lens barrel 3 side, it is conceivable to always perform sharing in terms of frequency components. Thereby, the above-described effective operation can always be executed regardless of the circumstances of communication between the lens and the body. Also, it may be determined according to the user's selection operation.

[0104] Also, it is conceivable to determine whether to perform sharing in terms of frequency components according to the model, model number, type, manufacturing year, software version, etc. of the lens barrel 3. That is, the communication performance is determined from the model, etc. of the lens barrel 3, and if the communication speed performance is equal to or higher than a predetermined value, sharing in terms of frequency components is not performed, and if the communication speed performance is less than a predetermined value, that is, if the communication speed is slow, sharing in terms of frequency components is performed.

[0105] In addition, as a result of the camera control unit 18 communicating with the lens control unit 20 with the lens barrel 3 connected, the communication speed (e.g., bps: bits per second) is measured. If the communication speed is equal to or higher than a predetermined value, sharing by frequency components is not performed. If the communication speed performance is not equal to or higher than the predetermined value, that is, if the communication speed is slow, sharing by frequency components may be performed.

[0106] Note that although it has been described that whether to perform sharing by frequency components is determined in the setting process of FIG. 7, whether to perform sharing by frequency components may be arbitrarily switched, or may be switched, for example, during video recording.

[0107] When sharing by frequency components is not performed, the camera control unit 18 proceeds to step S9. When sharing by frequency components is performed, the camera control unit 18 sets the frequency range in step S8. That is, after setting the range of the frequency of the blur as the low-frequency component to be borne by the lens barrel 3 side and the range of the frequency of the blur as the high-frequency component to be borne by the camera body 2 side, it proceeds to step S9.

[0108] The frequency for the low-frequency / high-frequency division in step S8 can be determined as follows. First, it can be considered to be determined according to the user's designated operation. Also, it can be considered to determine the frequency according to the model, model number, type, manufacturing year, software version, etc. of the lens barrel 3. That is, an appropriate frequency is determined according to the communication speed, the movable range of the shake correction lens mechanism 10c, the frequency response characteristics, etc. determined from these. It may also be determined according to the actually measured communication speed between the lens and the body. Also, in the case of the processing of the second and third embodiments described later, it is also appropriate to determine the frequency according to the capacity of the memory unit 26 on the lens barrel 3 side and the recordable time of the lens-side blur correction value.

[0109] In step S9, the camera control unit 18 selects whether to set the operation mode to temporarily store the data of the lens-side blur correction value in the memory unit 26 on the lens barrel 3 side. This operation will be described in the second and third embodiments. The lens control unit 20 stores the acquired lens-side blur correction value in the memory unit 26 once and then transmits it to the camera control unit 18 at a predetermined time point. Whether to select such an operation mode may depend on the user's operation, or it may be determined based on the communication speed determined from the model, model number, type, manufacturing year, software version, etc. of the lens barrel 3, or the actually measured value of the communication speed.

[0110] If such an operation mode is not set, the camera control unit 18 proceeds to step S13. When performing the transmission operation via such a memory unit 26, the camera control unit 18 performs transmission timing setting in step S10. As described in the second and third embodiments, as the transmission timing, periodic transmission, non-periodic transmission, timing according to the storage capacity of the memory unit 26, end of video recording, etc. are assumed. It is necessary to set at which timing to perform the transmission as such transmission timing.

[0111] In step S13, the camera control unit 18 receives the lens-side blur correction value through lens-body communication and sets it to be recorded as metadata. That is, in this case, in combination with the setting in step S5, both the body-side blur correction value and the lens-side blur correction value are included in the metadata as correction information.

[0112] In addition, at the stage of step S4, if it is determined that the camera body 2 side does not perform shake correction, the camera control unit 18 sets in step S11 to perform shake correction only on the lens barrel 3 side. Then, in step S12, the camera control unit 18 selects whether to set the mode to temporarily store the lens-side blur correction value in the memory unit 26. When setting this mode, it proceeds to step S10, and when not setting this mode, it proceeds to step S13.

[0113] <5. First Embodiment> As a first embodiment, a processing example of the control unit 40 (mainly the camera control unit 18 and the shake correction metadata processing unit 36) during video recording will be described with reference to FIGS. 8 and 9. Note that this example is a processing example when the camera control unit 18 is set to perform high-frequency component hand shake correction on the camera body 2 side and low-frequency component hand shake correction on the lens barrel 3 side. Also, on the lens barrel 3 side, it is assumed that the lens control unit 20 transmits the lens-side shake correction value in real time without temporarily storing it in the memory unit 26.

[0114] FIG. 8 shows a processing example related to shake correction that the control unit 40 executes at each frame timing during video recording. FIG. 9 shows a processing example in response to reception from the lens control unit 20.

[0115] In step S101, the control unit 40 detects the amount of shake. For example, from the IMU data by the shake detection unit 34, the amount of shake generated between the timings of the previous frame and the current frame is detected. In this case, the IMU data of the shake detection unit 25 in the lens barrel 3 may be referred to.

[0116] In step S102, the control unit 40 extracts the high-frequency component and the low-frequency component of the shake according to the setting of the frequency component sharing (setting of the shared frequency range) in step S8 of FIG. 7.

[0117] In step S103, the control unit 40 calculates the body-side shake correction amount. The body-side shake correction amount is the correction amount to be executed by the imaging plane hand shake correction unit 30, and is the correction amount corresponding to the high-frequency component of the shake extracted in step S102. Also, in step S104, the control unit 40 calculates the lens-side shake correction amount. The lens-side shake correction amount is the correction amount to be executed by the hand shake correction lens mechanism 10c, and is the correction amount corresponding to the low-frequency component of the shake extracted in step S102. For example, based on the high-frequency component and the low-frequency component of the amount of shake applied to the imaging device 1 detected in step S101, the control unit 40 calculates how much the imaging surface hand shake correction unit 30 and the hand shake correction lens mechanism 10c should operate respectively to perform shake correction, and sets the shake correction amount for each of them.

[0118] In step S105, the control unit 40 transmits a correction instruction value corresponding to the shake correction amount. That is, a correction instruction value indicating the position and the amount of position displacement for executing the correction of the correction amount for high-frequency shake is transmitted to the correction unit drive unit 32, and a correction instruction value indicating the position and the amount of position displacement for executing the correction for low-frequency shake is transmitted to the lens control unit 20. Thereby, the correction unit drive unit 32 drives the imaging surface hand shake correction unit 30 corresponding to the high-frequency component of the shake. Also, the lens control unit 20 transmits a correction instruction value to the correction lens drive unit 23, and the correction lens drive unit 23 drives the hand shake correction lens mechanism 10c corresponding to the low-frequency component of the shake. Thereby, optical hand shake correction is performed on both the lens barrel 3 side and the camera body 2 side, and it functions to reduce the shaking of the image due to hand shake or the like applied to the imaging device 1.

[0119] In step S106, the control unit 40 performs a process of using the body side shake correction value as metadata. This body side shake correction value is recorded corresponding to the current frame. On the other hand, regarding the lens side shake correction value, the control unit 40 waits to receive it from the lens control unit 20. In step S121 of FIG. 9, it waits to receive the lens side shake correction value from the lens control unit 20, and when the reception is successful, in step S122, it performs a process of using the lens side shake correction value as metadata.

[0120] The body-side shake correction value and the lens-side shake correction value set as metadata in the above steps S106 and S122 are respectively associated with the current frame, either individually or simultaneously, and are recorded on the recording medium by the recording control unit 14 as metadata.

[0121] By recording the metadata as described above, for example, in the image processing apparatus 5, for the image information of each frame, the amount of shake correction performed, that is, the correction amount indicated by the body-side shake correction value and the correction amount indicated by the lens-side shake correction value can be offset. Also, when distortion correction is performed in image processing, using the information of the metadata MTD1, a state where distortion correction has not been performed can be created, and an image can be created assuming that no shake correction or distortion correction has been performed.

[0122] Furthermore, by using the IMU data recorded as shown in FIG. 5C, the amount of shake of the camera body 2 can be recalculated from the information of the gyro sensor and the acceleration sensor, and shake correction and distortion correction can be performed again on the image in the state where the above shake correction and distortion correction have not been performed. That is, it is also possible to perform shake correction on the video data captured in the past later.

[0123] Note that FIG. 10 shows the processing of IMU data by the control unit 40. The control unit 40 takes in the IMU data detected by the shake detection unit 34 at step S161 at a predetermined sampling timing, and performs processing for recording as metadata at step S162. Since the IMU data is not necessarily synchronized with the frame timing of the video as described with reference to FIG. 5C, for example, control is performed such that a plurality of IMU data are recorded as metadata for each frame. Thereby, the time-series IMU data and the frames of the video are associated. Recording such IMU data as metadata is assumed to be executed also in the cases of each embodiment described below.

[0124] <6. Second Embodiment> As a second embodiment, an example will be described in which the lens control unit 20 transmits while temporarily storing the lens-side shake correction value in the memory unit 26. Hereinafter, the same step numbers will be assigned to the above-described processes, and detailed redundant explanations will be avoided.

[0125] The body-side shake correction value and the lens-side shake correction value may be finally recorded as metadata of the moving image. Therefore, it is conceivable that the lens control unit 20 does not send the lens-side shake correction value in real time, but temporarily stores it in the memory unit 26 and sends it when there is room in the lens-body communication.

[0126] In this case, if the lens barrel 3 side is performing shake correction of the low-frequency component, the change in the movement of the hand shake correction lens mechanism 10c that moves per unit time becomes smaller than when correcting the high-frequency component, and depending on the frequency, the accuracy can be maintained even if the data amount of the lens-side shake correction value is decimated to some extent. Also, for this reason, the lens-side shake correction value can be stored in a decimated form according to the low-frequency component that moves on the lens barrel 3 side, and the advantage of reducing the temporary storage capacity in the memory unit 26 can also be obtained.

[0127] Moreover, originally, the lens-body communication is very slow, and if the movement of the hand shake correction lens mechanism 10c is sampled at the communication interval, the change in movement per unit time that is lost is large, and even when the lens-side shake correction value in a state where the accuracy as data is maintained cannot be sent to the camera body 2 side in real time, the process of temporarily storing it in the memory unit 26 and sending it later is useful.

[0128] From such a viewpoint, the lens control unit 20 performs the process of FIG. 11 during moving image shooting or the like. In step S201, the lens control unit 20 determines whether it is the transmission timing. This transmission timing is the timing set in step S10 of FIG. 7. In the process of FIG. 7, the camera control unit 18 sets regular transmission, irregular transmission, timing according to the storage capacity of the memory unit 26, when video recording ends, etc., and notifies the lens control unit 20. The lens control unit 20 sets the transmission timing of the lens-side blur correction value according to the notified setting content, and in step S201, determines whether it is the current timing.

[0129] If it is not the transmission timing, the lens control unit 20 proceeds to step S202, and detects the lens-side blur correction value as the correction execution value in the hand-shake correction lens mechanism 10c. Then, in step S203, the lens-side blur correction value detected this time is stored in the memory unit 26 in association with a time stamp indicating the current time (the time corresponding to the current frame). This is because it is necessary to also set a time stamp and transmit it to the camera body 2 side in order to associate the lens-side blur correction value with the frame of the video.

[0130] Note that even when the lens-side blur correction value is sent in real time as in the first embodiment described above, there may be some delay. Therefore, also in the case of the first embodiment, it is desirable to send the lens-side blur correction value together with a time stamp. Also, when transmitting after temporarily storing as in the second embodiment, it is necessary to send the lens-side blur correction value together with a time stamp.

[0131] When the transmission timing is reached, the lens control unit 20 proceeds to step S204, and performs a process of transmitting the lens-side blur correction value recorded in the memory unit 26 to the camera control unit 18 together with a time stamp.

[0132] By the process of FIG. 11, when the set transmission timing is reached, the lens-side blur correction values of a plurality of samples stored in the memory unit 26 are collectively transmitted to the camera control unit 18. For example, as a regular timing, batch transmission is performed at regular intervals. Also, as an irregular timing, batch transmission is performed in response to some trigger. By performing batch transmission regularly or irregularly, the number of transmission opportunities can be reduced compared to the case of sequentially transmitting the lens-side blur correction values in real time, and the communication load can be reduced. Also, batch transmission may be performed at a timing according to the storage capacity of the lens-side blur correction values in the memory unit 26. Thereby, a situation where the lens-side blur correction values cannot be completely stored in the memory unit 26 can be avoided. Also, it may be batch-transmitted after the end of video imaging. Thereby, transmission can be performed at a time when there is margin in communication. These transmission timings may be used in combination.

[0133] In the control unit 40 of the camera body 2, during video recording, the process of FIG. 12 is performed together with the process of FIG. 8 described above. The control unit 40 waits for reception of the lens-side blur correction value from the lens control unit 20 in step S140 of FIG. 12. When the lens-side blur correction value is received from the lens control unit 20, in step S141, a process of using the lens-side blur correction value as metadata is performed. At this time, a time stamp is transmitted corresponding to each lens-side blur correction value. Therefore, for each received lens-side blur correction value, the control unit 40 determines which frame of the video the lens-side blur correction value corresponds to using the time stamp, and uses it as metadata corresponding to the determined frame.

[0134] The body-side blur correction value and the lens-side blur correction value set as metadata in step S106 of FIG. 8 and step S141 of FIG. 12 are associated with the current frame individually or simultaneously, and are recorded on the recording medium by the recording control unit 14 as metadata.

[0135] Similar to the first embodiment, the user may be allowed to select whether to transmit the lens-side shake correction value in real time or to transmit it at the timing set after temporarily storing it as in the second embodiment. Temporarily storing and then transmitting it later has advantages in terms of communication load as described above. On the other hand, transmitting it in real time has merits such as being able to reduce the memory unit 26 on the lens barrel 3 side, or being able to reduce the transmission time and metadata processing time due to later transmission. Therefore, it is also appropriate to allow the user to select according to the situation.

[0136] <7. Third Embodiment> The third embodiment is an example of using both real-time transmission and transmission after temporary storage. The processing of the lens control unit 20 is shown in FIG. 13.

[0137] In step S201A, the lens control unit 20 determines whether it is the timing for batch transmission of the lens-side shake correction value stored in the memory unit 26. If it is not the transmission timing, the lens control unit 20 proceeds to step S202 and detects the lens-side shake correction value as the correction execution value in the hand shake correction lens mechanism 10c.

[0138] In step S230, the lens control unit 20 determines whether there is currently a transmission margin. For example, the presence or absence of a transmission margin is determined based on the current communication speed, the amount of data to be transmitted, and other situations. If there is a transmission margin, the lens control unit 20 proceeds to step S231 and transmits the lens-side shake correction value detected this time to the camera control unit 18 together with a time stamp. On the other hand, if there is no transmission margin, in step S232, the lens-side shake correction value detected this time is stored in the memory unit 26 in association with a time stamp indicating the current time (the time corresponding to the current frame).

[0139] When the transmission timing arrives in step S201, the lens control unit 20 proceeds to step S204 and performs a process of transmitting the lens shake correction value recorded in the memory unit 26 to the camera control unit 18 together with the time stamp.

[0140] By the process of FIG. 13, for the lens shake correction value, both real-time transmission and batch transmission after temporary storage are used in combination. In either case, by being paired with the time stamp, on the control unit 40 side, it can be appropriately metadata-associated with the frames of the moving image. There is also an advantage that the capacity of the memory unit 26 can be reduced by using real-time transmission in combination.

[0141] <8. Fourth Embodiment> As a fourth embodiment, an example in which the control unit 40 performs interpolation of the lens shake correction value is described. The lens shake correction value transmitted from the lens control unit 20 to the camera control unit 18 has a low sampling rate and may be missing in the time series when associated with the moving image. That is, it is also assumed that it cannot be associated with all frames. Therefore, the lens shake correction value is generated by interpolation processing.

[0142] FIG. 14 shows an example of the processing of the control unit 40. The control unit 40 performs the processing of FIG. 14 for each frame of the moving image during or after the moving image recording. In step S130, the control unit 40 determines whether the lens shake correction value corresponding to the currently targeted frame has been received. If the corresponding lens shake correction value does not exist, it proceeds to step S131 and generates the lens shake correction value corresponding to the targeted frame by interpolation processing. For example, interpolation is performed by linear interpolation processing using the lens shake correction values in the previous and subsequent frames.

[0143] In step S121, the received lens shake correction value or the lens shake correction value generated by the interpolation processing is processed as metadata.

[0144] By doing so, even when the sampling rate is lowered, the lens-side blur correction value corresponding to each frame can be retained as metadata. In particular, since the hand-shake correction lens mechanism 10c performs hand-shake correction for low-frequency components and there are few minute position changes, the lens-side blur correction values obtained by linear interpolation or the like are also relatively highly accurate, and appropriate shake correction can be achieved even in the image processing apparatus 5 by using these values.

[0145] <9. Fifth Embodiment> The fifth embodiment is for enabling the reliability of the lens-side blur correction value to be indicated when both the body-side blur correction value and the lens-side blur correction value are recorded as metadata. As described so far, the control unit 40 needs to wait for a notification from the lens control unit 20 regarding the lens-side blur correction value.

[0146] In the fifth embodiment as well, similar to the first to fourth embodiments described above, the correction amount of the hand-shake correction lens mechanism 10c of the lens barrel 3 may be set on the side of the camera control unit 18 and the correction instruction value may be transmitted to the lens control unit 20, but on the side of the lens barrel 3, the lens control unit 20 may set the correction amount of the hand-shake correction lens mechanism 10c. That is, an example is where the camera control unit 18 sets the correction amount of the imaging surface hand-shake correction unit 30 according to the detection value of the blur detection unit 34 and executes optical hand-shake correction on the side of the camera body 2, and the lens control unit 20 sets the correction amount of the hand-shake correction lens mechanism 10c according to the detection value of the blur detection unit 25 and executes optical hand-shake correction on the side of the lens barrel 3.

[0147] In any case, the control unit 40 uses the body-side blur correction value and the lens-side blur correction value transmitted from the lens control unit 20 as metadata. In this case, the control unit 40 performs processing as shown in FIG. 15A, and also causes the image processing apparatus 5 to perform processing as shown in FIG. 15B.

[0148] Figure 15A shows an example in which the control unit 40 performs a process of recording information on the communication speed (e.g., bps) as metadata. In step S401, the control unit 40 detects and holds the communication speed of the lens-body communication, that is, the communication speed information between the lens control unit 20 and the camera control unit 18. In step S402, the control unit 40 records the information on the communication speed as metadata together with the image data (see the broken line part in FIG. 5B).

[0149] If the factor of the communication speed variation is dominated by the type of the lens barrel 3, for example, the communication speed may be recorded once for one video. If there is a factor causing the communication speed to vary during video recording, it is conceivable to record the communication speed as metadata in association with each frame.

[0150] The image processing apparatus 5 that processes the image file MF performs the process of FIG. 15B. In step S501, the image processing apparatus 5 acquires the image data VD1 and the metadata MTD1 as the image file MF. For example, the image file MF is read from a recording medium. Alternatively, the image file MF transmitted from the imaging apparatus 1 is received.

[0151] In step S502, the image processing apparatus 5 extracts the information on the communication speed included in the metadata MTD1 and compares it with a threshold value. This threshold value is a threshold value for determining whether the communication speed between the lens control unit 20 and the camera control unit 18 was in a fast state or a slow state. In other words, this threshold value is a threshold value for determining whether the lens-side blur correction value included in the metadata MTD1 is suitable for use in the shake change process.

[0152] If the communication speed is greater than the threshold value, the image processing apparatus 5 proceeds to step S503 and generates a corresponding flag, assuming that high-speed communication was performed between the lens control unit 20 and the camera control unit 18 during imaging by the imaging apparatus 1 of the image file MF and the reliability of the lens-side blur correction value was maintained. In that case, the image processing apparatus 5 performs processing settings based on the supportable flag in step S505. Specifically, the setting is such that it is possible to perform shake change processing using both the body-side shake correction value and the lens-side shake correction value.

[0153] On the other hand, if the communication speed is not greater than the threshold value, when the imaging device 1 of the image file MF is imaged by the image processing apparatus 5, the communication between the lens control unit 20 and the camera control unit 18 is slow, and the reliability of the lens-side shake correction value is not maintained. Then, the process proceeds to step S504 to generate a non-supportable flag. In that case, the image processing apparatus 5 performs processing settings based on the non-supportable flag in step S505. For example, the setting is such that the lens-side shake correction value cannot be used for shake change processing.

[0154] By doing so, in the image processing apparatus 5, it is possible to prevent shake change processing using an inappropriate lens-side shake correction value from being performed.

[0155] <10. Sixth Embodiment> An example of the processing of the control unit 40 in the sixth embodiment is shown in FIG. 16A, and an example of the processing of the image processing apparatus 5 is shown in FIG. 16B. These are other processing examples for the same purpose as FIGS. 15A and 15B above.

[0156] FIG. 16A is an example of the process in which the control unit 40 records flag information as communication speed information. In step S401, the control unit 40 detects and holds the communication speed (bps) between the lens barrel 3 and the camera body 2, that is, between the lens control unit 20 and the camera control unit 18.

[0157] In step S410, the control unit 40 compares the communication speed with the threshold value. This threshold value is the same as the threshold value described in FIG. 15B, and is a threshold value for determining whether the communication speed between the lens control unit 20 and the camera control unit 18 is in a fast state or a slow state.

[0158] If the communication speed is greater than the threshold value, the control unit 40 proceeds to step S412 and generates an available flag. If the communication speed is not greater than the threshold value, the control unit 40 proceeds to step S413 and generates an unavailable flag. Then, in step S414, the control unit 40 records the information on the communication speed together with the image data in the metadata (see the dashed line part in FIG. 5B). In this case, the available flag or the unavailable flag is recorded as the information on the communication speed.

[0159] The image processing apparatus 5 that processes the image file MF performs the processing in FIG. 16B. The image processing apparatus 5 acquires the image data VD1 and the metadata MTD1 as the image file MF in step S501.

[0160] In step S510, the image processing apparatus 5 performs a processing setting based on the available flag or the unavailable flag recorded in the metadata MTD1. That is, if the available flag can be confirmed, for the current image file MF, the setting is such that the shake change processing using both the body side shake correction value and the lens side shake correction value can be performed. On the other hand, if the unavailable flag is confirmed, for the current image file MF, the setting is such that the lens side shake correction value cannot be used for the shake change processing.

[0161] By doing so, in the image processing apparatus 5, it is possible to prevent the shake change processing using an inappropriate lens side shake correction value from being performed. Since the speed information included in the metadata MTD1 may be, for example, 1-bit flag information, it is also advantageous for reducing the data amount of the metadata MTD1. For example, for a certain specific bit in the metadata MTD1, for example, "0" may be set as the available flag and "1" may be set as the unavailable flag.

[0162] <11. Summary and Modification Examples> According to the above embodiments, the following effects can be obtained. As described in the first to sixth embodiments, in the imaging device 1, the control unit 40 generates correction information including a body-side blur correction value (first blur correction value) by the imaging surface hand shake correction unit 30 (first blur correction function) of the camera body 2 and a lens-side blur correction value (second blur correction value) by the hand shake correction lens mechanism 10c (second blur correction function) in the lens barrel 3, and performs a process of using the correction information as metadata associated with the captured image. Thereby, at a later time, for example, in an image processing device 5 or the like, the state of image shake can be changed. For example, it becomes possible to cancel both the blur correction by the hand shake correction lens mechanism 10c and the blur correction by the imaging surface hand shake correction unit 30, or to cancel only one of the blur corrections. Further, when the blur correction performed by the imaging device 1 is canceled, more precise shake correction can be performed in the image processing device 5, or a shake effect such as intentionally adding shake can be performed. In the embodiment, the description has focused on the mechanical anti-shake functions of the imaging surface hand shake correction unit 30 and the hand shake correction lens mechanism 10c, but the technology of the present disclosure can also be applied when an electronic hand shake correction function is adopted. For example, in a case where blur correction by the hand shake correction lens mechanism 10c is performed on the lens barrel 3 side and electronic hand shake correction is performed on the camera body 2 side, the electronic hand shake correction function may be regarded as the first blur correction function and the hand shake correction lens mechanism 10c may be regarded as the second blur correction function.

[0163] In the embodiment, the control unit 40 uses the lens-side blur correction value received from the lens barrel 3 side as correction information. Thereby, the camera control unit 18 can acquire the lens-side blur correction value by the hand shake correction lens mechanism 10c of the lens barrel 3 and record it as metadata. In particular, since the lens-side blur correction value is a correction execution value, highly accurate correction information can be recorded as metadata.

[0164] In the embodiment, an example has been described in which the imaging surface hand shake correction unit 30 performs blur correction of high-frequency components and the hand shake correction lens mechanism 10c performs blur correction of low-frequency components. The camera control unit 18 receives the lens-side blur correction value through communication with the lens control unit 20, and records the lens-side blur correction value as metadata. In this case, by correcting the low-frequency component on the lens side, the change in the movement of the lens that moves per unit time becomes smaller than when correcting the high-frequency component, and the sampling rate that can maintain the accuracy can be lowered. Moreover, particularly when the camera control unit 18 receives the lens-side blur correction value through communication with the lens control unit 20 and uses it as metadata, the communication volume between the camera control unit 18 and the lens control unit 20 can be reduced, and the advantage that the information accuracy is not degraded even when the communication speed is low can be obtained.

[0165] In the embodiment, the lens barrel 3 is configured to be detachable from the camera body 2, and the control unit 40 controls so that the high-frequency component blur correction is performed by the imaging plane hand-shake correction unit 30 and the low-frequency component blur correction is performed by the hand-shake correction lens mechanism 10c based on the determination of the communication speed with the mounted lens barrel 3. That is, at the time of step S7 in FIG. 7, the camera control unit 18 can determine the communication speed for the mounted lens barrel 3 by its type or communication trial, etc., and set whether to perform the distribution of the blur correction in terms of frequency components. Thereby, when a lens barrel 3 with a slow communication speed is mounted, it is possible to cause the lens barrel 3 side to perform the blur correction of the low-frequency component. Note that when a lens barrel 3 with a sufficiently high communication speed is mounted, by not performing such a distribution of the blur correction, components extraction in step S102 of FIG. 8 for the distribution becomes unnecessary. However, even in the case of a lens barrel 3 with a high communication speed, the distribution by frequency components may be performed, and in that case, the effect of reducing the communication volume can be obtained.

[0166] In the second and third embodiments, on the lens barrel 3 side, the lens-side blur correction value at each sample timing is stored in the memory unit 26 together with a time stamp (time information), and an example is given in which, at a predetermined time point, the lens-side blur correction value and the time stamp stored in the memory unit 26 are transmitted to the camera control unit 18. In this case, the control unit 40 performs processing to use the lens-side blur correction value received from the lens barrel 3 as metadata corresponding to the frame of the moving image based on the time stamp. The time stamp may be an absolute time, but for example, it may be the elapsed time since the start of recording of the moving image. In any case, it suffices that the time information is associated with the frame of the moving image. By doing so, regardless of the communication speed between the lens control unit 20 and the camera control unit 18, the camera control unit 18 can acquire the lens-side blur correction value at the time corresponding to each frame. Therefore, regardless of the communication speed, a highly accurate blur correction value can be recorded as metadata. Also, the communication processing load during moving image recording can be reduced. In addition, since the hand shake correction lens mechanism 10c is responsible for low-frequency blur correction, the sampling rate of the lens-side blur correction value can be reduced, which leads to a reduction in the amount of data of the lens-side blur correction value stored in the memory unit 26. For this reason, the capacity of the memory unit 26 does not become excessive. Also, the amount of transmission data when the lens-side blur correction value stored in the memory unit 26 is transmitted to the camera control unit 18 in a batch can be prevented from becoming excessive.

[0167] In the fourth embodiment, an example is described in which the control unit 40 uses the lens-side blur correction value received from the lens barrel 3 and the lens-side blur correction value generated by interpolation processing using the received lens-side blur correction value as correction information. Thereby, not limited to the sampling rate of the lens-side blur correction value and the like, for example, the lens-side blur correction value corresponding to each frame of the moving image can be obtained and recorded as metadata.

[0168] In the embodiment, an example is described in which the control unit 40 sets the frequency range of the low-frequency component to be handled on the lens barrel 3 side based on the communication speed with the lens barrel side. The slower the communication speed between the camera control unit 18 and the lens control unit 20, the more advantageous it is for the lens barrel 3 side to handle lower-frequency shake correction. Therefore, according to the communication speed, the low-frequency range of the shake for which the shake correction by the hand-shake correction lens mechanism 10c is to be executed is determined. Thereby, even when the communication speed is slow, it becomes easier to ensure the accuracy of the lens-side shake correction value used as metadata.

[0169] In the embodiment, an example was described in which the control unit 40 sets the frequency range of the low-frequency component to be handled on the lens barrel 3 side based on the storage capacity of the memory unit 26. When the lens control unit 20 on the lens barrel 3 side stores the lens-side shake correction value in the memory unit 26 and transmits it to the camera control unit 18 at a predetermined time, the lower the sampling rate by performing low-frequency shake correction, the larger the amount of lens-side shake correction values that can be stored in the memory unit 26. Therefore, based on the storage capacity of the memory unit 26, the low-frequency range of the shake for which the shake correction by the hand-shake correction lens mechanism 10c is to be executed is determined. Thereby, it is possible to reduce the amount of data accumulated in the memory unit 26 and reduce the communication opportunities. Also, effects such as reducing the amount of data to be transmitted even in a batch transmission at the end of a video can be obtained.

[0170] In the examples described in the fifth and sixth embodiments, it was assumed that the control unit 40 performs a process of recording the communication speed information between the lens barrel 3 (lens control unit 20) and the camera body 2 (camera control unit 18) on the recording medium as metadata. Thereby, it becomes possible to confirm from the metadata whether the information of the lens-side shake correction value is affected by the communication speed. For example, when the communication speed is low and the lens-side shake correction value is insufficient, it is also possible not to perform correction cancellation or the like using it.

[0171] In the fifth embodiment, the communication speed information was assumed to be a value indicating the communication speed (e.g., bps). Thereby, it is possible to confirm from the metadata whether the information of the lens-side shake correction value is delayed information or the degree of delay.

[0172] In the sixth embodiment, it is assumed that the communication speed information is result information (capable of corresponding flag / incompatible flag) obtained by comparing the communication speed with a predetermined value. Accordingly, it is possible to easily confirm from the metadata whether the information on the lens-side blur correction value is delayed information.

[0173] The lens barrel 3 in the embodiment includes a hand-shake correction lens mechanism 10c that displaces a blur correction lens, a memory unit 26, and a lens control unit 20. In the second and third embodiments, the lens control unit 20 performs a process of detecting a lens-side blur correction value at each sample timing, a process of storing the detected lens-side blur correction value in the memory unit 26 together with a time stamp, and a process of transmitting the lens-side blur correction value and the time stamp stored in the memory unit 26 to the camera control unit 18 of the camera body 2 which is the mounting destination at a predetermined time point. Accordingly, regardless of the communication speed between the lens control unit 20 and the camera control unit 18, the camera control unit 18 can appropriately acquire the lens-side blur correction value corresponding to the time of the frame. Therefore, the accuracy of the correction information in the metadata MTD1 can be improved.

[0174] In the second and third embodiments, an example is given in which the lens control unit 20 performs a process of transmitting the lens-side blur correction value and the time stamp stored in the memory unit 26 at intermittent timings during the video imaging period. During video recording, the lens control unit 20 can transmit the lens-side blur correction value to the camera control unit 18 according to circumstances such as the capacity of the memory unit 26. For example, periodic transmission may be performed, or transmission may be performed every time the amount of data stored in the memory unit 26 reaches a predetermined value.

[0175] In the second and third embodiments, an example is given in which the lens control unit 20 performs a process of transmitting the lens-side blur correction value and the time stamp stored in the memory unit 26 after the video imaging is completed. When there is a margin in communication after the video recording is completed, the lens-side blur correction value can be transmitted to the camera control unit 18. Note that the lens control unit 20 may collectively transmit all the lens-side blur correction values during video recording to the camera control unit 18 after the video recording ends. Alternatively, during video recording, while performing regular or irregular transmissions, after the video recording ends, the remaining lens-side blur correction values that have not been transmitted may be collectively transmitted. In any case, the advantage of transmitting during a period with sufficient margin for communication after the video recording ends can be obtained.

[0176] In the third embodiment, the lens control unit 20 selects whether to transmit the blur correction value detected at each sample timing to the camera main body unit or store it in the memory unit 26. For the blur correction value that is to be stored in the storage unit, it is stored in the memory unit 26 together with a time stamp, and is read from the memory unit 26 at a predetermined time and transmitted to the camera body 2 side together with the time stamp. That is, when there is sufficient margin for transmission, it is transmitted, and when delay is assumed, it is stored and transmitted later, thereby eliminating communication delay and preventing the capacity required for the memory unit 26 from becoming excessive.

[0177] In the embodiment, an example was described in which the control unit 40 records the information (IMU data) detected by the blur detection unit 34 as metadata. By recording the IMU data as metadata as information on the shake actually applied to the imaging device 1, it is possible to determine the shake that affected the actual image from the metadata. As a result, various shake changes become possible.

[0178] Note that the effects described in this specification are merely examples and are not limiting, and there may be other effects.

[0179] Note that this technology can also adopt the following configurations. (1) A control unit that generates correction information including a first shake correction value related to a first shake correction function for correcting the positional relationship between an optical image incident through a lens and an output captured image, and a second shake correction value related to a second shake correction function provided in a lens barrel including the lens, as metadata associated with the captured image. An imaging device. (2) The control unit uses the second shake correction value received from the lens barrel as the correction information. The imaging device according to (1) above. (3) The first shake correction function corrects high-frequency component shake, and the second shake correction function corrects low-frequency component shake. The imaging device according to (1) or (2) above. (4) The lens barrel is configured to be detachable from the camera body unit. Based on the determination of the communication speed with the attached lens barrel, the control unit controls so that the first shake correction function corrects high-frequency component shake and the second shake correction function corrects low-frequency component shake. The imaging device according to any one of (1) to (3) above. (5) In the lens barrel, the second shake correction value at each sampling timing is stored in a storage unit together with time information, and at a predetermined time, the process of transmitting the second shake correction value and the time information stored in the storage unit to the control unit is performed. The control unit performs a process of using the second shake correction value received from the lens barrel as metadata corresponding to a frame of a moving image based on the time information. The imaging device according to (1) to (3) above. (6) The control unit uses the second shake correction value received from the lens barrel and the second shake correction value generated by interpolation processing using the received second shake correction value as the correction information. The imaging device according to any one of (1) to (5) above. (7) The control unit controls so that blur correction of high-frequency components is performed by the first blur correction function and blur correction of low-frequency components is performed by the second blur correction function, and sets the frequency range of the low-frequency components based on the communication speed with the lens barrel side The imaging device according to any one of (1) to (6) above. (8) The control unit controls so that blur correction of high-frequency components is performed by the first blur correction function and blur correction of low-frequency components is performed by the second blur correction function, and sets the frequency range of the low-frequency components based on the storage capacity of the storage unit The imaging device according to (5) above. (9) The control unit performs a process of recording, as metadata, the communication speed information between the lens barrel and the camera body on a recording medium The imaging device according to any one of (1) to (8) above. (10) A blur correction function for displacing a blur correction lens, A storage unit, A control unit that performs a process of detecting a blur correction value related to the blur correction function at each sample timing, a process of storing the detected blur correction value in the storage unit together with time information, and a process of transmitting the blur correction value and the time information stored in the storage unit to the camera body which is the mounting destination at a predetermined time point, and A lens barrel device. (11) The control unit performs a process of transmitting the blur correction value and time information stored in the storage unit at intermittent timings during a moving image capturing period. The lens barrel device according to (10) above. (12) The control unit performs a process of transmitting the blur correction value and time information stored in the storage unit at a time point after the moving image capturing has ended. The lens barrel device according to (10) or (11) above. (13) The control unit selects whether to transmit the shake correction value detected at each sample timing to the camera body unit or store it in the storage unit, for the shake correction value to be stored in the storage unit, stores it in the storage unit together with the time information, and at a predetermined time, reads it from the storage unit and performs a process of transmitting it to the camera body unit together with the time information The lens barrel device according to any one of (10) to (12) above. (14) generating, as metadata associated with the captured image, correction information including a first shake correction value related to a first shake correction function for correcting the positional relationship between the optical image incident through the lens and the output captured image, and a second shake correction value related to a second shake correction function provided in the lens barrel including the lens An imaging method performed by the imaging device. (15) a shake correction function for displacing the shake correction lens, a storage unit, As a transmission method in a lens barrel device including a process of detecting a shake correction value related to the shake correction function at each sample timing, a process of storing the detected shake correction value in the storage unit together with the time information, at a predetermined time, a process of transmitting the shake correction value stored in the storage unit and the time information to the camera body unit which is the mounting destination A transmission method for performing the above.

Explanation of Signs

[0180] 1 Imaging device 2 Camera body 3 Lens barrel 5,6 Image processing device 10 Lens system 10a Zoom lens 10b Aperture mechanism 10c Handshake correction lens mechanism 10d Focus lens 11 Shutter 12 Image sensor unit 13 Camera signal processing unit 14 Recording control unit 15 Display unit 16 Output unit 17 Operation unit 18 Camera control unit 19 Memory unit 20 Lens control unit 21 Zoom drive unit 22 Aperture drive unit 23 Correction lens drive unit 24 Focus drive unit 25 Shake detection unit 26 Memory unit 27 Communication control unit 30 Imaging plane shake correction unit 31 Shutter drive unit 32 Correction unit drive unit 33 Communication control unit 34 Shake detection unit 35 Electronic shake correction control unit 36 Shake correction metadata processing unit 40 Control unit

Claims

1. A control unit that generates, as metadata associated with a captured image, correction information including a first shake correction value related to a first shake correction function for correcting the positional relationship between an optical image incident through a lens and an output captured image, and a second shake correction value related to a second shake correction function provided in a lens barrel including the lens. The lens barrel is configured to be detachable from the camera body unit. Based on a determination of the communication speed with the attached lens barrel, the control unit controls so that high-frequency component shake correction is performed by the first shake correction function and low-frequency component shake correction is performed by the second shake correction function. An imaging device.

2. A control unit that generates, as metadata associated with a captured image, correction information including a first shake correction value related to a first shake correction function for correcting the positional relationship between an optical image incident through a lens and an output captured image, and a second shake correction value related to a second shake correction function provided in a lens barrel including the lens. The control unit controls so that high-frequency component shake correction is performed by the first shake correction function and low-frequency component shake correction is performed by the second shake correction function, and sets the frequency range of the low-frequency component based on the communication speed with the lens barrel side. An imaging device.

3. A control unit that generates, as metadata associated with a captured image, correction information including a first shake correction value related to a first shake correction function for correcting the positional relationship between an optical image incident through a lens and an output captured image, and a second shake correction value related to a second shake correction function provided in a lens barrel including the lens. In the lens barrel, the second shake correction value at each sampling timing is stored in a storage unit together with time information, and at a predetermined time, a process of transmitting the second shake correction value and the time information stored in the storage unit to the control unit is performed. The control unit performs a process of using the second shake correction value received from the lens barrel as metadata corresponding to a frame of a moving image based on the time information. Also, the control unit performs a process of controlling so that high-frequency component shake correction is performed by the first shake correction function and low-frequency component shake correction is performed by the second shake correction function, and a process of setting the frequency range of the low-frequency component based on the storage capacity of the storage unit. An imaging device.

4. The control unit uses the second shake correction value received from the lens barrel as the correction information. The imaging device according to claim 1 or claim 2.

5. The control unit uses the second shake correction value received from the lens barrel and the second shake correction value generated by interpolation processing using the received second shake correction value as the correction information. The imaging device according to any one of claims 1 to 3.

6. The control unit performs a process of recording, as metadata, communication speed information between the lens barrel and the camera body unit on a recording medium. The imaging device according to any one of claims 1 to 5.

7. A shake correction function for displacing a shake correction lens, a storage unit, a process of detecting a shake correction value related to the shake correction function at each sample timing, a process of storing the detected shake correction value in the storage unit together with time information, and after video imaging is completed, the shake correction value and the time information stored in the storage unit are transmitted to the camera body unit which is the mounting destination, and a control unit for performing the process. A lens barrel device.

8. The control unit performs a process of transmitting the shake correction value and the time information stored in the storage unit at intermittent timings during video imaging. The lens barrel device according to claim 7.

9. The control unit selects whether to transmit the shake correction value detected at each sample timing to the camera body unit or store it in the storage unit, for the shake correction value to be stored in the storage unit, stores it in the storage unit together with the time information, and at a predetermined time, reads it from the storage unit and transmits it to the camera body unit together with the time information. The lens barrel device according to claim 7.

10. An imaging device having a configuration in which a lens barrel is detachable from a camera body unit, a process of generating, as metadata associated with an imaging image, correction information including a first shake correction value related to a first shake correction function for correcting the positional relationship between an optical image incident through a lens and an output imaging image, and a second shake correction value related to a second shake correction function provided in the lens barrel including the lens, a process of controlling such that high-frequency component shake correction is performed by the first shake correction function and low-frequency component shake correction is performed by the second shake correction function based on determination of the communication speed between the mounted lens barrel and the camera body unit. An imaging method.

11. An imaging device, a process of generating correction information including a first blur correction value related to a first blur correction function for correcting a positional relationship between an optical image incident through a lens and an output imaging image, and a second blur correction value related to a second blur correction function provided in a lens barrel including the lens, as metadata associated with the imaging image; a process of controlling so that high-frequency component blur correction is performed by the first blur correction function and low-frequency component blur correction is performed by the second blur correction function; a process of setting a frequency range of a low-frequency component based on a communication speed with the lens barrel side; An imaging method.

12. A control unit of an imaging device performs a process of generating correction information including a first blur correction value related to a first blur correction function for correcting a positional relationship between an optical image incident through a lens and an output imaging image, and a second blur correction value related to a second blur correction function provided in a lens barrel including the lens, as metadata associated with the imaging image. The lens barrel stores the second blur correction value at each sample timing in a storage unit together with time information, and at a predetermined time point, performs a process of transmitting the second blur correction value and the time information stored in the storage unit to the control unit. The control unit performs a process of using, as metadata corresponding to a frame of a moving image, the second blur correction value received from the lens barrel based on the time information. The control unit also performs a process of controlling so that high-frequency component blur correction is performed by the first blur correction function and low-frequency component blur correction is performed by the second blur correction function, and a process of setting a frequency range of a low-frequency component based on a storage capacity of the storage unit. An imaging method.

13. A blur correction function for displacing a blur correction lens; A storage unit; As a transmission method in a lens barrel device including: a process of detecting a blur correction value related to the blur correction function at each sample timing; a process of storing the detected blur correction value in the storage unit together with time information; a process of transmitting the blur correction value and the time information stored in the storage unit to a camera body part which is a mounting destination, at a time point after completion of moving image imaging; A transmission method.

Images