Imaging apparatus and imaging method

Abstract

An imaging device is provided, wherein the device is configured to perform the following processing: generating, as metadata associated with a captured image, correction information of a first shake correction value related to a first shake correction function based on a positional relationship between an optical image incident through a lens and an output captured image, and a second shake correction value related to a second shake correction function provided in a lens barrel including the lens.

Description

TECHNICAL FIELD

[0001] The present technology relates to an imaging device and an imaging method, and particularly relates to processing of blur correction data. BACKGROUND

[0002] For example, as a replaceable lens type camera or the like, an imaging device including a camera body portion (camera body) and a lens barrel and capable of recording a moving image on a recording medium is known.

[0003] Such an imaging device includes a device that performs blur correction by mechanical operation of a blur correction function in the camera body and a device that performs blur correction by mechanical operation of a blur correction function in the lens barrel. Further, an electronic blur correction function that changes a range in which an image signal is read from an imaging element or changes a cropping range of an image in image signal processing according to blur is also known.

[0004] The following Patent Literature 1 discloses a configuration that performs blur correction on each of the lens barrel side and the camera body side.

[0005] LIST OF CITATIONS

[0006] PATENT LITERATURE

[0007] Patent Literature 1: WO 2018 / 025639 SUMMARY

[0008] PROBLEMS TO BE SOLVED BY THE INVENTION

[0009] Recently, there is an environment in which a user can easily perform various image capturing, image adjustment, and the like using a mobile terminal such as a smartphone or a tablet, a camera itself, a personal computer, and the like. In this case, there is a case where an influence of camera shake at the time of imaging is removed with high precision, or, on the contrary, a case where a shake is actively applied to an image to obtain a performance effect.

[0010] Therefore, the present disclosure assumes that a shake of an image is added to or removed from a moving image after imaging, and proposes a technology for saving appropriate information in an imaging device for this purpose.

[0011] SOLUTION TO PROBLEM

[0012] An imaging device according to the present technology includes a control unit that generates correction information based on both a first blur correction value related to a first blur correction function that corrects a positional relationship between an optical image incident through a lens and an output captured image and a second blur correction value related to a second blur correction function provided in a lens barrel including the lens, as metadata associated with a captured image.

[0013] The first blur correction function is a function of correcting the positional relationship between the optical image and the output captured image (the image finally output from the imaging device) on the body side. The second blur correction function is a function provided on the lens barrel side. In a case where each of the lens barrel side and the camera body side has the mechanical or electronic blur correction function in this way, the correction information based on the blur correction values of both the lens barrel side and the camera body part side is associated with the moving image captured as metadata. For example, the metadata is recorded in the recording medium in a state associated with the image data.

[0014] It is conceivable that the control unit generates correction information having a smaller data amount than that obtained in a case where the data amount of the first blur correction value and the data amount of the second blur correction value are added in the above-described imaging device according to the present technology.

[0015] If the first blur correction value and the second blur correction value are both directly set as the correction information, the data amount recorded as metadata increases. Therefore, the information including the first blur correction value and the second blur correction value is obtained with a smaller data amount.

[0016] It is conceivable that the control unit sets one of the first blur correction value and the second blur correction value and conversion information configured to obtain the other blur correction value using the one blur correction value in the above-described imaging device according to the present technology as the correction information.

[0017] One of the first blur correction value and the second blur correction value is recorded, and conversion information corresponding to, for example, a ratio, a difference, or the like between the first blur correction value and the second blur correction value is recorded as the conversion information, so that the first blur correction value and the second blur correction value are obtained from the metadata.

[0018] It is conceivable that, in the above-described imaging device according to the present technology, the control unit sets the first blur correction value and conversion information configured to find the second blur correction value using the first blur correction value as the correction information.

[0019] Since the first blur correction value and the conversion information are recorded, the first blur correction value and the second blur correction value can be obtained from the metadata.

[0020] It is conceivable that the control unit generates correction information including a pair of one of the first blur correction value and the second blur correction value and conversion information configured to find the other blur correction value using the one blur correction value at each predetermined timing.

[0021] For example, in a case where the metadata is recorded in association with the frames of the image, the set of one blur correction value and conversion information is recorded in association with each frame.

[0022] It is conceivable that in the imaging device according to the present technology described above, the control unit generates correction information at each predetermined timing. The correction information includes a blur correction value, one of a first blur correction value and a second blur correction value, and conversion information configured to use the one blur correction value to obtain the other blur correction value at a timing when the ratio between the first blur correction value and the second blur correction value has changed.

[0023] That is, conversion information is only recorded when the ratio between the first fuzz correction value and the second fuzz correction value has changed.

[0024] It is conceivable that in the imaging device according to the present technology described above, the control unit sets the ratio between the first blur correction value and the second blur correction value to a fixed ratio from the start of recording the moving image to the end of recording, and generates correction information including one of the first blur correction value and the second blur correction value at each time interval, as well as conversion information according to the fixed ratio, which is configured to use the one blur correction value to obtain the other blur correction value.

[0025] That is, it is sufficient to record only one transformation information associated with the moving image.

[0026] It is conceivable that the control unit uses the composite correction value of the first blur correction value and the second blur correction value as the correction information in the imaging device according to the present technology.

[0027] The correction value of the entire imaging device is recorded as metadata, which is a composite correction value of the first and second blur correction values.

[0028] It is conceivable that in the imaging device according to the present technology described above, the control unit sets the first blur correction value and the second blur correction value as correction information, and performs the process of recording the communication speed information between the lens barrel and the camera body as metadata in the recording medium.

[0029] Both the first and second blur correction values ​​are directly used as correction information and recorded as metadata. Furthermore, the communication speed information between the lens barrel side and the camera body side is also recorded as metadata.

[0030] It is conceivable that the communication speed information is a value indicating the communication speed in the imaging device according to the present technology.

[0031] The communication speed between the lens barrel and the camera body is directly used as metadata.

[0032] It is conceivable that in the imaging device according to the present technology, the communication speed information is the result information obtained by comparing the communication speed with a predetermined value.

[0033] For example, information indicating whether the communication speed between the lens barrel and the camera body is fast or slow is used as metadata.

[0034] It can be conceived that in the imaging device according to the above-described technology, the blur correction value is a correction execution value that indicates the position or position displacement of the blur correction function.

[0035] The first fuzz correction value or the second fuzz correction value is set to the position or position displacement when the fuzz correction function is actually executed.

[0036] It is conceivable that in the imaging device according to the present technology, the blur correction value is a correction instruction value that gives instructions about the position or position displacement for the blur correction function.

[0037] The instruction specifies the position or displacement amount for fuzz correction used in the first or second fuzz correction function to perform the fuzz correction. The correction instruction value is used as the fuzz correction value to be reflected in the metadata.

[0038] It is conceivable that in the imaging device according to the present technology described above, the control unit uses the information detected by the blur detection unit as metadata.

[0039] For example, in the case of sensors such as gyroscopes, the metadata includes jitter information obtained through sensor sensing.

[0040] The imaging method according to this technology includes the following processing: generating correction information based on both a first blur correction value and a second blur correction value as metadata associated with the captured image, wherein the first blur correction value is related to a first blur correction function that corrects the positional relationship between the optical image incident through the lens and the output captured image, and the second blur correction value is related to a second blur correction function set in a lens barrel including the lens, and the processing is performed by an imaging device.

[0041] Therefore, when capturing an image, the blur correction values ​​on the lens barrel side and the subject side can be referenced from the metadata. Attached Figure Description

[0042] Figure 1 This is an explanatory diagram of an imaging device and an image processing device according to embodiments of the present technology.

[0043] Figure 2 This is an explanatory diagram of the data flow of the imaging device and the image processing device according to an embodiment.

[0044] Figure 3This is a block diagram illustrating an example configuration of an imaging device according to an embodiment.

[0045] Figure 4 This is an explanatory diagram of the correction process of an imaging device according to an embodiment.

[0046] Figure 5 This is an explanatory diagram of the metadata according to the embodiment.

[0047] Figure 6 This is an explanatory diagram of the function of an image processing device according to an embodiment.

[0048] Figure 7 This is a flowchart of a processing example related to fuzz correction according to the first embodiment.

[0049] Figure 8 This is a flowchart of IMU data recording according to an embodiment.

[0050] Figure 9 This is a flowchart of a processing example related to fuzz correction according to the second embodiment.

[0051] Figure 10 This is a flowchart of a processing example related to fuzz correction according to the third embodiment.

[0052] Figure 11 This is a flowchart of a processing example related to fuzz correction according to the fourth embodiment.

[0053] Figure 12 This is a flowchart of a processing example related to fuzz correction according to the fifth embodiment.

[0054] Figure 13 This is a flowchart illustrating an example of processing metadata recorded on the lens side according to the fifth embodiment.

[0055] Figure 14 This is a flowchart of recording communication speed information according to the fifth embodiment.

[0056] Figure 15 This is a flowchart of recording communication speed information according to the sixth embodiment.

[0057] Figure 16 This is a flowchart of the processing on the lens barrel side according to the seventh embodiment.

[0058] Figure 17 This is a flowchart of the processing on the camera body side according to the seventh embodiment.

[0059] Figure 18 This is a flowchart of the lens barrel side processing according to another example of the seventh embodiment. Detailed Implementation

[0060] The embodiments will be described below in the following order.

[0061] <1. Inter-frame jitter correction in imaging and image processing equipment>

[0062] <2. Configuration of Imaging Equipment and Metadata>

[0063] <3. Functions of Image Processing Equipment>

[0064] <4. First Embodiment>

[0065] <5. Second Embodiment>

[0066] <6. Third Embodiment>

[0067] <7. Fourth Embodiment>

[0068] <8. Fifth Embodiment>

[0069] <9. Sixth Embodiment>

[0070] <10. Seventh Embodiment>

[0071] <11. Summary and Revision>

[0072] <1. Inter-frame jitter correction in imaging and image processing equipment>

[0073] Figure 1 An example of an imaging device 1 and an image processing device (5, 6) for acquiring an image file MF captured by the imaging device 1 according to an embodiment is illustrated.

[0074] The diagram illustrates an example in which a mobile terminal 7 and a personal computer 8 are used as image processing devices 5 and 6. Although not shown, various devices (such as dedicated image editing equipment, cloud servers, television equipment, and video recording and playback devices) are assumed to be image processing devices 5 and 6. These devices can be used as either image processing device 5 or 6.

[0075] Image processing device 5 is a device that mainly performs inter-frame jitter modification processing on image data acquired from imaging device 1.

[0076] On the other hand, the image processing device 6 is a device that performs secondary inter-frame jitter modification processing on image data that has already undergone inter-frame jitter modification processing in other image processing devices.

[0077] Note that "jitter" refers to the inter-frame jitter of the image that constitutes a moving image. It is assumed that "jitter" refers to the vibration components generated between frames in an image captured by imaging device 1 (the fluctuation of the image between frames), such as image jitter caused by camera shake and jitter intentionally added by image processing.

[0078] The term "camera shake" is also used to refer to the "shaking" caused by camera shake or the like when imaging is performed by imaging device 1. The correction performed in imaging device 1 to reduce image shaking caused by camera shake or the like (including vibrations applied when imaging device 1 is fixed and not held by hand) is called "image stabilization" to distinguish it from the "inter-frame shake correction" processing in image processing devices 5 and 6.

[0079] "Inter-frame jitter modification" refers to changing the state of jitter in an image, such as reducing jitter in the image or adding jitter to the image.

[0080] Assume that "inter-frame jitter modification" includes "inter-frame jitter reduction" and "inter-frame jitter generation".

[0081] "Inter-frame jitter reduction" refers to eliminating (completely removing jitter) or reducing (partially removing jitter) the jitter that occurs in the image caused by camera shake, etc., during imaging in image processing devices 5 and 6.

[0082] "Inter-frame jitter generation" refers to changing the jitter state of an image in image processing devices 5 and 6. There are cases where reducing jitter results in this inter-frame jitter generation, and in this sense, this is similar to "inter-frame jitter reduction." However, in this embodiment, "inter-frame jitter generation" refers to changing the jitter state of an image according to an instruction, provided by user operation or automatic control regarding the amount of jitter change.

[0083] For example, based on user instructions, the jitter generated during imaging can be reduced or increased, or new jitter corresponding to "inter-frame jitter generation" can be added.

[0084] Note that, as an example of the purpose of inter-frame jitter, it is assumed that the image is deliberately jittered in order to leave an impression of a moving scene.

[0085] Figure 1 The imaging device 1 is configured as a so-called digital still camera or digital video camera to be able to capture at least moving images.

[0086] The main body of the camera in imaging device 1 is shown as camera body 2.

[0087] The lens barrel 3 serves as a so-called replaceable lens and can be attached to and detached from the camera body (camera body 2) of the imaging device 1. The user can replace and use the lens barrel 3 as needed.

[0088] Note that this replaceable lens type imaging device 1 is assumed in the embodiment, but the technology of this disclosure can also be applied to types in which the lens barrel 3 cannot be removed from the camera body 2.

[0089] Imaging device 1 can capture moving images and transmit the resulting image file MF via wired or wireless communication to a mobile terminal 7, personal computer 8, etc., which serves as image processing device 5. Alternatively, imaging device 1 can record the image file MF on a recording medium such as a memory card, and the mobile terminal 7 or personal computer 8 can read the image file MF from the memory card.

[0090] Furthermore, the image file MF includes not only image data as a moving image, but also metadata as additional information corresponding to the image data.

[0091] Figure 2 The diagram illustrates the information transmission status in imaging device 1, image processing device 5, and image processing device 6.

[0092] Image data VD1 and metadata MTD1 are transmitted from imaging device 1 to image processing device 5 via wired communication, wireless communication, or recording medium.

[0093] Image data VD1 and metadata MTD1 are, for example, information transmitted as image file MF.

[0094] In this embodiment, the metadata MTD1 includes, for example, information about image stabilization during imaging.

[0095] The image processing device 5 can receive image data VD1 and metadata MTD1 and perform various processes.

[0096] For example, the image processing device 5 can use the image stabilization information included in the metadata MTD1 to perform inter-frame jitter modification processing on the image data VD1.

[0097] As described above, inter-frame jitter modification can be used to cancel image stabilization to return to the original image with jitter, perform more advanced inter-frame jitter reduction, or add jitter to the image for use in the resulting processing.

[0098] Image processing device 5 can also transmit image data VD2 and metadata MTD2 obtained by performing inter-frame jitter modification processing to other image processing devices 6.

[0099] In this case, information related to inter-frame jitter modification processing is added as metadata MTD2, so that the image processing device 6 can also perform various inter-frame jitter modifications.

[0100] In this embodiment, based on this assumption of information transmission, at least appropriate inter-frame jitter modification in the image processing device 5 can be performed. For this purpose, a description focusing on the metadata recording during imaging in the imaging device 1 will be given.

[0101] <2. Configuration of Imaging Equipment and Metadata>

[0102] Figure 3 The illustration shows an example configuration of imaging device 1 and lens barrel 3.

[0103] A lens system 10 having multiple optical components is formed in the lens barrel 3. For example, the lens system 10 includes a zoom lens 10a, an aperture mechanism 10b, an image stabilizing lens mechanism 10c, a focusing lens 10d, etc.

[0104] The image stabilization lens mechanism 10c is a mechanism that reduces the shaking that occurs in the image by mechanically driving the lens relative to camera shake.

[0105] Light from the subject (incident light) is collected on the imaging element unit 12 via the lens system 10 and the shutter 11 in the camera body 2.

[0106] The imaging element unit 12 includes, for example, an image sensor (imaging element) of the complementary metal-oxide-semiconductor (CMOS) type or the charge-coupled device (CCD) type.

[0107] The imaging element unit 12 performs processes such as correlated double sampling (CDS) and automatic gain control (AGC) on the electrical signal obtained from the light received by the image sensor through photoelectric conversion, and further performs analog-to-digital (A / D) conversion. Then, the imaging signal, as digital data, is output to the camera signal processing unit 13 in the subsequent stage.

[0108] The imaging element unit 12 is provided with an imaging plane image stabilization unit 30.

[0109] The imaging plane image stabilization unit 30 is a mechanism that corrects image jitter by mechanically moving the image sensor relative to camera shake, etc.

[0110] The camera signal processing unit 13 is configured to use an image processing processor, such as a digital signal processor (DSP). The camera signal processing unit 13 performs various types of signal processing on the digital signals (captured image signals) from the imaging element unit 12. For example, as part of the camera process, the camera signal processing unit 13 performs preprocessing, synchronization processing, YC generation processing, various types of correction processing, resolution conversion processing, codec processing, etc.

[0111] In the preprocessing, clamping processing, such as clamping the black levels of R, G and B to predetermined levels, and correction processing between the color channels of R, G and B are performed on the image signal captured from the imaging element unit 12.

[0112] During synchronization processing, color separation is performed so that the image data for each pixel has all the color components of R, G, and B. For example, in the case of an imaging element using a Bayer array color filter, demosaicing is performed as a color separation process.

[0113] In the YC generation process, luminance (Y) and color (C) signals are generated (separated) from the image data of R, G, and B.

[0114] In resolution conversion processing, resolution conversion is performed on image data that has undergone various types of signal processing.

[0115] Figure 4 Various types of correction processing performed by the lens system 10 on the camera signal processing unit 13 are illustrated. Figure 4 The diagram illustrates, in the order of execution, the optical image stabilization performed by the image stabilization lens mechanism 10c and the imaging plane image stabilization unit 30, and the correction processing performed by the camera signal processing unit 13.

[0116] As optical image stabilization in process F1, the lens vibration suppression of the image stabilization lens mechanism 10c and the main body vibration suppression of the imaging plane image stabilization unit 30 are performed.

[0117] For example, the imaging plane image stabilization unit 30 performs image stabilization by shifting the image stabilizing lens mechanism 10c in the yaw and pitch directions to suppress lens vibration, and performs image stabilization by shifting the image sensor in the yaw and pitch directions to suppress subject vibration, so that an image of the subject is formed on the image sensor in a state where the effect of camera shake has been physically eliminated.

[0118] In some cases, only one of lens vibration suppression and body vibration suppression is performed, while in other cases both are performed.

[0119] It should be noted that, in addition to the optical image stabilization mentioned above, electronic image stabilization can also be performed as an image stabilization method.

[0120] In the camera signal processing unit 13, the processing from process F2 to process F6 is performed by transforming the spatial coordinates of each pixel.

[0121] In process F2, lens distortion correction is performed.

[0122] In process F3, focal plane distortion correction, as an element of electronic image stabilization, is performed. Note that this distortion correction is performed, for example, when readouts are performed by a CMOS image sensor using a rolling shutter scheme.

[0123] In process F4, rolling correction is performed. That is, rolling component correction, which is an element of electrical image stabilization, is performed.

[0124] In process F5, trapezoidal distortion correction is performed on the amount of trapezoidal distortion caused by electrical image stabilization. The amount of trapezoidal distortion caused by electrical image stabilization is perspective distortion caused by cutting off areas far from the image center.

[0125] In process F6, shifting or cropping in the pitch and yaw directions is performed as an element of electronic image stabilization.

[0126] For example, image stabilization, lens distortion correction, and trapezoidal distortion correction are performed during the above process.

[0127] Note that it is not necessary to execute all the procedures listed here, and the order of the procedures can be changed as appropriate.

[0128] exist Figure 3 In the codec processing of the camera signal processing unit 13, for example, encoding processing and file generation are performed on image data that has undergone the various types of processing described above for recording and communication. For example, an image file MF is generated in MP4 format for recording moving images and audio conforming to MPEG-4. Furthermore, it is conceivable to generate still image files from files in formats such as Joint Photo Experts Group (JPEG), Tagged Image File Format (TIFF), or Graphics Interchange Format (GIF).

[0129] Note that, Figure 3 The audio processing system is not shown, but an audio recording system and an audio processing system are actually set up, and the image file MF can include audio data as well as image data as moving images.

[0130] The camera control unit 18 is configured using a microcomputer (arithmetic processing device) including a central processing unit (CPU).

[0131] The memory unit 19 stores information for processing by the camera control unit 18. The memory unit 19 shown in the figure generally represents, for example, read-only memory (ROM), random access memory (RAM), flash memory, etc.

[0132] The RAM in memory unit 19 is used as a work area to temporarily store data, programs, etc. during various types of data processing by the CPU of camera control unit 18.

[0133] The ROM and flash memory (non-volatile memory) in memory unit 19 are used to store the operating system (OS) configured by the CPU to control the units, content files such as image files, applications for various operations, firmware, etc.

[0134] The memory unit 19 may be a memory region built into the microcomputer chip used as the camera control unit 18, or it may be configured using a separate memory chip.

[0135] The camera control unit 18 executes programs stored in the ROM, flash memory, etc. of the memory unit 19 to control the entire imaging device 1 and the lens barrel 3.

[0136] For example, the camera control unit 18 controls the shutter speed of the imaging element unit 12, the camera signal processing unit 13 provides various signal processing instructions, the camera performs imaging or recording operations based on the user's operation, the recording of image files is reproduced, the lens system 10 operates (such as zooming, focusing, and aperture adjustment in the lens barrel 3), and the user interface operates, etc.

[0137] In addition, the camera control unit 18 performs various types of processing and output control on the image data processed by the camera signal processing unit 13.

[0138] The camera control unit 18 enables the electronic image stabilization control unit 35 to perform electronic image stabilization processing on the image data.

[0139] Furthermore, the camera control unit 18 causes the blur correction metadata processing unit 36 ​​to perform metadata generation related to blur correction. The camera control unit 18 also performs control to generate metadata consisting of various types of information, including information about blur correction, and records the metadata as information related to the image file MF.

[0140] In addition, the camera control unit 18 communicates with the lens control unit 20 on the lens barrel 3 side via the communication control unit 33.

[0141] Note that, Figure 3 The electronic image stabilization control unit 35 and the blur correction metadata processing unit 36 ​​are illustrated as separate boxes from the camera control unit 18, but these can be considered as functions implemented by the microcomputer constituting the camera control unit 18. Therefore, for ease of description, the camera control unit 18, the electronic image stabilization control unit 35, and the blur correction metadata processing unit 36 ​​are collectively referred to as "control unit 40". These can also be configured using separate arithmetic processing devices.

[0142] For example, the recording control unit 14 performs recording and playback on a recording medium configured with non-volatile memory. For example, the recording control unit 14 performs the process of recording image files MF, such as moving image data or still image data, thumbnail images, etc., onto the recording medium.

[0143] The actual form of the recording control unit 14 can be considered in different ways. For example, the recording control unit 14 can be configured as a flash memory and its write / read circuitry built into the imaging device 1, or it can be provided as a card recording and playback unit that accesses and is detached from the imaging device 1, such as a memory card (portable flash memory, etc.) for recording and playback. Furthermore, the recording control unit 14 can be implemented as a hard disk drive (HDD) or the like, built into the imaging device 1.

[0144] Display unit 15 is a display unit that performs various displays for the user, and is, for example, a display panel or viewfinder of a display device such as a liquid crystal display (LCD) or an organic electroluminescent (EL) display arranged in the housing of imaging device 1.

[0145] The display unit 15 performs various displays on the display screen based on instructions from the camera control unit 18.

[0146] For example, display unit 15 displays a reproduced image of image data read from the recording medium in recording control unit 14.

[0147] Furthermore, there exists a situation where image data of a captured image, whose resolution has been converted by the camera signal processing unit 13 for display, is supplied to the display unit 15, and the display unit 15 performs display based on the image data of the captured image in response to instructions from the camera control unit 18. Therefore, a so-called through image (subject monitoring image) is displayed as an image captured during composition confirmation.

[0148] In addition, the display unit 15 performs various operation menus, icons, messages, etc. (i.e., graphical user interface (GUI)) display on the screen based on instructions from the camera control unit 18.

[0149] Output unit 16 performs data communication and network communication with external devices via wired or wireless means.

[0150] For example, captured image data (still image files or moving image files) is transmitted and output to external display devices, recording devices, playback devices, etc.

[0151] In addition, the output unit 16 can perform communication via various networks (such as the Internet, home networks and local area networks (LANs)) to transmit and receive various types of data to and from servers, terminals and the like on the network.

[0152] The operation unit 17 collectively represents an input device configured to allow the user to perform various operation inputs. Specifically, the operation unit 17 represents various operating elements (keys, dials, touch panels, touch pads, etc.) disposed in the housing of the imaging device 1.

[0153] The operation unit 17 detects the user's operation and transmits the signal corresponding to the input operation to the camera control unit 18.

[0154] The shutter drive unit 31 drives the shutter 11 based on instructions from the camera control unit 18.

[0155] The correction unit drive unit 32 drives the imaging plane image stabilization unit 30 based on instructions from the camera control unit 18, and shifts the image sensor in the imaging element unit 12 for optical image stabilization.

[0156] The blur detection unit 34 represents a sensor that detects the shaking applied to the camera body 2. As the blur detection unit 34, for example, an inertial measurement unit (IMU) is installed, and the angular velocity can be detected by, for example, angular velocity (gyroscope) sensors of the pitch, yaw, and roll axes, and the acceleration can be detected by an accelerometer sensor.

[0157] Note that the blur detection unit 34 only needs to include a sensor that can detect camera shake during imaging, and does not need to include both a gyroscope sensor and an accelerometer sensor.

[0158] The lens barrel 3 is equipped with a lens control unit 20, for example, configured using a microcomputer.

[0159] With the lens barrel 3 mounted on the camera body 2, the camera control unit 18 and the lens control unit 20 can communicate with each other via the communication control units 27 and 33.

[0160] Communication control units 27 and 33 are connected by a wire to perform communication when the lens barrel 3 is mounted on the camera body 2. However, both can be configured to perform wireless communication.

[0161] The lens control unit 20 and the camera control unit 18 periodically perform bidirectional data communication at a certain communication speed.

[0162] For example, the camera control unit 18 instructs the lens control unit 20 to drive the zoom lens 10a, the focusing lens 10d, the aperture mechanism 10b, and the image stabilizing lens mechanism 10c. The lens control unit 20 causes the operation of the lens system 10 to be executed in response to instructions for driving them.

[0163] In addition, the lens control unit 20 transmits lens distortion correction information, focal length information, and the position of the correction lens of the image stabilization lens mechanism 10c to the camera control unit 18.

[0164] The lens barrel 3 is provided, for example, with a zoom drive unit 21 including a motor and a motor driver for driving the zoom lens 10a, an aperture drive unit 22 including a motor and a motor driver for driving the aperture mechanism 10b, a correction lens drive unit 23 including a motor and a motor driver for driving the image stabilizing lens mechanism 10c, and a focus drive unit 24 including a motor and a motor driver for driving the focus lens 10d.

[0165] The zoom drive unit 21, aperture drive unit 22, correction lens drive unit 23, and focus drive unit 24 apply drive current to their respective motors 18 in response to commands from the lens control unit 20, based on commands from the camera control unit 18. Thus, zoom operation, aperture opening / closing operation, optical image stabilization operation, and focusing operation are performed.

[0166] Memory unit 26 stores information for processing by lens control unit 20. Memory unit 26 can be represented by, for example, ROM, RAM, flash memory, etc. Memory unit 26 is sometimes used to temporarily store information transmitted from lens control unit 20 to camera control unit 18.

[0167] Blur detection unit 25 represents a sensor that detects shake in the lens barrel 3, and is similar to blur detection unit 34 on the side of camera body 2, for example, it is assumed that an IMU is installed. Note that it is also assumed that the shake detection unit 34 is not installed in the lens barrel 3.

[0168] Next, the contents of the image file MF and its metadata, which are transmitted from the imaging device 1 to the image processing device 5, will be described.

[0169] Figure 5 Figure A illustrates the data included in an image file MF. As shown, the image file MF includes various types of data as "title," "sound," "movie," and "metadata."

[0170] The “Title” will describe information indicating the presence or absence of metadata, along with information such as filename and file size.

[0171] "Sound" is audio data recorded along with moving images. For example, storing two-channel stereo audio data.

[0172] "Movie" is moving image data and includes multiple image data as frames (#1, #2, #3, etc.) that make up the moving image.

[0173] As “metadata,” it describes additional information associated with the corresponding frames (#1, #2, #3, etc.) that constitute the moving image.

[0174] Examples of metadata content are in Figure 5As shown in B. For example, a frame describes IMU data, coordinate transformation parameters HP, timing information TM, and camera parameters CP. Note that these are part of the metadata content, and other information may also exist. Furthermore, there are cases where no information shown in the figure is included.

[0175] also, Figure 5 B illustrates the case where the metadata includes communication speed information, which is the case where processing according to the fifth or sixth embodiment described later is performed.

[0176] As IMU data, it describes the gyroscope (angular velocity data), accelerometer (accelerometer data), and sampling rate.

[0177] The IMUs, which are installed on the imaging device 1 as blur detection units 34 and 25, output angular velocity and acceleration data at a predetermined sampling rate. Generally, this sampling rate is higher than the frame rate of the captured image, thus obtaining many IMU data samples within one frame period.

[0178] Therefore, as angular velocity data, n samples are associated with one frame, such as Figure 5 The gyroscope samples shown in C are #1, #2, ... and #n.

[0179] In addition, as acceleration data, m samples are associated with a frame, such as accelerator sample #1, accelerator sample #2, ... and accelerator sample #m.

[0180] There are cases where n = m, and there are also cases where n ≠ m.

[0181] Note that this paper has described an example of metadata associated with each frame, but there are cases where IMU data is not perfectly synchronized with the frames. In such cases, for example, the time information associated with the time information of each frame is stored as an IMU sample timing offset in the timing information TM.

[0182] The coordinate transformation parameter HP is a collective term for parameters used to correct the coordinate transformations of each pixel in an image. It also includes, for example, nonlinear coordinate transformations such as those for lens distortion.

[0183] Therefore, the coordinate transformation parameter HP can include at least the lens distortion correction parameter, trapezoidal distortion correction parameter, focal plane distortion correction parameter, electrical image stabilization parameter, and optical image stabilization parameter.

[0184] Lens distortion correction parameters are used to directly or indirectly determine how distortions such as barrel aberration and pincushion aberration are corrected and restored to the image before lens distortion correction.

[0185] The trapezoidal distortion correction parameter is the correction amount when correcting trapezoidal distortion caused by the shift of the cropped region from the center through electrical image stabilization, and also has a value based on the correction amount of electrical image stabilization.

[0186] The focal plane distortion correction parameter is a value that indicates the amount of correction for each line with respect to focal plane distortion.

[0187] For electrical and optical image stabilization, parameters are used that indicate the correction amounts for the corresponding axial directions of yaw, pitch, and roll.

[0188] Here, in this embodiment, as Figure 3 As shown, an image stabilization lens mechanism 10c and an imaging plane image stabilization unit 30 are provided as optical image stabilization mechanisms. Therefore, correction information indicating the correction amount for optical image stabilization, for example, such as... Figure 5 As shown in D, record the blur correction values ​​on the subject side and the blur correction values ​​on the lens side.

[0189] The subject-side blur correction value is the blur correction value in the imaging plane image stabilization unit 30.

[0190] The lens-side blur correction value is the blur correction value in the image stabilizing lens mechanism 10c.

[0191] Assume these blur correction values ​​are set as the actual correction execution values ​​for the imaging plane image stabilization unit 30 and the image stabilization lens mechanism 10c. This effective correction value represents the actual displacement performed as optical image stabilization, such as the position information changed by the actual correction or the displacement of the position information detected by the position sensor provided in the imaging plane image stabilization unit 30 or the image stabilization lens mechanism 10c relative to the previous frame.

[0192] Alternatively, these blur correction values ​​can also be correction instruction values ​​output from the camera control unit 18 to the correction unit drive unit 32, and correction command values ​​transmitted from the camera control unit 18 to the correction lens drive unit 23 via the lens control unit 20. This is because the imaging plane image stabilization unit 30 and the image stabilization lens mechanism 10c are driven to have positions and displacements corresponding to these correction command values.

[0193] Furthermore, as correction information regarding optical image stabilization, there are examples such as... Figure 5 E shows the recorded subject-side blur correction values ​​and conversion information. This will be described in the first to third embodiments.

[0194] In addition, as correction information related to optical image stabilization, there exist such as Figure 5The example shown in F combines both the subject-side blur correction value and the lens-side blur correction value, and is recorded as a composite correction value added to the entire imaging device 1 as a blur correction amount. This will be described in the fourth embodiment.

[0195] Note that the corresponding parameters for lens distortion correction, trapezoidal distortion correction, focal plane distortion correction, and electronic image stabilization are collectively referred to as coordinate transformation parameters, because this correction process is a correction process for the image formed on each pixel of the image sensor of the imaging element unit 12, and the parameters are the parameters of the correction process accompanying the coordinate transformation of each pixel.

[0196] Furthermore, since the jitter correction of inter-frame components is a process that accompanies the coordinate transformation of each pixel in optical image stabilization, the correction information of optical image stabilization is also set as one of the coordinate transformation parameters for description.

[0197] That is, when inverse correction is performed using these parameters, the image data that has undergone lens distortion correction, trapezoidal distortion correction, focal plane distortion correction, electrical image stabilization, and optical image stabilization can be returned to the state before each correction process, that is, the state when the image is formed on the image sensor of the imaging element unit 12.

[0198] Furthermore, the corresponding parameters for lens distortion correction, trapezoidal distortion correction, and focal plane distortion correction are collectively referred to as optical distortion correction parameters, because this correction process is for the case where the optical image from the subject itself is captured in an optically distorted state and is intended to correct the optical distortion.

[0199] That is, when inverse correction is performed using these parameters, the image data after lens distortion correction, trapezoidal distortion correction and focal plane distortion correction can be returned to the state before optical distortion correction.

[0200] The timing information TM in the metadata includes information on each of the following: exposure time (shutter speed), exposure start timing, readout time (curtain speed), number of exposure frames (long second exposure information), IMU sample offset, and frame rate.

[0201] These are primarily used to associate one line of each frame with IMU data.

[0202] However, even if the image sensor 12a is a CCD or CMOS with a global shutter scheme, if the exposure center of gravity is shifted by using an electronic shutter or a mechanical shutter, it is possible to perform correction based on the exposure center of gravity using the exposure start timing and curtain speed.

[0203] The camera parameter CP, as part of the metadata, describes the field of view (focal length), zoom position, and lens distortion information.

[0204] <3. Functions of Image Processing Equipment>

[0205] The image processing device 5 can perform inter-frame jitter modification processing on the image file MF generated by imaging in the imaging device 1.

[0206] The image processing device 5 for this purpose is configured with the following functions: Figure 6 As shown in the figure. Note that it is assumed that image processing device 6 has a similar configuration.

[0207] It is conceivable that image processing device 5 is such as Figure 1 Information processing devices such as mobile terminals 7 or personal computers 8, and for example Figure 6 The functionality shown is formed by the application in this case.

[0208] For example, the image processing device 5 has the functions of a correction cancellation unit 51, an inter-frame jitter generation unit 52, a processing setting unit 53, and a user interface unit 54. Note that the "user interface" is also called the "UI", and the user interface unit 54 is referred to as the "UI unit 54" below.

[0209] The correction cancellation unit 51 and the inter-frame jitter generation unit 52 are functions that perform some inter-frame jitter modifications on the image.

[0210] The correction cancellation unit 51 is a function that performs inter-frame jitter modification to cancel the optical image stabilization and electrical image stabilization performed by the imaging device 1 and return to the state where image stabilization was not performed (i.e., the state where the image is jittery due to the original camera shake).

[0211] Alternatively, the correction cancellation unit 51 may also perform a process that cancels only the correction performed by the image stabilization lens mechanism 10c, a process that cancels only the correction performed by the imaging plane image stabilization unit 30, or a process that cancels only the electronic image stabilization.

[0212] The inter-frame jitter generation unit 52 is a function that performs the process of changing the jitter state of image data according to the parameters or instructions input by the user.

[0213] The inter-frame jitter generation unit 52 can perform inter-frame jitter generation processing (such as adding or removing jitter) on image data VD1 that has not undergone the cancellation processing in the correction cancellation unit 51 or on image data VD1 after the cancellation processing in the correction cancellation unit 51.

[0214] As part of the inter-frame jitter generation process, it is assumed that the jitter of the image is reduced, or that inter-frame jitter reduction is performed with higher accuracy than the image stabilization of imaging device 1, or that jitter is applied to the image.

[0215] UI unit 54 is a function that performs the process of presenting operation elements related to correction cancellation or inter-frame jitter modification to the user and obtaining operation information of such operation elements.

[0216] The processing setting unit 53 sets processing parameters for correction cancellation based on the metadata MTD1, and causes the correction cancellation unit 51 to perform processing. Furthermore, the processing setting unit 53 sets processing parameters for inter-frame jitter modification based on user operations detected by the UI unit 54 or the metadata MTD1, and causes the inter-frame jitter generation unit 52 to perform processing.

[0217] Using this image processing device 5, the user can perform desired inter-frame jitter modifications on the image file MF obtained by the imaging device 1.

[0218] In particular, since the metadata MTD1 includes information on optical image stabilization, it is possible to identify corrections performed by the image stabilization lens mechanism 10c and corrections performed by the imaging plane image stabilization unit 30, and these corrections can be performed based on the processing of both. For example, it becomes possible to cancel image stabilization, add appropriate jitter, etc.

[0219] <4. First Embodiment>

[0220] In the following sections, various examples will be described as processes according to embodiments, with particular focus on optical image stabilization and the recording of metadata thereunder.

[0221] For example, such as Figure 5 As shown in D, the imaging device 1 uses the subject-side blur correction value and the lens-side blur correction value as metadata and associates the metadata with the image data as time-series information, so that inter-frame jitter modification (correction cancellation or inter-frame jitter generation) using the subject-side blur correction value and the lens-side blur correction value can be performed in the image processing device 5.

[0222] However, as Figure 5 As shown in D, if the subject-side blur correction value and the lens-side blur correction value are recorded as metadata for each frame, then there is an increase in the amount of metadata MTD1, which may be undesirable in some cases. For example, when the subject-side blur correction value and the lens-side blur correction value are recorded as metadata for each frame of a moving image captured over a relatively long period of time, the increase in the amount of data in the image file MF becomes significant.

[0223] Therefore, in the first embodiment, the amount of data in the metadata MTD1 can be reduced, while the metadata MTD1 can be used to detect, for example, subject-side blur correction values ​​and lens-side blur correction values ​​in the image processing device 5.

[0224] Figure 7The illustration shows a processing example of the control unit 40 (camera control unit 18, etc.) in the camera body 2. Figure 7 This is an example of processing related to blur correction performed by the control unit 40 for each timing of each frame when recording moving images.

[0225] In step S101, the control unit 40 detects the amount of blur. For example, it detects the amount of jitter generated between the timing of the previous frame and the current frame from the IMU data of the blur detection unit 34. In this case, the IMU data of the blur detection unit 25 in the lens barrel 3 can be referenced.

[0226] In step S102, the control unit 40 calculates the subject-side blur correction amount. The subject-side blur correction amount is the correction amount performed by the imaging plane image stabilization unit 30.

[0227] Furthermore, in step S103, the control unit 40 calculates the lens-side blur correction amount. The lens-side blur correction amount is the correction amount performed by the image stabilizing lens mechanism 10c.

[0228] For example, the control unit 40 calculates how much the imaging plane image stabilization unit 30 and the image stabilization lens mechanism 10c should be operated to perform blur correction based on the amount of blur applied to the imaging device 1 detected in step S101, and sets the blur correction amount for both.

[0229] In this case, the control unit 40 calculates the blur correction amount of each of the imaging plane image stabilization unit 30 and the image stabilization lens mechanism 10c based on the blur magnitude, the frequency component of the blur (vibration), the movable range of the imaging plane image stabilization unit 30, and the type and performance of the mounted lens barrel 3.

[0230] It should be noted that one reason for considering the type and performance of the lens barrel 3 in this case is to assume that a replaceable lens is used as the lens barrel 3. This is because the movable range of the image stabilizing lens mechanism 10c may sometimes vary depending on the lens barrel 3, and there may be lens barrels that do not originally include the image stabilizing lens mechanism 10c.

[0231] Then, when the overall blur correction amount for blur is set to "1", the control unit 40 obtains the corresponding blur correction amount by setting the blur correction amount of the imaging plane image stabilization unit 30 to "α" and the blur correction amount of the image stabilization lens mechanism 10c to "1-α" (where 0≤α≤1).

[0232] In step S104, the control unit 40 transmits the correction command value corresponding to the fuzzy correction amount.

[0233] That is, a correction instruction value that gives instructions on the position and position displacement for performing the fuzz correction amount "α" is transmitted to the correction unit drive unit 32, and a correction instruction value that gives instructions on the position and position displacement for performing the fuzz correction amount "1-α" is transmitted to the lens control unit 20.

[0234] Therefore, the correction unit driving unit 32 executes the driving of the imaging plane image stabilization unit 30 corresponding to the blur correction amount "α".

[0235] Furthermore, the lens control unit 20 transmits the correction command value to the correction lens drive unit 23, and the correction lens drive unit 23 executes the drive of the image stabilization lens mechanism 10c corresponding to the blur correction amount "1-α".

[0236] Therefore, optical image stabilization is performed on both the lens barrel 3 side and the camera body 2 side, thereby reducing image jitter caused by camera shake applied to the imaging device 1.

[0237] In step S105, the control unit 40 calculates the ratio. This ratio is information indicating the ratio between the subject-side correction value and the lens-side correction value. That is, this ratio is information indicating the ratio between the blur correction amount "α" and the blur correction amount "1-α".

[0238] Specific information about the ratio can be information indicating the ratio itself or information from which the ratio can be obtained. In this embodiment, such ratio information is set as conversion information.

[0239] The conversion information is information that enables a blur correction value to be obtained from another blur correction value, and is, for example, information used to obtain a lens-side blur correction value from a subject-side blur correction value.

[0240] The ratio set as conversion information can be in the form of "α:(1-α)", or it can be a value of "α" or "1-α". Alternatively, the conversion information can be information indicating "α" or "1-α" as a percentage.

[0241] Furthermore, considering that the conversion information is, for example, a value that enables the lens-side blur correction value to be obtained from the subject-side blur correction value, it can be used as a division value of (lens-side blur correction value) / (subject-side blur correction value) or the difference between the lens-side blur correction value and the subject-side blur correction value.

[0242] In step S106, the control unit 40 performs the process of generating metadata about optical image stabilization and recording the metadata as information corresponding to the current frame.

[0243] That is, metadata, including a set of subject-side blur correction values ​​and transformation information, is generated and recorded in association with the current frame, such as... Figure 5 As shown in E.

[0244] The subject-side blur correction value is the correction execution value or correction command value as described above. When the subject-side blur correction value is the correction execution value, the control unit 40 detects the actual position information of the image sensor through the position sensor of the imaging plane image stabilization unit 30, and sets the position information itself or the timing displacement of the position information relative to the previous frame as the subject-side blur correction value.

[0245] When the correction command value is set as the subject-side blur correction value, the control unit 40 only needs to directly use the correction command value that gives the command to the correction unit drive unit 32 in step S104 as the subject-side blur correction value.

[0246] Note that here the set of subject-side blur correction values ​​and transformation information is used as metadata, but it is also conceivable to use the set of lens-side blur correction values ​​and transformation information as metadata. Of course, in this case, the transformation information is the information that enables the use of lens-side blur correction values ​​to obtain subject-side blur correction values.

[0247] As described above, if the subject-side blur correction value and conversion information are set as metadata as time-series information for each frame, then the lens-side blur correction value can also be calculated based on the subject-side blur correction value.

[0248] Then, it is not necessary to record the lens-side blur correction value as metadata, which helps to reduce the recording capacity of metadata.

[0249] For example, assume that the subject-side blur correction value and the lens-side blur correction value are used in 4 bytes in each of the pitch, yaw, and roll directions. When... Figure 5 As shown in D, a total of 24 bytes are recorded when recording the blur correction values ​​on the subject side and the lens side.

[0250] On the other hand, the transformation information (e.g., the value of "α" mentioned above) can similarly be 4 bytes, but it can also be fewer bytes. Even using 4 bytes, in Figure 5 In case E, the total number of bytes is 16, and the amount of data can be reduced. This is especially effective when recording moving images for extended periods.

[0251] Furthermore, recording the subject-side blur correction value and conversion information is also applicable when the data communication speed from the lens control unit 20 to the camera control unit 18 is slow. That is, even if the actual lens-side blur correction value (correction execution value) cannot be obtained from the lens control unit 20 in a timely manner, metadata is recorded, and the lens-side blur correction value can be estimated based on the conversion information.

[0252] Because the metadata is recorded as described above, for example, the image processing device 5 can offset the amount of blur correction performed on the image information of each frame, that is, the amount of correction indicated by the subject-side blur correction value and the amount of correction indicated by the lens-side blur correction value obtained from the conversion information. Furthermore, even if distortion correction is performed through image processing, the information of the metadata MTD1 is used to create a state in which distortion correction is not performed, and an image in which neither blur correction nor distortion correction is performed can be created.

[0253] Then, when using such Figure 5 When recording the IMU data as shown in C, the blur amount of the camera body 2 can be recalculated based on the information from the gyroscope sensor and the accelerometer sensor. Furthermore, blur and distortion correction can be performed again on images in the aforementioned state where blur and distortion correction were not performed. That is, blur correction can also be performed retrospectively on previously captured motion image data.

[0254] Note that, Figure 8 The diagram illustrates the processing of IMU data performed by the control unit 40.

[0255] The control unit 40 acquires the IMU data detected by the fuzzy detection unit 34 in step S161 at each predetermined sampling time, and performs processing to record the IMU data as metadata in step S162. Figure 5 As described in C, since IMU data is not necessarily synchronized with the frame timing of the moving image, for example, the execution control causes multiple IMU data to be recorded as metadata for each frame.

[0256] Therefore, each time series IMU data is associated with a frame of the moving image.

[0257] It is assumed that this recording of IMU data as metadata is also performed in each of the embodiments described below.

[0258] <5. Second Embodiment>

[0259] A processing example of the control unit 40 in the second embodiment is shown below. Figure 9 As shown in the diagram. Note that the above process is represented by the same step numbers, and detailed repetition is avoided.

[0260] Figure 9The processing example in the example is to keep the ratio between the subject-side blur correction amount and the lens-side blur correction amount constant over a certain period of time.

[0261] Control unit 40 executes at a time corresponding to each frame of the moving image. Figure 9 The processing.

[0262] In step S110, the control unit 40 branches the processing based on whether the timing is a change in ratio.

[0263] For example, the timing for changing the ratio can be set as follows.

[0264] • Start time of moving image recording

[0265] • Timing of detecting scene changes during moving image recording

[0266] • Timed pause of recording during motion image recording

[0267] • Timing of zoom movement during image recording

[0268] For example, at the point when image recording begins, the ratio between the subject-side blur correction and the lens-side blur correction is first set. For instance, a predetermined initial value can be set, or a ratio determined based on the type or model of the lens barrel 3 can be used.

[0269] After the start of moving image recording, for example, the time points at which the possibility of scene changes due to changes in the person or object identified as the subject, changes in average brightness, changes in the average blur amount detected by the blur detection unit 34, etc., are analyzed are used as the timing for changing the ratio setting. This is to reset to a more suitable ratio according to changes in the imaging situation.

[0270] Similarly, recording pauses during moving image recording are also used as timing for adjusting the rate setting based on the possibility of scene changes.

[0271] Furthermore, when the zoom lens 10a is operated and the angle of view is changed during image recording, the degree of influence of camera shake on the image changes, so this is also used as the timing for changing the ratio setting.

[0272] The above is an example, and other timing scenarios where the ratio setting needs to be changed can be considered. If it is determined in step S110 that the timing is for changing this ratio, the control unit 40 proceeds to step S111 and sets the ratio between the subject-side blur correction amount and the lens-side blur correction amount. That is, the setting of the ratio between the blur correction amounts is updated.

[0273] Then, in step S112, the control unit 40 performs the process of generating conversion information corresponding to the newly set ratio and recording the conversion information in association with the current frame.

[0274] If it is determined in step S110 that the timing is not for changing the ratio, steps S111 and S112 are not executed.

[0275] In step S101, the control unit 40 detects the ambiguity.

[0276] In step S102A, the control unit 40 calculates the subject-side blur correction amount. Furthermore, in step S103A, the control unit 40 calculates the lens-side blur correction amount. In the calculations of steps S102A and S103A, the subject-side blur correction amount and the lens-side blur correction amount are obtained based on the ratio most recently set in step S111.

[0277] In step S104, the control unit 40 transmits the correction command value corresponding to the blur correction amount on the subject side to the correction unit drive unit 32, and transmits the correction command value corresponding to the blur correction amount on the lens side to the lens control unit 20, so that both blur correction operations are executed.

[0278] Then, in step S106, the control unit 40 performs the process of generating metadata about optical image stabilization and recording the metadata as information corresponding to the current frame.

[0279] At this point, if a ratio is set at the current frame time, the subject-side blur correction value (correction effective value or correction instruction value) and conversion information are set as metadata related to optical image stabilization.

[0280] If no ratio is set at the current frame time point, then only the subject-side blur correction value is set as metadata related to optical image stabilization.

[0281] Therefore, conversion information is only recorded as metadata when the ratio setting is changed, which can promote the reduction of the amount of data as metadata MTD1.

[0282] It should be noted that, for example, in the absence of conversion information for a particular frame, needless to say, the image processing device 5 only needs to detect the conversion information retrospectively from that frame.

[0283] <6. Third Embodiment>

[0284] A processing example of the control unit 40 in the third embodiment is shown below. Figure 10 As shown in the image.

[0285] Figure 10An example of such processing is one in which the ratio between the subject-side blur correction and the lens-side blur correction is constant for a single moving image recording.

[0286] Control unit 40 executes at a time corresponding to each frame of the moving image. Figure 10 The processing.

[0287] Then, the control unit 40 proceeds from step S115 to step S111, and sets the ratio between the subject-side blur correction amount and the lens-side blur correction amount only at the start of recording.

[0288] In step S112, the control unit 40 performs a process of generating conversion information that indicates a newly set ratio, etc., and recording the conversion information in association with the current frame (in this case, the first frame of the moving image).

[0289] The control unit 40 proceeds from step S115 to step S101, and does not execute the processes in steps S111 and S112 except when recording begins.

[0290] In step S101, the control unit 40 detects the ambiguity.

[0291] In step S102B, the control unit 40 calculates the subject-side blur correction amount. Furthermore, in step S103B, the control unit 40 calculates the lens-side blur correction amount. In the calculations of steps S102B and S103B, the subject-side blur correction amount and the lens-side blur correction amount are obtained based on the ratio set in step S111 at the start of moving image recording.

[0292] In step S104, the control unit 40 transmits the correction command value indicating the blur correction amount on the subject side to the correction unit drive unit 32, and transmits the correction command value indicating the blur correction amount on the lens side to the lens control unit 20, so that both blur correction operations are executed.

[0293] Then, in step S106, the control unit 40 performs the process of generating metadata about optical image stabilization and recording the metadata as information corresponding to the current frame.

[0294] At this point, the subject-side blur correction value and transformation information are set as metadata related to the optical image stabilization at the time point of the first frame of the moving image.

[0295] For the second and subsequent frames, only the subject-side blur correction value is set to metadata related to optical image stabilization.

[0296] Therefore, the conversion information is recorded as metadata only for the first frame of the image file MF that constitutes the moving image, thereby facilitating a reduction in the amount of data as metadata MTD1.

[0297] Note that, for example, in image processing device 5, by acquiring the conversion information of the opening frame, it is possible to know the ratio between the subject-side blur correction value and the lens-side blur correction value of each frame.

[0298] <7. Fourth Embodiment>

[0299] A processing example of the control unit 40 in the fourth embodiment is shown below. Figure 11 As shown in the image.

[0300] Figure 11 The processing example is as follows Figure 5 The example shown in F is a composite correction value of the subject-side blur correction value and the lens-side blur correction value, which is recorded as metadata.

[0301] Steps S101 to S104 and Figure 7 Similarity in.

[0302] In step S120, the control unit 40 calculates a composite correction value of the subject-side blur correction value and the lens-side blur correction value.

[0303] Then, in step S121, the control unit 40 performs the process of recording the synthesized correction value as metadata about optical image stabilization associated with the current frame.

[0304] This is more effective in reducing the amount of data in the metadata MTD1.

[0305] Note that, for example, in the image processing device 5, a synthetic correction value can be detected for each frame. Therefore, inter-frame jitter modification for canceling image stabilization can be performed by both the image stabilization lens mechanism 10c and the imaging plane image stabilization unit 30. In this case, for example, it would be difficult to cancel the correction based solely on the image stabilization lens mechanism 10c, but it would be advantageous in applications where such processing is not required.

[0306] Conversely, if it is desired in the image processing device 5 to perform cancellation processing only on the entire image stabilization, it is also possible to obtain the advantage of not having to obtain the combined blur correction amount of both the image stabilization lens mechanism 10c and the imaging plane image stabilization unit 30.

[0307] <8. Fifth Embodiment>

[0308] The fifth embodiment is based on, as Figure 5 The subject-side blur correction value and lens-side blur correction value shown in D are both recorded as prerequisites for metadata.

[0309] Furthermore, the lens-side blur correction value is a correction execution value such as the actual position information of the image stabilizing lens mechanism 10c or its displacement. Therefore, the control unit 40 needs to wait for notification of the lens-side blur correction value from the lens control unit 20.

[0310] Note that in the fifth embodiment, similar to the first to fourth embodiments described above, the correction amount of the image stabilizing lens mechanism 10c of the lens barrel 3 can also be set on the camera control unit 18 side, and the correction command value can also be transmitted to the lens control unit 20. However, the lens control unit 20 can set the correction amount of the image stabilizing lens mechanism 10c on the lens barrel 3 side.

[0311] That is, in this example, the camera control unit 18 sets the correction amount of the imaging plane image stabilization unit 30 according to the detection value of the blur detection unit 34 and performs optical image stabilization on the camera body 2 side, and the lens control unit 20 sets the correction amount of the image stabilization lens mechanism 10c according to the detection value of the blur detection unit 25 and performs optical image stabilization on the lens barrel 3 side.

[0312] first, Figure 12 The illustration shows an example of optical image stabilization processing performed by the control unit 40. This is an example where the correction amount is also set on the lens barrel 3 side at the control unit 40 side, and steps S101 to S104 are... Figure 7 Similarity in.

[0313] As described above, there are cases where the camera control unit 18 and the lens control unit 20 respectively perform control on the camera body 2 side and the lens barrel 3 side.

[0314] exist Figure 12 In step S130, the control unit 40 acquires the subject-side blur correction value and sets the subject-side blur correction value so that it is recorded as metadata associated with the current frame at a predetermined time point.

[0315] Control unit 40, for example, with Figure 12 The processing is executed asynchronously. Figure 13 The processing involves the following steps: In step S130, the control unit 40 determines whether it has received a notification of the lens-side blur correction value from the lens control unit 20, and if the notification has been received, in step S131, the lens-side blur correction value is set so as to be recorded in association with the current frame.

[0316] For example, in Figure 12 Step S130 and Figure 13 In step S131, the subject-side blur correction value and lens-side blur correction value, which are set as metadata, are recorded as metadata corresponding to a specific frame, such as... Figure 5 As shown in D.

[0317] However, as Figure 13 As in step S131, the lens-side blur correction value obtained through communication with the lens control unit 20 may be insufficient in terms of information accuracy or may not be available in time series.

[0318] For example, when the communication speed between the camera control unit 18 and the lens control unit 20 decreases, it may become impossible to obtain lens-side blur correction values ​​for every frame. However, in some cases, lens-side blur correction values ​​for different frames can be obtained.

[0319] Therefore, depending on the communication speed, insufficient lens-side blur correction values ​​may easily occur during subsequent processing by the image processing device 5.

[0320] It should be noted that the communication speed between the camera body 2 side and the lens barrel 3 side varies significantly depending on the model and performance of the lens barrel 3. For example, the communication speed will be reduced when an older model of lens barrel 3 is installed.

[0321] Considering the above situation, the control unit 40 performs the following... Figure 14 The processing shown in A is performed, and the image processing device 5 performs the following: Figure 14 The process shown in B.

[0322] Figure 14 Figure A illustrates an example of the process by which the control unit 40 performs the recording of information about communication speed (e.g., bits per second (bps)) as metadata.

[0323] In step S201, the control unit 40 detects and maintains the communication speed information between the lens barrel 3 and the camera body 2 (i.e., between the lens control unit 20 and the camera control unit 18).

[0324] In step S202, the control unit 40 records the communication speed information along with the image data in the metadata (see...). Figure 5 (The dashed line in B).

[0325] Note that, for example, if the main factor affecting the communication speed is the type of lens barrel 3, then the communication speed only needs to be recorded once for a moving image.

[0326] If there are factors that cause changes in communication speed during the recording of moving images, it is conceivable to record the communication speed as metadata associated with each frame.

[0327] Image processing device 5 that processes image files MF Figure 14 The handling of B.

[0328] In step S301, the image processing device 5 acquires image data VD1 and metadata MTD1 as an image file MF. For example, the image file MF is read from a recording medium. Alternatively, the image file MF is received from the imaging device 1.

[0329] In step S302, the image processing device 5 extracts communication speed information included in the metadata MTD1 and compares this information with a threshold. This threshold is used to determine whether the communication speed between the lens control unit 20 and the camera control unit 18 is fast or slow. In other words, this threshold is used to determine whether the lens-side blur correction value included in the metadata MTD1 is suitable for inter-frame jitter correction processing.

[0330] If the communication speed is higher than the threshold, the image processing device 5 determines that high-speed communication is performed between the lens control unit 20 and the camera control unit 18 when imaging the image file MF in the imaging device 1, so as to maintain the reliability of the lens-side blur correction value, and proceeds to step S303 to generate a compatibility flag.

[0331] In this case, the image processing device 5 sets the processing based on a compatibility flag in step S305. Specifically, it is set to enable inter-frame jitter correction processing that uses both subject-side blur correction values ​​and lens-side blur correction values.

[0332] On the other hand, if the communication speed is not higher than the threshold, the image processing device 5 determines that the communication between the lens control unit 20 and the camera control unit 18 is performed at a low speed when the image file MF in the imaging device 1 is being imaged, so that the reliability of the lens-side blur correction value is not maintained, and proceeds to step S304 to generate an incompatibility flag.

[0333] In this case, the image processing device 5 performs processing based on the incompatibility flag in step S305. For example, it is set so that the lens-side blur correction value cannot be used in the inter-frame jitter correction processing.

[0334] This configuration prevents the image processing device 5 from performing inter-frame jitter modification processing that uses inappropriate lens-side blur correction values.

[0335] <9. Sixth Embodiment>

[0336] A processing example of the control unit 40 in the sixth embodiment is shown below. Figure 15 As shown in A, and the processing example of image processing device 5 is in Figure 15 As shown in B. Each of these has the same characteristics as described above. Figure 14 Another processing example for each of the same purposes in A and 14B.

[0337] Figure 15 Figure A illustrates an example of the process by which the control unit 40 executes the recording of flag information as communication speed information.

[0338] In step S201, the control unit 40 detects and maintains the communication speed (bps) between the lens barrel 3 and the camera body 2 (i.e., between the lens control unit 20 and the camera control unit 18).

[0339] In step S210, the control unit 40 compares the communication speed with a threshold. This threshold is used to determine whether the communication speed between the lens control unit 20 and the camera control unit 18 is in a fast or slow state, similar to... Figure 14 The threshold described in B.

[0340] If the communication speed is higher than the threshold, then the control unit 40 proceeds to step S212 and generates a compatibility flag.

[0341] If the communication speed is not higher than the threshold, the control unit 40 proceeds to step S213 and generates an incompatibility flag.

[0342] Then, in step S214, the control unit 40 records the communication speed information along with the image data in the metadata (see...). Figure 5 (The dashed line in B). In this case, the compatibility or incompatibility flag is recorded as communication speed information.

[0343] Image processing device 5 that processes image files MF Figure 15 The handling of B.

[0344] In step S301, the image processing device 5 acquires image data VD1 and metadata MTD1 as image file MF.

[0345] In step S310, the image processing device 5 sets the processing based on the compatibility flag or incompatibility flag recorded in the metadata MTD1. That is, if the compatibility flag can be confirmed, then inter-frame jitter correction processing using both the subject-side blur correction value and the lens-side blur correction value can be performed on the current image file MF.

[0346] On the other hand, if the incompatibility flag is confirmed, then it is set to make the lens-side blur correction value unavailable in the inter-frame jitter modification processing of the current image file MF.

[0347] This configuration also prevents the execution of inter-frame jitter modification processing using inappropriate lens-side blur correction values ​​in the image processing device 5.

[0348] The speed information included in the metadata MTD1 can be, for example, a 1-bit flag, which also helps reduce the amount of data in the metadata MTD1.

[0349] For example, regarding specific locations in the metadata MTD1, it is only required to set, for example, "0" and "1" as compatibility and incompatibility flags, respectively.

[0350] <10. Seventh Embodiment>

[0351] like Figure 13 As shown, an example of the control unit 40 using the lens-side blur correction value received from the lens control unit 20 as metadata has been described in the fifth and sixth embodiments above.

[0352] In this case, it is also conceivable that the lens control unit 20 transmits the lens-side blur correction value while temporarily storing the lens-side blur correction value in the memory unit 26, and this will be described as a seventh embodiment.

[0353] The subject-side blur correction value and the lens-side blur correction value only need to be recorded as metadata of the moving image. Therefore, it is conceivable that the lens control unit 20 does not send the lens-side blur correction value in real time, but instead temporarily stores the lens-side blur correction value in the memory unit 26 and sends the lens-side blur correction value when there is sufficient margin in lens-subject communication.

[0354] For example, even when lens-body communication is very slow, if the movement of the image stabilizing lens mechanism 10c is sampled at communication intervals, the change in movement loss per unit time is large, and it is difficult to send the lens-side blur correction value, which is in a state of maintaining accuracy, as data to the camera body 2 in real time. It is advantageous to temporarily store the lens-side blur correction value in the memory unit 26 and perform the transmission process later.

[0355] From this perspective, the lens control unit 20 performs operations such as capturing moving images. Figure 16 The processing.

[0356] In step S501, the lens control unit 20 determines whether it is a transmission timing. The transmission timing is a set timing.

[0357] For example, one could consider setting timed transmissions, irregular transmissions, timings depending on the storage capacity of the memory unit 26, or timings for the end of moving image recording. In step S501, the lens control unit 20 determines whether such a timing is currently in effect based on these settings.

[0358] If it is not a transmission timing, then the lens control unit 20 proceeds to step S502 and detects the lens-side blur correction value as the correction execution value in the image stabilization lens mechanism 10c. Then, in step S503, the currently detected lens-side blur correction value is stored in the memory unit 26 in association with a timestamp indicating the current time (the time corresponding to the current frame).

[0359] This is because, in order to associate the lens-side blur correction value with the frame of the moving image, the timestamp needs to be transmitted as a set to the camera body 2 side.

[0360] If it is a transmission timing, then the lens control unit 20 proceeds to step S504 and executes the process of transmitting the lens-side blur correction value recorded in the memory unit 26 together with the timestamp to the camera control unit 18.

[0361] pass Figure 16 When the set transmission time arrives, the lens-side blur correction values ​​of multiple samples stored in the memory unit 26 are transmitted to the camera control unit 18.

[0362] For example, batch transmissions can be performed periodically at set intervals. Alternatively, batch transmissions can be performed at irregular intervals in response to a trigger. Performing batch transmissions periodically or irregularly reduces transmission opportunities compared to sequentially transmitting lens-side blur correction values ​​in real time, thus alleviating communication overhead.

[0363] Furthermore, batch transfers can be performed at timings that depend on the storage capacity of the lens-side blur correction values ​​in memory unit 26. Therefore, it is possible to avoid situations where it is difficult to store more lens-side blur correction values ​​in memory unit 26.

[0364] In addition, batch transmissions are sometimes performed after the capture of moving images has ended. Therefore, transmissions can be performed at times when there is a communication margin.

[0365] These transmission timings can be used in combination.

[0366] Control unit 40 when recording moving images Figure 12 The above processes are executed together. Figure 17 The processing.

[0367] exist Figure 17 In step S601, the control unit 40 waits to receive the lens-side blur correction value from the lens control unit 20.

[0368] When a lens-side blur correction value is received from the lens control unit 20, a process of setting the lens-side blur correction value as metadata is performed in step S602. At this time, timestamps are transmitted in association with each lens-side blur correction value. Therefore, the control unit 40 uses each timestamp to determine which frame of the moving image is associated with each received lens-side blur correction value, and sets the lens-side blur correction value as metadata associated with the determined frame.

[0369] exist Figure 12 Step S130 and Figure 17 In step S602, the subject-side blur correction value and lens-side blur correction value of the metadata are set to be associated with the current frame, either separately or simultaneously, and recorded as metadata in the recording medium in the recording control unit 14.

[0370] Note that users can choose to transmit the lens-side blur correction value in real time or temporarily store the lens-side blur correction value and transmit it at the timed settings described above.

[0371] Transmission after temporary storage has the advantages in terms of communication load as described above, while real-time transmission also has the advantages of reducing the memory unit 26 on the lens barrel 3 side and reducing the transmission time and metadata processing time caused by subsequent transmissions. Therefore, it is appropriate to allow the user to make a choice according to the situation.

[0372] In addition, one could think of examples that combine real-time transmission with transmission after temporary storage.

[0373] Figure 18 The diagram illustrates the processing of the lens control unit 20.

[0374] In step S601A, the lens control unit 20 determines whether it is the transmission timing for bulk transmission of lens-side blur correction values ​​stored in the memory unit 26.

[0375] If it is not a transmission timing, then the lens control unit 20 proceeds to step S502 and detects the lens-side blur correction value as the correction execution value in the image stabilization lens mechanism 10c.

[0376] In step S530, the lens control unit 20 determines whether there is a transmission margin under the current circumstances. For example, it determines whether there is a transmission margin based on factors such as the current communication speed and the amount of data to be transmitted.

[0377] If there is a transmission margin, the lens control unit 20 proceeds to step S531 and transmits the currently detected lens-side blur correction value along with the timestamp to the camera control unit 18.

[0378] On the other hand, if there is no transmission margin, then in step S532, the currently detected lens-side blur correction value is stored in memory unit 26 in association with a timestamp indicating the current time (the time corresponding to the current frame).

[0379] If the transmission timing is in step S501A, then the lens control unit 20 proceeds to step S504 and executes the process of transmitting the lens-side blur correction value recorded in the memory unit 26 along with the timestamp to the camera control unit 18.

[0380] pass Figure 18 The processing of lens-side blur correction values ​​is used in conjunction with real-time transmission and batch transmission after temporary storage. They are all paired with timestamps, so they can be appropriately associated with frames of moving images for use as metadata on the control unit 40 side.

[0381] When used together with real-time transmission, another advantage is that the capacity of memory unit 26 can be reduced.

[0382] <11. Summary and Revision>

[0383] According to the above embodiments, the following effects can be obtained.

[0384] As described in the first to seventh embodiments, the imaging device 1 causes the control unit 40 to perform processing to generate correction information based on two values: a subject-side blur correction value (first blur correction value) related to the imaging plane image stabilization unit 30 (first blur correction function) of the camera body 2 and a lens-side blur correction value (second blur correction value) related to the image stabilization lens mechanism 10c (second blur correction function) in the lens barrel 3, and to set the correction information as metadata associated with the captured image.

[0385] That is, when mechanical blur correction functions are set on the lens barrel 3 side and the camera body 2 side respectively, as an imaging device 1, the correction information based on the corresponding blur correction value is associated with the moving image being captured as metadata.

[0386] Therefore, the jitter state of the image can be changed later, for example, in the image processing device 5, etc. For example, blur correction by the image stabilizing lens mechanism 10c and blur correction by the imaging plane image stabilization unit 30 can both be canceled, or only the blur correction of either one can be canceled. Furthermore, if blur correction already performed by the imaging device 1 is canceled, more accurate blur correction can be performed in the image processing device 5, or inter-frame jitter generation that intentionally adds jitter can be performed.

[0387] It should be noted that while the embodiment has described a function focused on preventing mechanical blur using the imaging plane image stabilization unit 30 and the image stabilization lens mechanism 10c, the techniques disclosed herein can also be applied to cases employing an electrical image stabilization function. For example, when blur correction is performed by the image stabilization lens mechanism 10c on the lens barrel 3 side and electrical image stabilization is performed on the camera body 2 side, the electrical image stabilization function can be considered as a first blur correction function and the image stabilization lens mechanism 10c can be considered as a second blur correction function.

[0388] In the examples described in the first, second, third, and fourth embodiments, the control unit 40 generates correction information with a smaller amount of data than the amount of data obtained by adding the corresponding data amounts of the subject-side blur correction value and the lens-side blur correction value. Figure 5 Information on optical image stabilization in B can be found in [link / reference]. Figure 5 E and 5F).

[0389] Therefore, the amount of data to be recorded as metadata MTD1 can be reduced. In other words, if it is desired to store the corresponding blur correction values ​​on the lens barrel 3 side and the camera body 2 side respectively, an unnecessary increase in the amount of data can be prevented.

[0390] In the examples described in the first, second, and third embodiments, the control unit 40 uses one of the subject-side blur correction values ​​and the lens-side blur correction values, along with conversion information indicating the ratio, division value, difference, etc. between them, as correction information for optical image stabilization.

[0391] Therefore, the blur correction values ​​of the image stabilizing lens mechanism 10c and the imaging plane image stabilizing unit 30 can be obtained from the correction information recorded as metadata, and it is possible to obtain an effect similar to that in the case where each of these blur correction values ​​is recorded as metadata. In this case, the amount of data to be recorded as metadata can be reduced. In other words, if it is desired to store the corresponding correction values ​​on the lens barrel 3 side and the camera body 2 side, an unnecessary increase in the amount of data can be prevented.

[0392] Note that examples in the first, second, and third embodiments where the blur correction value and conversion information of the imaging plane image stabilization unit 30 are recorded as metadata have been described; however, the blur correction value and conversion information of the image stabilizing lens mechanism 10c can also be recorded as metadata. In this case, the conversion information can be the aforementioned ratio "α:(1-α)", or it can be a value such as "α" or "1-α". Furthermore, considering that the conversion information is, for example, a value that enables the acquisition of the subject-side blur correction value from the lens-side blur correction value, it can be used as a division value of (subject-side blur correction value) / (lens-side blur correction value) or the difference between the lens-side blur correction value and the subject-side blur correction value.

[0393] In the examples described in the first, second, and third embodiments, the control unit 40 uses the subject-side blur correction value and conversion information as correction information for optical image stabilization.

[0394] Therefore, the amount of data to be recorded as metadata can be reduced, and the control unit 40 can generate metadata without receiving blur correction values ​​from the lens barrel 3 side. Thus, metadata can be generated and recorded without being affected by the communication speed or communication delay between the camera control unit 18 and the lens control unit 20.

[0395] In the example described in the first embodiment, the control unit 40 generates a pair of correction information at each predetermined timing (e.g., at the timing of each frame), including a blur correction value (one of the subject-side blur correction value and the lens-side blur correction value) and conversion information.

[0396] Therefore, for example, the blur correction value through the image stabilization lens mechanism 10c and the blur correction value through the imaging plane image stabilization unit 30 can be obtained from the metadata for each frame, and the correction cancellation and inter-frame jitter generation in the image processing device 5 and the like can be performed appropriately.

[0397] In the example described in the second embodiment, the control unit 40 generates correction information including a blur correction value, one of a subject-side blur correction value and a lens-side blur correction value, and timing information of the conversion of the ratio between the subject-side blur correction value and the lens-side blur correction value.

[0398] Therefore, for example, it is not necessary to record the conversion information for every frame every time, and the amount of data in the metadata MTD1 can be reduced.

[0399] In the example described in the third embodiment, the control unit 40 sets the ratio between the subject-side blur correction value and the lens-side blur correction value to a fixed ratio from the start to the end of recording the moving image, and generates correction information including a blur correction value in the subject-side blur correction value and the lens-side blur correction value at each timed interval, and conversion information indicating the fixed ratio.

[0400] Therefore, for example, it is not necessary to record the transformation information for every frame every time, and the amount of metadata can be reduced. In particular, only one piece of transformation information is required to be recorded in association with the moving image, and the reduction in data volume is significant.

[0401] In the example described in the fourth embodiment, the control unit 40 uses a composite correction value of the subject-side blur correction value and the lens-side blur correction value as correction information.

[0402] Therefore, the amount of data recorded as metadata can be reduced. Furthermore, since the synthesized correction value indicates the total amount of correction applied to the image, correction cancellation can be appropriately performed in, for example, the image processing device 5.

[0403] In the examples described in the fifth and sixth embodiments, the control unit 40 sets the subject-side blur correction value and the lens-side blur correction value as correction information, and performs the process of recording the communication speed information between the lens barrel 3 (lens control unit 20) and the camera body 2 (camera control unit 18) as metadata in the recording medium.

[0404] Therefore, it is possible to confirm from the metadata whether the information regarding lens-side blur correction values ​​is affected by communication speed. For example, in cases of low communication speed and insufficient lens-side blur correction values, it is possible that these values ​​may not be used to perform correction cancellation, etc.

[0405] In the fifth embodiment, the communication speed information is a value indicating the communication speed (e.g., bps). Therefore, it is possible to confirm from the metadata whether the information of the lens-side blur correction value is delay information, the degree of delay, etc.

[0406] In the sixth embodiment, the communication speed information is the result information (compatibility flag / incompatibility flag) obtained by comparing the communication speed with a predetermined value.

[0407] Therefore, it is easy to confirm from the metadata whether the information on the lens-side blur correction value has been delayed.

[0408] In the embodiments, examples have been described whereby the blur correction value, which is either the subject-side blur correction value or the lens-side blur correction value, is a correction execution value indicating the position or position displacement amount corrected according to the blur correction function (image plane image stabilization unit 30 or image stabilization lens mechanism 10c).

[0409] This is a value that indicates the amount of correction that actually affects the image, and is, for example, the most accurate value that takes into account correction cancellation in the image processing device 5.

[0410] Furthermore, examples have been described in the embodiments where the fuzz correction value is a correction instruction value that gives instructions about the position or position displacement amount for the fuzz correction function.

[0411] The correction command value does not directly indicate the amount of blur correction actually applied to the image, but if the blur correction function is highly accurate relative to the operation of the command, then it is acceptable as a value that substantially indicates the actual amount of blur correction. Furthermore, since the correction command value is generated by the camera control unit 18, it has the advantage of being able to generate and record metadata without being affected by communication with the lens barrel 3.

[0412] An example has been described in the embodiments in which the control unit 40 records the information (IMU data) detected by the blur detection unit 34 as metadata.

[0413] Since the IMU data, which serves as information about the jitter actually applied to imaging device 1, is recorded as metadata, the jitter that actually affects the image can be determined based on the metadata. Therefore, various inter-frame jitter modifications become possible.

[0414] It should be noted that the effects described in this manual are merely examples and not limitations, and other effects may exist.

[0415] Note that this technology can also have the following configurations. (1)

[0417] An imaging device, comprising:

[0418] The control unit generates correction information based on both a first blur correction value and a second blur correction value as metadata associated with the captured image. The first blur correction value is related to a first blur correction function that corrects the positional relationship between the optical image incident through the lens and the output captured image, and the second blur correction value is related to a second blur correction function set in the lens barrel including the lens. (2)

[0420] According to the imaging device described in (1) above, wherein,

[0421] Control Unit

[0422] Correction information is generated with a smaller amount of data than the sum of the data amounts of the first fuzziness correction value and the second fuzziness correction value. (3)

[0424] According to the imaging device described in (1) or (2) above, wherein,

[0425] Control unit settings

[0426] One of the first and second fuzzy correction values, and the fuzzy correction value.

[0427] Transformation information configured to use one fuzz correction value to derive another fuzz correction value.

[0428] As correction information. (4)

[0430] The imaging device according to any one of (1) to (3) above, wherein,

[0431] Control unit settings

[0432] First fuzzy correction value and

[0433] The conversion information is configured to use the first fuzz correction value to obtain the second fuzz correction value as the correction information. (5)

[0435] The imaging device according to any one of (1) to (4) above, wherein,

[0436] Control Unit

[0437] At each predetermined time interval, correction information is generated, which includes a pair of fuzz correction values, one of the first and second fuzz correction values, and transformation information configured to use the first fuzz correction value to derive the other fuzz correction value. (6)

[0439] The imaging device according to any one of (1) to (4) above, wherein,

[0440] Control Unit

[0441] Generate calibration information, which includes:

[0442] One of the first and second fuzzy correction values ​​at each timing, and

[0443] The conversion information is configured to use the timing of the change in the ratio between the first fuzz correction value and the second fuzz correction value to determine the transformation information of the other fuzz correction value. (7)

[0445] The imaging device according to any one of (1) to (4) above, wherein,

[0446] Control Unit

[0447] From the start to the end of recording the moving image, the ratio between the first blur correction value and the second blur correction value is set to a fixed ratio, and

[0448] Generate calibration information, which includes:

[0449] In each timed first and second fuzzy correction value, one of the fuzzy correction values ​​is a fuzzy correction value, and

[0450] Based on the conversion information of a fixed ratio, the conversion information is configured to use the one fuzz correction value to obtain another fuzz correction value. (8)

[0452] According to the imaging device described in (1) or (2) above, wherein,

[0453] Control Unit

[0454] The combined correction value of the first fuzzy correction value and the second fuzzy correction value is set as the correction information. (9)

[0456] According to the imaging device described in (1) above, wherein,

[0457] Control Unit

[0458] Set the first fuzzy correction value and the second fuzzy correction value as the correction information, and

[0459] The process of recording the communication speed information between the lens barrel and the camera body as metadata in the recording medium is performed. (10)

[0461] According to the imaging device described in (9) above, wherein,

[0462] Communication speed information is a value that indicates the speed of communication. (11)

[0464] According to the imaging device described in (9) above, wherein,

[0465] Communication speed information is obtained by comparing the communication speed with a predetermined value. (12)

[0467] The imaging device according to any one of (1) to (11) above, wherein,

[0468] The fuzz correction value is the correction execution value that indicates the position or position displacement amount corrected according to the fuzz correction function. (13)

[0470] The imaging device according to any one of (1) to (11) above, wherein,

[0471] The fuzz correction value is a correction instruction value that indicates the position or position displacement used for fuzz correction function. (14)

[0473] The imaging device according to any one of (1) to (13) above, wherein,

[0474] The control unit sets the jitter information detected by the jitter detection unit as metadata. (15)

[0476] An imaging method, comprising:

[0477] The process of generating correction information based on both a first blur correction value and a second blur correction value as metadata associated with the captured image, wherein the first blur correction value is related to a first blur correction function that corrects the positional relationship between the optical image incident through the lens and the output captured image, and the second blur correction value is related to a second blur correction function set in the lens barrel including the lens, is performed by the imaging device.

[0478] List of reference numerals

[0479] 1 Imaging equipment

[0480] 2 Camera body

[0481] 3-lens tube

[0482] 5 and 6 Image processing equipment

[0483] 10-lens system

[0484] 10a zoom lens

[0485] 10b aperture mechanism

[0486] 10c Image Stabilizing Lens Mechanism

[0487] 10D focusing lens

[0488] 11 shutter speeds

[0489] 12 imaging element units

[0490] 13 Camera signal processing unit

[0491] 14 Recording Control Unit

[0492] 15 display units

[0493] 16 output units

[0494] 17 operating units

[0495] 18 Camera control unit

[0496] Memory units 19 and 26

[0497] 20 Lens Control Unit

[0498] 21 zoom drive units

[0499] 22 aperture drive units

[0500] 23 Correction Lens Drive Unit

[0501] 24 Focusing drive units

[0502] 25 and 34 fuzzy detection units

[0503] 27, 33 Communication Control Unit

[0504] 30 Imaging Plane Image Stabilization Unit

[0505] 31 Shutter drive unit

[0506] 32 Correction Unit Drive Unit

[0507] 35. Electromagnetic image stabilization control unit

[0508] 36 Fuzzy Correction Metadata Processing Unit

[0509] 40 control units

Claims

1. An imaging device, comprising: Control Unit Correction information based on both a first blur correction value and a second blur correction value is generated as metadata associated with the captured image. The first blur correction value is related to a first blur correction function that corrects the positional relationship between the optical image incident through the lens and the output captured image. The second blur correction value is related to a second blur correction function set in the lens barrel including the lens. The first and second blur correction values ​​can be obtained based on the generated correction information, and the amount of data in the generated correction information is less than the sum of the amounts of the first and second blur correction values. The communication speed information between the lens barrel and the camera body is recorded in the metadata. If the communication speed information indicates a communication speed higher than a predetermined value, the processing device is instructed to perform inter-frame jitter correction processing using both the first and second blur correction values. If the communication speed information indicates a communication speed lower than the predetermined value, the processing device is instructed to perform inter-frame jitter correction processing using the first blur correction value but not the second blur correction value.

2. The imaging device according to claim 1, wherein, Control unit settings One of the first and second fuzzy correction values, and the fuzzy correction value. Transformation information configured to use one fuzz correction value to derive another fuzz correction value. As correction information.

3. The imaging device according to claim 1, wherein, Control unit settings First fuzzy correction value and Transformation information configured to use the first fuzz correction value to derive the second fuzz correction value. As correction information.

4. The imaging device according to claim 1, wherein, Control Unit At each predetermined time interval, correction information is generated, which includes a pair of fuzz correction values, one of the first and second fuzz correction values, and transformation information configured to use the first fuzz correction value to derive the other fuzz correction value.

5. The imaging device according to claim 1, wherein, Control Unit Generate calibration information, which includes: One of the first and second fuzzy correction values ​​at each timing, and The conversion information is configured to use the timing of the change in the ratio between the first fuzz correction value and the second fuzz correction value to determine the transformation information of the other fuzz correction value.

6. The imaging device according to claim 1, wherein, Control Unit From the start to the end of recording the moving image, the ratio between the first blur correction value and the second blur correction value is set to a fixed ratio, and Generate calibration information, which includes: In each timed first and second fuzzy correction value, one of the fuzzy correction values ​​is a fuzzy correction value, and Based on the conversion information of a fixed ratio, the conversion information is configured to use the one fuzz correction value to obtain another fuzz correction value.

7. The imaging device according to claim 1, wherein, Control Unit The combined correction value of the first fuzzy correction value and the second fuzzy correction value is set as the correction information.

8. The imaging device according to claim 1, wherein, Control Unit Perform the process of recording metadata in the recording medium.

9. The imaging device according to claim 8, wherein, Communication speed information is a value that indicates the speed of communication.

10. The imaging device according to claim 8, wherein, Communication speed information is obtained by comparing the communication speed with a predetermined value.

11. The imaging device according to claim 1, wherein, The fuzz correction value is the correction execution value that indicates the position or position displacement amount corrected according to the fuzz correction function.

12. The imaging device according to claim 1, wherein, The fuzz correction value is a correction instruction value that indicates the position or position displacement used for fuzz correction function.

13. The imaging device according to claim 1, wherein, The control unit sets the jitter information detected by the jitter detection unit as metadata.

14. An imaging method performed by an imaging device, comprising: The process generates correction information based on both a first blur correction value and a second blur correction value as metadata associated with the captured image. The first blur correction value is related to a first blur correction function that corrects the positional relationship between the optical image incident through the lens and the output captured image. The second blur correction value is related to a second blur correction function set in a lens barrel including the lens. The first and second blur correction values ​​can be obtained based on the generated correction information, and the amount of data in the generated correction information is less than the sum of the amounts of the first and second blur correction values. The process of recording communication speed information between the lens barrel and the camera body in the metadata includes the following: if the communication speed information indicates a communication speed higher than a predetermined value, the processing device is instructed to perform inter-frame jitter correction processing using both a first blur correction value and a second blur correction value; if the communication speed information indicates a communication speed lower than the predetermined value, the processing device is instructed to perform inter-frame jitter correction processing using the first blur correction value but not the second blur correction value.

Images