Image processing device and imaging device

Abstract

To perform good anti-vibration and correction processing on a video image obtained by imaging using an anamorphic optical system.SOLUTION: An image processing device has means 106, 108 for generating video image data for recording by imaging through an imaging optical system 101 and means 106 for adding to the video image data for recording information regarding a compression ratio of an anamorphic optical system and information indicating a center position of the anamorphic optical system in each frame image, each frame information constituting the video image data for recording, when the imaging optical system is an anamorphic optical system.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging apparatus having an image stabilization function for reducing (correcting) image blurring. [Background technology]

[0002] One imaging technique is to compress (squeeze) an optical image in the horizontal direction using an anamorphic lens, and then perform an expansion process (desqueeze) to undo the compression of the video data obtained by capturing the optical image using image processing, thereby obtaining a Cinemascope (2.39:1) video. Patent Document 1 discloses a method for preventing image stabilization for such images. Patent Document 1 discloses a method for detecting a motion vector from video data captured using an anamorphic lens, and using the motion vector to detect the motion vector and perform anti-stabilization processing (electronic anti-stabilization) on the video data that has been uncompressed.

[0003] Furthermore, Patent Document 2 discloses an imaging method in which image data obtained by performing an anti-shake process during imaging is recorded, and the anti-shake process is then cancelled for the recorded image data based on information regarding the anti-shake process, and then the anti-shake process is performed again on the image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 3278206 [Patent Document 2] JP 2015-216510 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, Patent Document 2 does not disclose any suitable image stabilization process for effectively correcting distortion and vignetting that are specific to the use of an anamorphic lens.

[0006] The present invention provides an image processing device and an imaging device that are capable of performing good vibration prevention and correction processing on video data obtained by imaging using an anamorphic lens. [Means for solving the problem]

[0007] An image processing device according to one aspect of the present invention is characterized in having a video data generation means for generating video data for recording by imaging via an imaging optical system, and, when the imaging optical system is an anamorphic optical system, an information addition means for adding, to the video data for recording, information regarding the compression ratio of the anamorphic optical system and information indicating the center position of the anamorphic optical system in each frame image constituting the video data for recording.

[0008] According to another aspect of the present invention, there is provided an image processing device, comprising: an electronic image stabilization unit that performs electronic image stabilization for each frame image of captured video data generated by imaging through an imaging optical system, by cutting out a partial image area of ​​the frame image so as to reduce image shake, thereby generating video data for recording; and a processing unit that performs at least one of roll correction and distortion correction on the frame image of the video data for recording. The processing unit is characterized in that the correction process is limited when the imaging optical system is an anamorphic optical system.

[0009] An image processing device according to another aspect of the present invention is characterized in that it comprises: electronic image stabilization means for performing electronic image stabilization for each frame image of an image captured by imaging through an imaging optical system, in which a part of an image area of ​​the frame image is cut out so as to reduce image shake, thereby generating video data for recording; and control means for limiting electronic image stabilization or performing a warning operation when the imaging optical system is an anamorphic optical system. An imaging device having the image processing device also constitutes another aspect of the present invention. An image processing method executed by the image processing device and a computer program for causing a computer to execute processing according to the image processing method also constitute another aspect of the present invention. Effect of the Invention

[0010] According to the present invention, it is possible to realize an image processing device and an imaging device that are capable of satisfactorily carrying out anti-shake processing and correction processing on an image obtained by imaging using an anamorphic optical system. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of an imaging apparatus according to an embodiment of the present invention. [Diagram 2] FIG. 4 is a diagram showing a coordinate system in the first and second embodiments. [Diagram 3] 4 is a flowchart showing a process in the first embodiment. [Figure 4] 10 is a flowchart showing another process in the first embodiment. [Diagram 5] 11 is a flowchart showing a process according to the second embodiment. [Figure 6] FIG. 11 is a diagram showing a display example of a user interface in the second embodiment. [Figure 7] 13A and 13B are diagrams showing a recorded video and an image after editing processing in the third embodiment. [Figure 8] 11 is a flowchart showing a process according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. EXAMPLES

[0013] 1 shows the configuration of an image capture device equipped with an image processing device according to a first embodiment of the present invention. The image capture device of this embodiment is a digital camera with interchangeable lenses or a digital camera with an integrated lens. Note that the present invention can also be applied to various image capture devices other than the above-mentioned digital cameras.

[0014] In this embodiment, the shake applied to the imaging device due to camera shake or the like is called "camera shake", and the shake of the captured image caused by camera shake is called "image shake". As shown in Fig. 2, the axis extending vertically in a plane perpendicular to the optical axis of the imaging optical system (the central axis of horizontal rotation) is called the Yaw axis, and the axis extending horizontally (the central axis of vertical rotation) is called the Pitch axis. Furthermore, the axis extending in the optical axis direction (the central axis of rotation around the optical axis) is called the Roll axis.

[0015] 1, an imaging lens (imaging optical system) 101 includes a zoom lens 102 that changes magnification, a shift lens 103 as a correction lens for optically correcting image shake (lens vibration reduction), a focus lens 104 that adjusts focus, and an aperture (not shown) that adjusts the amount of light. The imaging lens 101 focuses light from a subject (not shown) and forms an optical image (subject image) on an imaging element 105.

[0016] The imaging element 105 is configured with a CMOS sensor or the like, photoelectrically converts an optical image formed by the imaging lens 101 to generate an imaging signal, and outputs the imaging signal to a signal processing unit 106. The imaging element 105 is shifted in the horizontal and vertical directions within a plane perpendicular to the optical axis and rotated around the optical axis by a motor 127. This makes it possible to perform sensor vibration reduction as image blur correction by moving (shifting and rotating) the imaging element 105.

[0017] The signal processor 106 performs signal processing such as white balance adjustment and gamma correction on the imaging signal output from the imaging element 105 to generate captured video data (a plurality of frame images) as moving image data. The captured video data is stored in an image memory 107.

[0018] An image cut-out control unit (electronic image stabilization means) 108 cuts out a predetermined partial image area (cut-out area) from each frame image of the captured video data stored in the image memory 107, generates new frame images that constitute video data for recording and display, and outputs these to the recording control unit 111 and the display control unit 109. At this time, electronic image stabilization is performed to correct (reduce) image shake, which is the movement of the subject between frame images caused by camera shake, by moving (shifting or rotating) the cut-out area in response to camera shake.

[0019] If the imaging signal is a video signal conforming to the NTSC format, for example, the series of processes performed by the signal processing unit 106 and the image clipping control unit 108 is executed at a cycle of 60 Hz, thereby generating video data for recording and display. That is, in this embodiment, the signal processing unit 106 and the image clipping control unit 108 function as a video data generating means for generating video data for recording.

[0020] The display control unit 109 causes the display device 110 to display the display video data output from the image cropping control unit 108 as a live view image, and causes the display device 110 to display menu images for the user to perform various settings, recorded video data, and the like. The display device 110 is composed of a liquid crystal display element (LCD) or the like.

[0021] The recording control unit 111 controls the recording and reading of video data, meta information, etc. onto the recording medium 112. The recording medium 112 is configured from a semiconductor memory, a hard disk, or the like.

[0022] The angular velocity sensor 113 is a sensor for detecting camera shake around the yaw axis, pitch axis, and roll axis applied to the imaging device 100, and outputs a shake signal having a voltage value indicating the angular velocity of the camera shake. The A / D converter (1) 114 converts the shake signal (voltage signal) output from the angular velocity sensor 113 into angular velocity data as digital data, and outputs the angular velocity data to the correction amount calculation unit 117.

[0023] The motion vector detection unit 115 detects motion vectors in the horizontal and vertical directions in a plane perpendicular to the optical axis, for example, using a block matching method. Note that a correlation method may be used instead of the block matching method. In the block matching method, the motion vector detection unit 115 divides the input frame image into a plurality of blocks (each block is, for example, 16×16 pixels) and calculates the difference from the previous frame image for each block (target block). Then, it searches for a block in the previous frame image that minimizes the sum of the absolute values ​​of these differences, and detects the relative deviation between the block identified as a result of the search and the target block as the motion vector in the target block. In this way, the horizontal and vertical motion vectors are detected in pixel units of the frame image. The detected horizontal motion vector is sent to the correction amount calculation unit 117 as H_Vect, and the vertical motion vector is sent to the correction amount calculation unit 117 as V_Vect.

[0024] The lens information acquisition unit 116 acquires information indicating the characteristics of the imaging lens 101 (hereinafter referred to as optical system information). The optical system information includes focal length, aperture value, focus position, shift lens movement amount, distortion amount, compression ratio of an anamorphic lens described later, center position and diameter of an effective image circle, etc. If the imaging lens 101 is an interchangeable lens, the optical system information may be acquired by communication with the interchangeable lens, or may be acquired by a user input via a user interface provided in the imaging device 100. The acquired optical system information is sent to a correction amount calculation unit 117.

[0025] The correction amount calculation unit 117 calculates a correction amount for correcting image shake caused by camera shake, and sends it to the correction amount division control unit 118. The correction amount calculation unit 117 calculates not the correction amount for each of the above-mentioned lens shake reduction, sensor shake reduction, and electronic shake reduction, but the correction amount for the lens shake reduction, sensor shake reduction, and electronic shake reduction combined (hereinafter referred to as the total correction amount). The correction amount division control unit 118 divides the total correction amount calculated by the correction amount calculation unit 117 into the correction amounts for lens shake reduction, sensor shake reduction, and electronic shake reduction. However, in the following explanation, it is assumed that only sensor shake reduction and electronic shake reduction are performed. For this reason, the correction amount calculation unit 117 divides the total correction amount into the correction amount for sensor shake reduction (hereinafter referred to as the sensor correction amount) and the correction amount for electronic shake reduction (hereinafter referred to as the electronic correction amount).

[0026] The sensor correction amount conversion unit 119 converts the sensor correction amount from the correction amount division control unit 118 into a movement amount of the image sensor 105, and outputs a target position of the image sensor 105 corresponding to the movement amount (hereinafter referred to as a sensor target position). The electronic correction amount conversion unit 120 converts the electronic correction amount from the correction amount division control unit 118 into a cut-out position which is the center position of the cut-out area in the image cut-out control unit 108, and sets the cut-out position in the image cut-out control unit 108.

[0027] The position detection unit 121 outputs a sensor position signal having a voltage value indicating the position of the image sensor 105. The sensor position signal is amplified by an amplifier 122 to a signal within a predetermined voltage range, and then converted by an A / D converter (2) 123 into sensor position data (deviation data) as digital data.

[0028] The control filter 124 performs signal processing such as amplification and phase compensation on the sensor position data, and sends the sensor position data after the signal processing to a pulse width modulation unit 125 .

[0029] The pulse width modulation unit 125 converts the sensor position data from the control filter 124 into a PWM signal and outputs the PWM signal to the motor drive unit 126. The motor 127 is composed of a voice coil motor or the like, and is driven by the PWM signal from the motor drive unit 126 to move the image sensor 105 in the horizontal and vertical directions. The position of the moved image sensor 105 is detected by the position detection unit 121 and used to calculate the next sensor position data. As a result, feedback control is performed so that the difference between the sensor target position and the sensor position data becomes small. The control filter 124, the pulse width modulation unit 125, the motor drive unit 126, and the motor 127 form a sensor vibration isolation means.

[0030] The flowchart in Fig. 3 shows the process of adding meta-information to video data for recording (frame images). This process is a VD (Vertical Driving Pulse) process performed for each frame. A signal processing unit (video data generating means and information adding means) 106 as a computer repeatedly executes this process at a predetermined cycle (for example, 60 Hz) according to a computer program. Here, a case where the imaging lens 101 is an anamorphic lens (anamorphic optical system) will be described.

[0031] First, in step S301, the signal processing unit 106 determines whether or not the execution of electronic stabilization has been selected (ON) by the user, and if it is ON, the process proceeds to step S302, and if not, the process proceeds to step S303. In step S303, the signal processing unit 106 adds meta information indicating that electronic stabilization is OFF to the frame image and records it.

[0032] Meanwhile, in step S302, the signal processing unit 106 adds information to be used in post-processing of the frame image after electronic image stabilization as meta-information for each frame image, and records the information. The information used in this post-processing includes information on the compression ratio of the anamorphic lens, information on the position of the cut-out area in electronic image stabilization, and information on the cut-out magnification of the cut-out area. The information used in the post-processing also includes information on the roll angle, which is the tilt angle of the frame image around the optical center position, and information on the distortion of the frame image caused by rolling shutter distortion and optical distortion caused by the imaging lens 101. The "information about A" mentioned here includes not only information directly indicating "A", but also information that indirectly indicates "A", such as information that can be converted to "A".

[0033] The information on the compression ratio of the anamorphic lens may be, for example, the compression ratio (1 / 2, 1 / 1.3, etc.) determined for each attached anamorphic lens. It may also be a table number that can be converted to a compression ratio (1 / 2 for No. 1, 1 / 1.3 for No. 2, etc.), or it may be the model number of the interchangeable lens if a table that associates model numbers with compression ratios is stored so that the compression ratio can be obtained from the model number of the interchangeable lens. The information on the position of the cutout region is expressed, for example, by the (horizontal coordinate, vertical coordinate) of the center position of the cutout region. For example, in a frame image of 3840 x 2160 pixels, (1920, 1080) indicates that the center position of the cutout region in electronic image stabilization coincides with the optical axis position of the imaging lens 101, that is, the optical center position. Also, (0, 0) when the center position of the cutout region in electronic image stabilization is expressed as an offset position from the optical center position also indicates that the center position of the cutout region is the optical center position.

[0034] For example, if the angle of view of the image after electronic image stabilization is 0.9 times the shooting angle of view, the information on the cropping magnification may be the cropping magnification of 0.9 itself, or may be a table number that can be converted into the cropping magnification.

[0035] Furthermore, the information on the roll angle may be, for example, the roll angle of 0.5 itself if the roll angle corrected by electronic image stabilization is 0.5 degrees, or may be a table number that can be converted to a roll angle.

[0036] Furthermore, the information on distortion may be, for example, distortion aberration determined by an interchangeable lens and the state (focal length, focus position) of that interchangeable lens, distortion parameters based on that, or information that can be converted into those.

[0037] However, if roll correction or distortion correction is performed together with electronic image stabilization during image capture, it becomes difficult to remove these correction components in subsequent processing (hereinafter referred to as post-processing).For this reason, when using an anamorphic lens and performing electronic image stabilization, it is preferable to limit (disable) correction processing such as roll correction and distortion correction.

[0038] The flowchart in Fig. 4 shows the process of restricting the above correction process when an anamorphic lens is used and electronic stabilization is performed. In step S401, the signal processing unit 106 as a processing means determines whether or not the execution of electronic stabilization has been selected (ON) by the user, and if it is ON, proceeds to step S402, and if not, proceeds to step S405. In step S405, the signal processing unit 106 adds meta information indicating that electronic stabilization is OFF to the frame image and records it.

[0039] On the other hand, in step S402, the signal processing unit 106 determines whether or not the imaging lens 101 is an anamorphic lens, and if it is an anamorphic lens, skips step S403 and proceeds to step S404, and if not, proceeds to step S403.

[0040] In step S403, the signal processing unit 106 disables the roll correction and the distortion correction, and proceeds to the next step S404.

[0041] In step S404, the signal processing unit 106 adds information to be used in the above-mentioned post-processing as meta-information to each frame image and records it.

[0042] As described above, in this embodiment, when electronic image stabilization is ON, information used in post-processing of frame images after electronic image stabilization is added to each frame image and recorded. This makes it easier to perform post-processing such as desqueezing and correction processing based on the optical center position for distortion, vignetting, etc.

[0043] In addition, when an interchangeable lens is used as an anamorphic lens, if the compression ratio of the anamorphic lens cannot be automatically acquired through communication with the interchangeable lens, a compression ratio selected by a user operation may be recorded. In this case, it is preferable to provide a user interface that allows the user to select whether to use the recorded compression ratio or the compression ratio specified in post-processing, so that the recorded compression ratio is not inaccurate and post-processing cannot be handled. EXAMPLES

[0044] Next, a second embodiment of the present invention will be described with reference to FIG. 5. The basic configuration of the imaging device 100 is the same as that of the first embodiment. However, in this embodiment, a case will be described in which all functions that may affect post-processing are disabled when the imaging lens 101 is an anamorphic lens. If an interchangeable lens serving as an anamorphic lens does not have a communication function with the imaging device, the imaging device cannot detect that the lens is attached. In this case, the imaging device determines whether or not an anamorphic lens, the information of which cannot be acquired through communication, is attached in response to a user's selection of ON / OFF of desqueeze display, which is a horizontally stretched display process (display process including stretching process) of video data.

[0045] In step S501, the signal processing unit 106 determines whether or not it is possible to acquire information indicating the type of the interchangeable lens by communicating with the interchangeable lens, and if it is possible to acquire the information, the process proceeds to step S502, and if not, the process proceeds to step S503.

[0046] In step S503, the signal processing unit 106 determines whether or not the desqueeze display is set to ON. If it is set to ON, the process proceeds to step S504; if not, the process ends.

[0047] On the other hand, in step S502, the signal processing unit 106 determines whether or not an anamorphic lens is attached, and if it is attached, the process proceeds to step S504, and if not, the process ends.

[0048] In step S504, the signal processing unit 106 determines whether or not the execution of electronic image stabilization has been selected by the user (ON), and if it is ON, the process proceeds to step S505, and if not, the process ends.

[0049] In step S505, the signal processing unit 106 as a control means displays a warning (warning operation) about the possibility that correct imaging cannot be performed using the anamorphic lens, and further in the next step S506, displays a user interface for the user to select whether or not to turn electronic stabilization OFF, as shown in Fig. 6. As a result, if OFF of electronic stabilization is selected, the signal processing unit 106 proceeds to step S507, turns OFF electronic stabilization, and ends this process. On the other hand, if ON of electronic stabilization is selected, the signal processing unit 106 ends this process.

[0050] As described above, in this embodiment, by turning off (limiting) electronic image stabilization when an anamorphic lens is attached, it is possible to make it easier to perform correction processing based on the optical center position in post-processing to correct distortion, vignetting, etc.

[0051] In this embodiment, the case where electronic image stabilization is turned off has been described. However, as in the first embodiment, functions that become difficult to perform in post-processing by performing electronic image stabilization at the time of image capture, such as roll correction and distortion correction, may also be turned off.

[0052] In the above embodiments, the image processing device is built into the imaging device. However, a personal computer that performs electronic image stabilization and post-processing on the video data acquired from the imaging device may also be used as the image processing device. EXAMPLES

[0053] Next, a third embodiment of the present invention will be described with reference to Figs. 7(A) to (G). Here, we will explain how the information added to the recording video data (hereinafter also referred to as recorded video) in the above embodiment is used in the editing device. In each figure, the optical center of the imaging optical system is indicated by a black circle in order to make it easier to understand.

[0054] Fig. 7(A) shows an image of the subject when shooting. In contrast, when shooting with an anamorphic lens, the recorded image is recorded as image data compressed to the left and right as shown in Fig. 7(B). The editing device, which serves as an image processing device, performs a desqueeze process on this recorded image based on the optical center, thereby restoring the compressed recorded image to its original state and obtaining an image as shown in Fig. 7(C).

[0055] Furthermore, the editing device corrects the image distorted by the anamorphic lens to convert it into an image as shown in Fig. 7(D), and performs various color management processes on this to produce the video. The distortion correction data used at this time is currently calculated by the user capturing an image of a grid chart or the like and calculating the amount of distortion corresponding to various lens positions. Note that there are also cases where the distorted image itself has a "flavor" and a process of adding distortion to the completed video is performed.

[0056] Here, as can be seen from FIG. 7B, distortion of an anamorphic lens occurs with respect to the optical center position. Also, even with the left-right compression that is unique to anamorphic lenses, the compression ratio differs the further away from the optical center position. If electronic image stabilization is performed when a lens with this characteristic is attached to an imaging device, the optical center position will shift. Also, because a cropping process using electronic magnification is performed during the electronic image stabilization process, the resulting distortion will differ from the amount of distortion correction that the user previously acquired.

[0057] If the de-squeezing process in Fig. 7(C) or the distortion correction process in Fig. 7(D) is performed on this image as if the center of the recorded image were the optical center, the de-squeezing process or distortion correction process would not be performed correctly. For this reason, it is necessary to perform the process based on the cut-out position and cut-out magnification added to the meta information.

[0058] FIG. 7(E) shows a recorded image that has been electronically stabilized using an anamorphic lens. This recorded image is slightly enlarged compared to FIG. 7(B), and is an example of a frame that has been recorded with the optical center position shifted. In contrast, FIG. 7(F) shows an image that has been desqueezed while taking into account the cropping magnification and cropping position added to the meta information recorded in the frame of FIG. 7(E), i.e., the optical center position. Furthermore, FIG. 7(G) shows an image that has been subjected to distortion correction processing while taking into account the cropping magnification and cropping position added to the meta information recorded in the frame of FIG. 7(E), i.e., the optical center position as described above. These are images that have been properly desqueezed and distortion corrected.

[0059] The flowchart in Fig. 8 shows the flow of processing when de-squeezing and distortion correction are performed on a frame-by-frame basis in an editing device for video that has been subjected to electronic image stabilization processing and recorded using an anamorphic lens. The editing device has an information reading means and a post-processing means.

[0060] In step S801, the editing device determines whether meta-information such as the cut-out position and cut-out magnification related to electronic image stabilization is present in the recorded video. If meta-information is present, the process proceeds to step S802; if not, the process proceeds to step S803.

[0061] In step S802, the editing device reads meta information to obtain the cut-out position and cut-out magnification, determines the position to be regarded as the optical center position, and proceeds to step S804.

[0062] On the other hand, in step S803, the editing device regards the center position of the recorded video (frame image) as the optical center position, sets the enlargement magnification (desqueeze magnification) to 1, and proceeds to step S804.

[0063] In step S804, the editing device performs desqueeze processing based on the determined center position and desqueeze magnification to restore the compression of the recorded video, and proceeds to step S805. At this time, if the compression ratio of the anamorphic lens is recorded in the meta information, the reciprocal of the compression ratio may be used as the desqueeze magnification, or the user may be allowed to specify an arbitrary desqueeze magnification. Also, if the lens attached to the imaging device is not an anamorphic lens, a desqueeze process of 1x can be performed, which is equivalent to no processing at all.

[0064] In step S805, the editing device performs distortion correction processing based on the determined optical center position and desqueeze magnification. Here, the correction processing is not limited to distortion correction processing, and other correction processing may be performed based on the determined optical center position and desqueeze magnification. Also, in this embodiment, only electronic image stabilization is mentioned, but if an optical image stabilization function or an image pickup surface image stabilization function is provided, it is desirable to record the position information thereof as meta information. (Other Examples) The above-described embodiments are merely representative examples, and various modifications and changes are possible for each embodiment when implementing the present invention. For example, in the above-described embodiments, information on the cut-out position is used as information indicating the optical center position, but information on the optical center position in the cut-out image itself may be used.

[0065] For example, in a 3840 x 2160 pixel frame image, if the center position of the cropping position and the optical center position coincide, the coordinates of the optical center position in the cropped image are (1920, 1080). On the other hand, if the cropping position is shifted above the sensor (the imaging device is tilted downward), the coordinates of the optical center position in the cropped image will be shifted from the center of the cropped image, such as (1920, 1500). Information on these coordinates may be added to the video data as information indicating the optical center position in each frame image.

[0066] Also, instead of these coordinates, an offset position (0, 420) from the center position of the cut-out image may be used. Note that (0, 420) also indicates that the cut-out position is shifted upward.

[0067] In the above embodiment, the image pickup device has been described as having an electronic image stabilization function, but the image stabilization method is not limited to this. As described above, it is preferable to perform the correction process based on the optical center position during the extension process, the distortion correction process, the vignetting correction process, etc.

[0068] On the other hand, even when performing sensor vibration isolation, which moves the image sensor in a direction intersecting the optical axis, or lens vibration isolation, which moves a correction lens in a direction intersecting the optical axis, it is difficult to identify the optical center position from the video data. Therefore, by adding information indicating the optical center position of each frame image to the video data as in the above embodiment, it is possible to make it easier to perform expansion processing and correction processing in the post-processing step using the optical center position.

[0069] It is also possible to use a plurality of types of image stabilization in combination. For example, when electronic image stabilization and sensor image stabilization are used in combination, information on the cut-out position and the position of the image sensor may be added to each frame image, and the editing device may determine the optical center position based on the cut-out position and the position of the image sensor. In this way, information that can identify the optical center position, even if it is not the optical center position itself, is information that indicates the optical center position.

[0070] In the above-described embodiment, an example of performing correction processing such as distortion correction processing and vignetting correction processing has been described. However, even if these corrections are not performed, the extension processing can be performed using the information on the optical center position, so that the effect of adding the information on the optical center position in each frame image can be obtained.

[0071] Furthermore, the present invention can be applied to imaging devices that do not have a vibration-proof function. For example, in addition to electronic vibration proofing, there is a function for cutting out a part of a captured image and using the cut-out image as an image for recording, such as a function for changing the cut-out position according to the movement of the subject in order to track the subject. The present invention can also be applied to cases where the optical center position of each frame image moves due to these functions.

[0072] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Explanation of symbols]

[0073] 100 Imaging device 101 Imaging lens 106 Signal Processing Section 108 Image cropping control section

Claims

1. a video data generating means for generating video data for recording by capturing an image via an imaging optical system; and an information adding means for adding, when the imaging optical system is an anamorphic optical system, information regarding a compression ratio of the anamorphic optical system and information indicating a center position of the anamorphic optical system in each frame image constituting the recording video data to the recording video data.

2. the video data generating means includes electronic image stabilization means for performing electronic image stabilization for each frame image of the captured video data generated by imaging, the electronic image stabilization being configured to cut out a partial image area of ​​the frame image of the captured video data so as to reduce image shake; Each frame image of the recording video data is data of an image area cut out by the electronic image stabilization, 2. The image processing apparatus according to claim 1, wherein said information adding means adds information regarding a cropping magnification of said image area to said video data for recording.

3. 3. The image processing apparatus according to claim 2, wherein the information indicating the center position is information indicating a cut-out position of the image area.

4. 4. The image processing apparatus according to claim 2, wherein said information adding means further adds information about a roll angle of a frame image of said record video data and information about distortion of said frame image to said record video data.

5. The information adding means includes: An image processing device according to any one of claims 1 to 4, characterized in that when information acquired by communication from an interchangeable lens having the imaging optical system indicates that the imaging optical system is an anamorphic optical system, information regarding the compression ratio and information indicating the center position are added to the recording video data.

6. The image processing device further includes a processing unit for performing at least one of roll correction and distortion correction on the frame images of the recording video data, 6. The image processing apparatus according to claim 1, wherein the processing means limits the correction process when the imaging optical system is an anamorphic optical system.

7. 7. The image processing apparatus according to claim 1, wherein the information regarding the compression ratio is information based on a compression ratio input by a user.

8. a video data generating means for generating video data for recording by performing electronic image stabilization for each frame image constituting captured video data generated by imaging through an imaging optical system, the image stabilization being performed by cutting out a partial image area of ​​the frame image so as to reduce image shake; a processing means for performing at least one of roll correction and distortion correction on the frame images of the recording video data, The image processing device according to claim 1, wherein the processing means limits the correction process when the imaging optical system is an anamorphic optical system.

9. an electronic image stabilization unit that performs electronic image stabilization for each frame image constituting captured image data generated by imaging through an imaging optical system, by cutting out a partial image area of ​​the frame image so as to reduce image shake, thereby generating image data for recording; 23. An image processing apparatus comprising: a control unit that limits the electronic image stabilization or performs a warning operation when the imaging optical system is an anamorphic optical system.

10. A user can select whether or not to perform decompression processing on the recording video data, 9. The image processing apparatus according to claim 1, wherein the image capturing optical system is determined to be the anamorphic optical system when a selection is made to perform the expansion process.

11. an imaging element that captures an optical image formed by the imaging optical system; An imaging device comprising the image processing device according to claim 1 .

12. 12. The imaging apparatus according to claim 11, further comprising a sensor vibration isolation means for moving the imaging element in a direction intersecting an optical axis of the imaging optical system.

13. the imaging optical system has a correction lens that is movable in a direction intersecting the optical axis so as to move the optical axis, 13. The imaging apparatus according to claim 11, further comprising a control unit for controlling movement of the correction lens in a direction intersecting the optical axis.

14. an information reading means for reading information added to the recording video data generated by imaging; and a post-processing means for performing an expansion process on the recording video data as post-processing, based on information regarding the compression rate of the anamorphic optical system used for imaging, read out by the information reading means, and information indicating a center position of the anamorphic optical system in each frame image constituting the recording video data.

15. the information reading means further reads information regarding at least one of a roll angle and a distortion as information added to the recording video data; The image processing device according to claim 14, characterized in that the post-processing means performs, as post-processing, correction processing corresponding to the information regarding at least one of roll correction and distortion correction for the frame image based on the information regarding the at least one, the information regarding the compression ratio, and the information indicating the center position.

16. 16. An image processing device as claimed in claim 14 or 15, characterized in that in the post-processing, a user interface is displayed that allows the user to select whether to use the compression ratio read out by the information reading means or a compression ratio specified separately from the read compression ratio.

17. generating video data for recording by capturing an image via an imaging optical system; an image processing method comprising the step of, when the imaging optical system is an anamorphic optical system, adding to the recording video data information regarding a compression ratio of the anamorphic optical system and information indicating a center position of the anamorphic optical system in each frame image constituting the recording video data.

18. generating video data for recording by performing electronic image stabilization for each frame image constituting captured video data generated by imaging through an imaging optical system, the image stabilization comprising cutting out a partial image area of ​​the frame image so as to reduce image shake; performing at least one of roll correction and distortion correction on the frame images during the image capture; An image processing method, comprising: limiting the correction process when the imaging optical system is an anamorphic optical system.

19. performing electronic image stabilization for each frame image constituting captured video data generated by imaging via an imaging optical system, in which a partial image area of ​​the frame image is cut out so as to reduce image shake; and limiting the electronic image stabilization or performing a warning operation when the imaging optical system is an anamorphic optical system.

20. A step of reading information added to the recording video data generated by imaging; and performing an expansion process on the recording video data as post-processing based on information regarding the compression rate of the anamorphic optical system used for imaging, which was read out in the step, and information indicating the center position of the anamorphic optical system in each frame image constituting the recording video data.

21. A computer program causing a computer to execute a process according to the image processing method according to any one of claims 17 to 20.

Images