Focusing apparatus and method, imaging apparatus, imaging system, program, and storage medium

Abstract

To prevent unintended out-of-focus due to gimbal tracking, especially in moving image photographing.SOLUTION: A focusing apparatus has: detection means that holds an imaging unit including an optical system including a focus lens and imaging means, and detects the rotation angle of holding means including revolving means that drives to revolve the imaging unit; focusing means that controls the position of the focus lens to perform focusing; and calculation means that, when the detection means detects the rotation angle of the holding means, determines a correction amount for correcting a shift of a focal position according to the amount of change in distance between the imaging unit corresponding to the rotation angle and a focused subject. The focusing means can perform focusing in an AF mode for focusing on the subject on the basis of a signal obtained from the imaging means. The calculation means determines the correction amount when the focusing is not performed in the AF mode and the subject is tracked by using the revolving means. The focusing means controls the position of the focus lens on the basis of the correction amount.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a focus adjustment device and method, an imaging device, an imaging system, a program, and a storage medium.

Background Art

[0002] Conventionally, there has been a case where an imaging device is mounted on a gimbal that can rotate in the pan / tilt / roll directions for shooting. Many imaging devices mounted on such a gimbal are equipped with a function to track a subject by controlling the gimbal movable part so as to continuously capture the main subject detected by the imaging device (hereinafter referred to as "gimbal tracking").

[0003] On the other hand, particularly in video shooting, there are needs such as in conditions where autofocus is difficult, such as low brightness and low contrast, and when it is desired to fix the focus with manual focus when it is not desired to accidentally move the focus.

[0004] However, in the combination of gimbal shooting and manual focus, there is a problem that due to gimbal tracking, the distance between the imaging device and the subject unintentionally changes, resulting in out-of-focus.

[0005] In Patent Document 1, a method for correcting defocus according to the movement amount of a camera during focus lock is disclosed.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] However, the prior art disclosed in Patent Document 1 obtains the moving distance of the camera by a three-axis acceleration sensor after autofocusing by half-pressing the shutter button, and performs focus correction during still image shooting. Therefore, it cannot cope with focus correction during video shooting when it is desired to always keep the focus.

[0008] Also, although the linear moving distance of the camera can be calculated by a three-axis acceleration sensor, it is impossible to calculate the change in subject distance (so-called cosine error) associated with the arc movement due to gimbal tracking.

[0009] The present invention has been made in view of the above problems, and particularly aims to suppress unintentional focus shift due to gimbal tracking during video shooting.

Means for Solving the Problems

[0010] In order to achieve the above object, a focus adjustment device of the present invention holds an imaging unit including an optical system including a focus lens and an imaging means, and has a first detection means for detecting a rotation angle of a holding means for pivotally driving the imaging unit, a focus adjustment means for controlling the position of the focus lens to perform focus adjustment, and an arithmetic means for obtaining a correction amount for correcting a deviation of the in-focus position according to a change amount of the distance between the imaging unit and the subject adjusted in focus corresponding to the rotation angle when the rotation angle of the holding means is detected by the first detection means. The focus adjustment means can perform focus adjustment in an AF mode of focusing on the subject based on a signal obtained from the imaging means, and the arithmetic means obtains the correction amount when the focus adjustment is not performed in the AF mode and the subject is being tracked using the pivot means, and the focus adjustment means controls the position of the focus lens based on the correction amount.

Effect of the Invention

[0011] According to the present invention, it is possible to suppress unintentional focus shift due to gimbal tracking, particularly during video shooting.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Mode for Carrying Out the Invention

[0013] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0014] <First Embodiment> FIG. 1 is a block diagram showing a functional configuration example of a digital camera as an example of an imaging device according to the present embodiment. The digital camera of the present embodiment is a lens-integrated camera and includes a lens unit 10 as an imaging optical system and a camera unit 20 for performing shooting. Further, the digital camera of the present embodiment is integrated with a gimbal, and the lens unit 10 and the camera unit 20 (imaging unit) are assumed to be mounted on the gimbal unit 30.

[0015] Here, although the lens unit 10, the camera unit 20, and the gimbal unit 30 are described as being integrally configured, the present invention is not limited to this, and may be configured as an imaging system in which some or all are detachable. In that case, when each configuration is connected, signal lines for exchanging mutual information are also connected. Also, a part of the configuration of the gimbal unit 30 shown below may be configured in the camera unit 20.

[0016] In the present embodiment, a digital camera is taken as an example of the imaging device for description, but any electronic device having a camera function may be used. For example, other cameras such as a digital video camera may be used, or a mobile phone with a camera, a computer with a camera, a game machine, etc. may also be used.

[0017] (Lens unit 10) The lens unit 10 is an imaging lens having an optical system including a first lens group 101, a diaphragm 102, a second lens group 103, and a focus lens group (hereinafter simply referred to as "focus lens") 104, and a drive / control system, and forms an optical image of a subject.

[0018] The first lens group 101 is disposed at the tip of the lens unit 10 and is held so as to be movable in the optical axis direction. The aperture 102 has a function of adjusting the amount of light during shooting by adjusting its aperture diameter. The aperture 102 and the second lens group 103 are integrally movable in the optical axis direction, and move in conjunction with the first lens group 101 to realize a zoom function. Also, the focus lens 104 is movable in the optical axis direction, and the distance (focal length) to the subject on which the lens unit 10 is focused changes according to its position. Focus adjustment for adjusting the focal length of the lens unit 10 is performed by controlling the position of the focus lens 104 in the optical axis direction.

[0019] The drive / control system includes a zoom actuator 105, an aperture actuator 106, and a focus actuator 107. It also includes a zoom drive circuit 108, an aperture drive circuit 109, a focus drive circuit 110, and a lens control unit 111.

[0020] The zoom drive circuit 108 drives the zoom actuator 105 to drive the first lens group 101 and the second lens group 103 in the optical axis direction, thereby controlling the angle of view of the optical system of the lens unit 10. The aperture drive circuit 109 drives the aperture 102 using the aperture actuator 106 to control the aperture diameter of the aperture 102. The focus drive circuit 110 drives the focus lens 104 in the optical axis direction using the focus actuator 107 to change the focal length of the optical system of the lens unit 10. Also, the focus drive circuit 110 detects the current position of the focus lens 104 using the focus actuator 107.

[0021] The lens control unit 111 controls the zoom drive circuit 108, the aperture drive circuit 109, and the focus drive circuit 110. Also, the lens control unit 111 communicates with the camera control unit 204. For example, the lens control unit 111 detects the position of the focus lens 104 on the optical axis in response to a request from the camera control unit 204, and notifies the camera control unit 204 of the focus lens position information. Also, the lens control unit 111 controls the zoom drive circuit 108, the aperture drive circuit 109, and the focus drive circuit 110 in response to a processing instruction from the camera control unit 204.

[0022] (Camera unit 20) The camera unit 20 has an image sensor 201 and controls the entire camera including the lens unit 10 and the gimbal unit 30.

[0023] In this embodiment, the image sensor 201 is composed of a CMOS image sensor and peripheral circuits, and m pixels in the horizontal direction and n pixels in the vertical direction (m and n are integers of 2 or more) are arranged. Further, the image sensor 201 of this embodiment has a pupil division function and can perform phase difference AF (auto focus) using image data.

[0024] Here, the configuration of the image sensor 201 in this embodiment will be described. FIG. 2(a) is a diagram schematically showing the pixel arrangement of the image sensor 201 in this embodiment, showing a state of observing a range of 6 rows in the vertical (Y direction) and 8 columns in the horizontal (X direction) of a two-dimensional CMOS area sensor from the lens unit 10 side. The image sensor 201 is provided with a color filter of a Bayer array, and color filters of red (R) and green (G) are alternately arranged in order from the left for the pixels in odd rows, and color filters of green (G) and blue (B) are alternately arranged in order from the left for the pixels in even rows.

[0025] FIG. 2(b) is a diagram showing the pixel 211R. The circle 211i represents an on-chip microlens, and the plurality of rectangles 211A and 211B arranged inside the on-chip microlens 211i each represent a photoelectric conversion section. The pixels 211Gr, 211Gb, and 211B have the same configuration.

[0026] Further, in the image sensor 201 of this embodiment, there are pixels (211R, 211Gr, 211Gb, 211B) in which the photoelectric conversion section of the imaging pixel is divided into two in the X direction. The photoelectric conversion signals corresponding to the individual photoelectric conversion sections can be used as data for phase difference AF or for generating a parallax image constituting a 3D (3-Dimensional) image. Also, the sum of the photoelectric conversion signals can be used as normal photographed image data.

[0027] Here, the pixel signals in the case of performing phase difference AF will be described. In the present embodiment, in the configuration shown in FIG. 2(b), the micro lens 211i, the divided photoelectric conversion units 211A and 211B are configured to be able to split the emitted light beam of the imaging optical system in terms of pupil. By using the photoelectric conversion unit 211A and the photoelectric conversion unit 211B in FIG. 2(b) as a pair, focus detection based on the image shift amount (phase difference) in the X direction can be performed.

[0028] More specifically, first, in FIG. 2(a), the signals of the photoelectric conversion unit 211A arranged in a plurality of pixels 211R, 211Gr, 211Gb, 211B within a predetermined range arranged in the same pixel row are collected to generate an A image signal for AF. Similarly, the signals of the photoelectric conversion unit 211B are collected as a B image signal for AF, and a pair of image signals are generated. At this time, as the output of the photoelectric conversion unit 211A and the photoelectric conversion unit 211B, a pseudo luminance (Y) signal calculated by adding the outputs of green, red, blue, and green included in the unit arrangement of the color filter is used. However, the A image signal and the B image signal for AF may be organized for each color of red, blue, and green.

[0029] For the A image signal and the B image signal generated in this way, a correlation operation is performed as described later, and the image shift amount [bit] can be obtained based on the obtained correlation amount. The camera control unit 204 can detect the defocus amount [mm] of a predetermined region by multiplying the obtained image shift amount by a conversion coefficient.

[0030] Note that in the above-described example, the case where the photoelectric conversion unit of the imaging pixel is divided into two in the X direction has been described, but it is not limited thereto. For example, it may be divided in the Y direction, or pixels divided in the X direction and the Y direction may be mixed. In addition, although various methods of dividing the photoelectric conversion unit are conceivable, since it is possible to use known techniques, detailed description is omitted here.

[0031] Returning to the description of FIG. 1, the imaging element driving circuit 202 controls the operation of the imaging element 201, and performs A / D conversion and gain processing on the acquired image signal and the signal for phase difference AF, and transmits them to the camera control unit 204.

[0032] The image processing circuit 203 performs general image processing performed in a digital camera, such as γ conversion, white balance adjustment processing, color interpolation processing, compression encoding processing, etc., on the image signal output from the imaging element driving circuit 202. Then, it generates image data for recording, an image signal for display, and a pair of image signals (A image signal and B image signal) for phase difference AF.

[0033] Also, the camera control unit 204 performs all operations and controls related to the camera unit 20, and controls the imaging element driving circuit 202 and the image processing circuit 203. As one of them, the camera control unit 204 performs photometry using the image signal output from the imaging element driving circuit 202, and based on the obtained photometry value, adjusts the aperture diameter of the diaphragm 102, adjusts the exposure time of the imaging element 201, and adjusts the gain value in the imaging element driving circuit 202. Note that photometry may be performed by separately providing an exposure meter instead of performing photometry using the image signal.

[0034] Also, the camera control unit 204 makes a request to the lens control unit 111 for acquiring the focus lens position, changing the aperture 102, the focus lens 104, the zoom magnification by a predetermined driving amount, and acquiring optical information unique to the lens unit 10, etc.

[0035] The camera control unit 204 incorporates a ROM (Read Only Memory) 204a storing a program for controlling the camera operation, a RAM (Random Access Memory) 204b for storing variables, and an EEPROM (Electrically Erasable Programmable Read-Only Memory) 204c for storing various parameters and various setting information of the camera unit 20 set by the user.

[0036] In addition, the camera control unit 204 receives processing commands from the user via the operation unit 303 through the gimbal control unit 301 of the gimbal unit 30 in order to issue control commands to the lens control unit 111 and change the operations of the imaging element drive circuit 202 and the image processing circuit 203. Also, via the gimbal control unit 301, it displays an image signal on the display 302 and records an imaging signal in the memory 304.

[0037] The focus detection unit 205 performs phase difference detection type focus detection processing using a pair of image signals (A image signal and B image signal) for phase difference AF obtained by the image processing circuit 203, and detects the defocus amount based on the deviation amount between the A image signal and the B image signal. The camera control unit 204 performs focus adjustment by adjusting the position of the focus lens 104 via the lens control unit 111 so that the defocus amount becomes smaller.

[0038] The subject detection unit 206 performs subject detection based on the image data obtained by the image processing circuit 203. In this embodiment, the camera control unit 204 uses the subject detection that estimates the target subject and detects its position in the image data to select the focus adjustment result by the focus detection unit 205 for driving the focus lens 104 via the lens control unit 111. Examples of the subject to be detected include a person's face and the pupils included therein, an animal's body and the face / pupils included therein, and the entire vehicle and the characteristic parts included therein (such as the vehicle operator and the cockpit), but are not limited thereto. Also, a subject existing at a position specified by the user within the imaging screen may be detected by a touch operation or the like by the user on the display 302.

[0039] (Gimbal unit 30) The gimbal unit 30 includes a gimbal control unit 301, a display 302, an operation unit 303, a memory 304, an inertial measurement unit 305, a yaw-axis turning mechanism 306, a pitch-axis turning mechanism 307, and a roll-axis turning mechanism 308.

[0040] The gimbal control unit 301 controls the operations of each part of the gimbal unit 30. It also exchanges information with the camera control unit 204 and provides information for changing the operations of the imaging element driving circuit 202 and the image processing circuit 203. Further, it can control the zoom driving circuit 108, the aperture driving circuit 109, and the focus driving circuit 110 via the camera control unit 204 with respect to the lens control unit 111. Also, it receives the image signal for display from the camera control unit 204 and performs display on the display 302 and recording in the memory 304.

[0041] The display 302 is composed of an LCD (liquid crystal display) or the like, and displays information regarding the shooting mode of the camera, the preview image before shooting and the confirmation image after shooting, the in-focus state display image at the time of focus detection, and the like.

[0042] The operation unit 303 is composed of a power switch, a focus adjustment start switch, a shooting trigger switch, a zoom operation switch, a gimbal operation switch, a touch panel configured on the display 302, and the like. Also, by operating the operation unit 303, it is possible to switch between the AF mode and the MF mode and give a drive instruction to the focus lens 104 in the MF mode. Also, various operations can be performed by selecting and operating regions of various function icons and images displayed on the display 302 via the touch panel. The memory 304 is a removable flash memory and records the image data obtained by shooting.

[0043] The inertial measurement unit 305 has an acceleration sensor and a gyro sensor, and detects the acceleration in the three axial directions of front-rear, left-right, and up-down of the entire imaging device including the lens unit 10, the camera unit 20, and the gimbal unit 30. Also, it detects the angular velocity in the three axial directions of the yaw axis, pitch axis, and roll axis. By integrating the detected angular velocity over time, the rotation angle (rotation angle) can be obtained.

[0044] The yaw axis turning mechanism 306 has a yaw axis motor that turns and drives the lens unit 10 and the camera unit 20 in the yaw axis direction. The pitch-axis turning mechanism 307 has a pitch-axis motor that turns the lens unit 10 and the camera unit 20 in the pitch-axis direction. The roll-axis turning mechanism 308 has a roll-axis motor that turns the lens unit 10 and the camera unit 20 in the roll-axis direction.

[0045] The gimbal control unit 301 controls the yaw-axis turning mechanism 306, the pitch-axis turning mechanism 307, and the roll-axis turning mechanism 308 in order to suppress camera shake during shooting by the photographer. Also, in response to an operation of a gimbal operation switch included in the operation unit 303, when the photographer gives a drive command to the yaw-axis turning mechanism 306, the pitch-axis turning mechanism 307, or the roll-axis turning mechanism 308, control corresponding to the command is performed. Further, when gimbal tracking is set, in order to keep the subject detected by the subject detection unit 206 within the imaging screen, the yaw-axis turning mechanism 306, the pitch-axis turning mechanism 307, and the roll-axis turning mechanism 308 are controlled for framing. Gimbal tracking refers to an operation in which, when the main subject is detected by the subject detection unit 206, even when the photographer moves the camera unit 20 in an arbitrary direction, for example, as shown in FIGS. 4(a) to 4(b) or FIGS. 4(a) to 4(c), to change the framing, the yaw-axis turning mechanism 306 or the pitch-axis turning mechanism 307 is controlled to automatically change the orientations of the lens unit 10 and the camera unit 20 so as to keep the main subject within the shooting angle of view.

[0046] (Focus adjustment process) Next, the focus adjustment process in this embodiment will be described. Note that the focus adjustment process described below is implemented by the camera control unit 204 in the camera unit 20 executing an imaging processing program stored in the ROM 204a.

[0047] FIG. 3 is a flowchart showing the focus adjustment process. Note that in this embodiment, the focus adjustment process during moving image shooting will be described, but it may also be performed during the LV operation in still image shooting.

[0048] First, in S301, the camera control unit 204 determines, via the gimbal control unit 301, whether the AF mode is set by an operation of the operation unit 303 or a touch operation on the display 302. If it is the AF mode, the process proceeds to S302; otherwise, it proceeds to S304.

[0049] In S302, the camera control unit 204 causes the focus detection unit 205 to perform focus detection processing. In the focus detection processing, defocus amount and reliability information for performing imaging plane phase difference AF are acquired. The area within the imaging screen for acquiring a pair of image signals for phase difference AF is set according to the state of the camera unit 20. Details of the focus detection processing performed here will be described later.

[0050] Subsequently, in S303, the camera control unit 204 controls the focus lens 104 based on the defocus amount calculated in the focus detection processing of S302, performs AF control to follow the focus on the subject, and returns to S301. Thereby, while the AF mode is set, it is possible to continuously maintain the focus on an arbitrary subject.

[0051] On the other hand, when the AF mode is not set, assuming that the MF (manual focus) mode is set, the processing after S304 is performed. First, in S304, the camera control unit 204 determines whether a specific subject (main subject) such as a person's face is detected by the subject detection unit 206. If a subject is detected, the process proceeds to S305; otherwise, it proceeds to S310. Regarding the method for determining the main subject, there are pressing the shooting direction switching button of the operation unit 303, touching the shooting direction switching icon on the touch panel of the display 302, etc., but other methods may also be used.

[0052] When a subject is detected, in S305, the camera control unit 204 determines, via the gimbal control unit 301, whether the gimbal tracking mode is enabled by an operation of the operation unit 303 or a touch operation on the display 302. If the gimbal tracking mode is enabled, the process proceeds to S306; otherwise, it proceeds to S310. When the gimbal tracking mode is enabled, in S306, the camera control unit 204 causes the gimbal to perform tracking via the gimbal control unit 301.

[0053] Subsequently, in S307, the camera control unit 204 determines whether the angular velocity in the yaw direction and / or pitch direction of the camera unit 20 is detected by the inertial measurement unit 305 during gimbal tracking. If the angular velocity is detected, the process proceeds to S308; otherwise, it proceeds to S310.

[0054] When the photographer moves the camera unit 20 in the yaw direction or pitch direction, for example, as shown in FIGS. 4(a) to 4(b) and FIGS. 4(a) to 4(c), an arc motion (rotation) occurs centered on the photographer, and strictly speaking, the distance between the camera unit 20 and the subject changes.

[0055] FIG. 5 is a diagram for explaining the relationship between the position of the camera unit 20 before and after the arc motion (rotation) and the distance to the subject. In a sphere with the length r of the grip including the gimbal unit 30 as the radius, when the camera moves from the position A on the surface of the sphere with the declination angle θ A to the position B on the circle on the surface of the sphere with the declination angle θ B a difference of Δd (= distance OA - distance OB) occurs in the distance to the subject O. Thus, in the MF mode where the focus is fixed for shooting, there is a problem that the focus is shifted by that amount due to the arc motion.

[0056] Therefore, when the angular velocity is detected, in S308, the camera control unit 204 calculates the change amount Δd in order to correct the defocus caused by the arc movement of the camera unit 20. Here, the method for obtaining the change amount Δd will be described in detail with reference to FIG. 6. As described above, the change amount Δd can be expressed by Equation (1). In the following equations, the distance OA is denoted as (OA), and the distance OB is denoted as (OB).

[0057] Δd = (OA) - (OB) …(1) Since the distance OA is the distance to the subject being photographed, it can be obtained from the amount of image shift, the amount of defocus, the position of the focus lens 104, etc. Therefore, if the distance OB is known, Δd can be obtained.

[0058] Here, as shown in FIG. 6(a), in the triangle composed of the subject O, the center C of the arc movement, and the position B of the camera after movement, the distance OB can be expressed as follows by the cosine theorem using the lengths of the sides and the angles:

[0059] (OB) 2 = (BC) 2 + (OC) 2 - 2(BC)(OC) cosθ B …(2) Among the terms in the above Equation (2), (BC) is r, and (OC) can be obtained in advance by the cosine theorem from triangle OAC if the angle of angle CAO is known. As shown in FIG. 6(b), since the camera unit 20 is facing the subject due to gimbal tracking, the angle of angle CAO can be obtained by adding 90° to the angle α formed by the camera unit 20 and the gimbal unit 30 obtained from the yaw-axis turning mechanism 306 and the pitch-axis turning mechanism 307 in the state where the camera unit 20 is at position A. Also, the deflection angle θ B at the position B after movement can be obtained by integrating the angular velocity obtained from the inertial measurement unit 305 with respect to time.

[0060] Substituting Equation (2) into Equation (1), Δd = (OA) - √((BC) 2 + (OC)2 - 2(BC)(OC) cosθ B ) …(3) This results in the ability to obtain the change amount Δd of the distance to the subject.

[0061] Subsequently, in S309, the camera control unit 204 converts the change amount Δd obtained in S308 into a focus correction amount, and drives the focus lens 104 via the lens control unit 111. As a result, it is possible to correct the defocus caused by the circular motion of the camera unit 20 during gimbal tracking. Note that since the process shown in FIG. 3 is executed for each frame, it is desirable to drive the focus lens 104 in synchronization with the control of the yaw-axis turning mechanism 306 and / or the pitch-axis turning mechanism 307 due to gimbal tracking in S306 of the next frame for the focus correction here.

[0062] On the other hand, if it is not determined in S304 that the subject has been detected, or if it is not determined in S305 that the gimbal tracking mode is valid, or if the angular velocity is not detected in S307, then in S310, the camera control unit 204 maintains the state where the focus is fixed according to the setting of the MF mode.

[0063] (Focus detection process) Next, the details of the focus detection process executed by the focus detection unit 205 in S302 will be described. FIG. 7 is a flowchart illustrating the flow of the focus detection process.

[0064] First, in S701, the camera control unit 204 acquires a pair of image signals for AF from a plurality of pixels included in the focus detection area in the image sensor 201. FIG. 8 shows an example of the focus detection area 802 set on the pixel array 801 of the image sensor 201. The shift areas 803 on both sides of the focus detection area 802 are areas necessary for the correlation operation. Therefore, the area 804 combining the focus detection area 802 and the shift areas 803 is the pixel area necessary for the correlation operation. In the figure, p, q, s, and t respectively represent coordinates in the horizontal direction (x-axis direction), p and q are the x-coordinates of the start point and the end point of the pixel area 804, and s and t are the x-coordinates of the start point and the end point of the focus detection area 802, respectively.

[0065] Also, FIG. 9(a) shows an example of a pair of image signals for AF obtained based on the signals acquired from the plurality of pixels included in the focus detection area 802 shown in FIG. 8. The solid line 901 is one A image signal, and the broken line 902 is the other B image signal. FIG. 9(a) shows the A image signal 901 and the B image signal 902 before the shift, and FIGS. 9(b) and 9(c) show the states in which the A image signal 901 and the B image signal 902 are shifted in the plus direction and the minus direction from the state of FIG. 9(a) for calculating the correlation amount in S702, respectively.

[0066] Next, in S702, the focus detection unit 205 calculates the correlation amount of the acquired pair of image signals while relatively shifting them one pixel (1 bit) at a time. In each of a plurality of pixel lines (hereinafter referred to as "scanning lines") provided in the focus detection area, as shown in FIGS. 9(b) and 9(c), both the A image signal 901 and the B image signal 902 are shifted one bit at a time in the direction of the arrow, and the sum of the absolute values of the differences between the A image signal 901 and the B image signal 902 in the focus detection area 802 (from s to t) set in each shift state is calculated. Here, the maximum shift amount in the minus direction is p - s, and the maximum shift amount in the plus direction is q - t. Also, when the shift amount is represented by i, x is the start coordinate of the focus detection area 802, and y is the end coordinate of the focus detection area 802, the correlation amount COR can be calculated by the following formula (4). TIFF2025098719000002.tif2087 In this way, for each pixel line, the correlation amount between a pair of A image signals 901 and B image signals 902 is calculated, and by adding and averaging the respective correlation amounts, one correlation amount is calculated.

[0067] Here, the description has been given assuming that a pair of image signals are relatively shifted by one pixel at a time when calculating the correlation amount. However, for example, a configuration in which they are relatively shifted in units of more pixels, such as two pixels at a time, may also be used. Also, the description has been given assuming that one correlation amount is calculated by adding and averaging the correlation amounts of each scanning line. However, for example, a configuration in which addition and averaging are performed on a pair of image signals of each scanning line, and then the correlation amount is calculated for the pair of image signals that have been added and averaged, may also be used.

[0068] FIG. 10(a) is a diagram showing an example of the relationship between the shift amount and the correlation amount COR. The horizontal axis represents the shift amount, and the vertical axis represents the correlation amount COR. Among the vicinity of the extreme values 1002 and 1003 in the correlation amount 1001 that changes with the shift amount, the degree of coincidence between the A image signal and the B image signal is the highest at the shift amount corresponding to the smaller correlation amount.

[0069] Subsequently, in S703, the focus detection unit 205 calculates a correlation change amount from the correlation amount calculated in S702. The difference between the correlation amounts at every other shift in the waveform of the correlation amount 1001 shown in FIG. 10(a) is calculated as the correlation change amount. Assuming the shift amount is i, the minimum shift amount is p - s, and the maximum shift amount is q - t, the correlation change amount ΔCOR can be calculated by the following equation (5). TIFF2025098719000003.tif1484Next, in S704, the focus detection unit 205 calculates the amount of image shift using the amount of change in correlation calculated in S703. FIG. 11(a) shows an example of the relationship between the shift amount and the amount of change in correlation ΔCOR. The horizontal axis represents the shift amount, and the vertical axis represents the amount of change in correlation ΔCOR. The amount of change in correlation 1101 that changes with the shift amount becomes positive to negative in parts 1102 and 1103. The state where the amount of change in correlation becomes 0 is called zero cross, and in this state, the degree of coincidence between the A image signal and the B image signal is the highest. Therefore, the shift amount that gives the zero cross becomes the amount of image shift.

[0070] FIG. 11(b) is an enlarged view of part 1102 in FIG. 11(a), and 1104 is a part of the amount of change in correlation 1101. First, the shift amount (k - 1 + α) that gives the zero cross is divided into an integer part β (= k - 1) and a fractional part α. The fractional part α can be calculated by the following formula (6) from the similarity relationship between triangle ABC and triangle ADE in FIG. 11(b). TIFF2025098719000004.tif25119 On the other hand, the integer part β can be calculated by the following formula (7) from FIG. 11(b).

[0071] β = k - 1 …(7) From the sum of α and β obtained as described above, the amount of image shift PRD can be calculated.

[0072] Also, as shown in FIG. 11(a), when there are multiple zero crosses of the amount of change in correlation ΔCOR, the one with the greater steepness of the change in the amount of change in correlation ΔCOR in the vicinity thereof is set as the first zero cross. This steepness is an index indicating the ease of specifying the coincidence position, and indicates that the coincidence position can be specified with higher accuracy as the value is larger. The steepness maxder can be calculated by the following formula (8).

[0073] maxder = |ΔCOR[k - 1]| + |ΔCOR[k]| …(8) As described above, in this embodiment, when there are a plurality of zero crossings of the correlation change amount, the first zero crossing is determined based on its steepness, and the shift amount that gives this first zero crossing is defined as the image shift amount.

[0074] Subsequently, in S705, the focus detection unit 205 calculates the reliability representing the high reliability of the image shift amount calculated in S704. The reliability of the image shift amount can be defined by the degree of coincidence (hereinafter referred to as the two-image coincidence degree) fnclvl between the A image signal and the B image signal, which are a pair of image signals, and the steepness of the correlation change amount described above. The two-image coincidence degree is an index representing the accuracy of the image shift amount, and here, the smaller the value, the better the accuracy. FIG. 10(b) is an enlarged view of the portion indicated by 1002 in FIG. 10(a), and 1004 is a part of the correlation amount 1001. The two-image coincidence degree fnclvl can be calculated by the following formula (9). TIFF2025098719000005.tif28106Finally, in S706, the focus detection unit 205 calculates the defocus amount of the focus detection area using the image shift amount of the focus detection area calculated in S704.

[0075] As described above, according to this embodiment, particularly in video shooting, even in the case of manual focus, it is possible to suppress unintentional out-of-focus due to gimbal tracking.

[0076] <Second Embodiment> Next, a second embodiment of the present invention will be described. Note that in the second embodiment as well, since the imaging device described with reference to FIGS. 1 and 2 in the first embodiment can be used, the description thereof will be omitted here.

[0077] Next, the focus adjustment process in this embodiment will be described. The focus adjustment process described below is implemented by the camera control unit 204 in the camera unit 20 executing the imaging process program stored in the ROM 204a.

[0078] FIG. 12 is a flowchart showing the procedure of focus adjustment processing. In the present embodiment, the focus adjustment processing during video shooting will be described, but it may also be performed during the LV operation in still image shooting.

[0079] First, in S1201, the camera control unit 204 causes the focus detection unit 205 to perform focus detection processing. The focus detection processing is a process of acquiring information on the defocus amount and reliability for performing imaging plane phase difference AF. The area within the imaging screen for acquiring the information is set according to the state of the camera unit 20. Since the details of the focus detection processing are the same as the processing described with reference to FIGS. 7 to 11 in the first embodiment, the description is omitted here.

[0080] Next, in S1202, the camera control unit 204 determines whether AF control is possible from the focus detection result calculated in the focus detection processing of S1201. Whether AF control is possible can be determined, for example, by whether the reliability obtained from the above-described two-image coincidence degree or the steepness of the image shift amount satisfies a predetermined threshold. Also, if it can be determined that it is difficult to perform AF properly as shooting conditions, such as when the luminance during video shooting is not within a predetermined luminance range, or when the subject is dark and a large gain is applied, it is not necessarily required to use the focus detection result.

[0081] If AF control is possible, the process proceeds to S303, and the focus lens 104 is controlled based on the defocus amount calculated in the focus detection processing of S1201, and AF control for following the focus on the subject is performed, and then the process returns to S1201. On the other hand, if AF control is not possible, the process proceeds to S304.

[0082] The processing shown from S304 to S310 is the same as the processing described with reference to FIG. 3 in the first embodiment, so the description is omitted. After the processing of S309 and S310, the process returns to S1201, and the focus detection processing is performed again.

[0083] According to the second embodiment as described above, especially in video shooting, it is possible to suppress unintentional defocus due to gimbal tracking even in a state where AF is difficult.

[0084] In the above-described first and second embodiments, the imaging surface phase difference AF method has been described as an example of the focus detection method. However, the present invention is not limited to this. For example, a so-called contrast AF method may be used in which a predetermined frequency component is extracted by passing the signal data obtained by the image processing circuit 203 through a band-pass filter, and focus detection processing is performed using the obtained contrast evaluation value. In that case, in S1202, it may be determined that AF is not performed when the contrast is lower than a predetermined threshold value.

[0085] <Other Embodiments> Note that the present invention may be applied to a system composed of a plurality of devices or to an apparatus composed of a single device.

[0086] Further, the present invention can also be realized by a process in which a program that realizes one or more functions of the above-described embodiments is supplied to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0087] <Summary> The disclosure of this embodiment includes the following configurations.

[0088] (Item 1) A first detection means for detecting a rotation angle of a holding means including a turning means for turning and driving an imaging unit that holds an imaging unit including an optical system including a focus lens and an imaging means; Focus adjustment means for controlling the position of the focus lens to perform focus adjustment; When the rotation angle of the holding means is detected by the first detection means, an arithmetic means for obtaining a correction amount for correcting a deviation of the combined image position according to a change amount of a distance between the imaging unit corresponding to the rotation angle and the subject focused; The focus adjustment means is capable of focus adjustment in an AF mode of focusing on the subject based on a signal obtained from the imaging means, The arithmetic means obtains the correction amount when focus adjustment is not performed in the AF mode and the subject is being tracked using the turning means. The focus adjustment means is characterized in that it controls the position of the focus lens based on the correction amount. A focus adjustment device. (Item 2) The apparatus further includes a second detection means for detecting an angle of turning drive by the turning means. The arithmetic means obtains the change amount based on the rotation angle of the holding means, the angle of the turning drive, the length from the center of rotation of the holding means to the imaging unit, and the distance from the imaging unit to the subject before the holding means rotates. The focus adjustment device according to Item 1. (Item 3) The rotation angle of the holding means detected by the first detection means is a deflection angle with respect to an axis of a line connecting the center of rotation of the holding means and the subject. The focus adjustment device according to Item 2. (Item 4) The arithmetic means obtains the change amount using the cosine theorem. The focus adjustment device according to Item 2 or 3. (Item 5) The holding means is a gimbal. The focus adjustment device according to any one of Items 1 to 4. (Item 6) The focus adjustment means is capable of focus adjustment in an MF mode of controlling the position of the focus lens according to an operation from an operation means for instructing the position of the focus lens, The apparatus further includes a switching means for switching between the AF mode and the MF mode. The focus adjustment device according to any one of Items 1 to 5, wherein when the MF mode is selected, the focus adjustment means does not perform focus adjustment in the AF mode. (Item 7) The focus adjustment device according to any one of Items 1 to 6, wherein when the reliability of the defocus amount obtained based on the signal obtained from the imaging means is lower than a predetermined reliability, the focus adjustment means does not perform focus adjustment in the AF mode. (Item 8) The focus adjustment device according to any one of Items 1 to 7, wherein the focus adjustment means determines whether to perform control in the AF mode based on shooting conditions. (Item 9) The focus adjustment device according to any one of Items 1 to 8, wherein when the luminance of the subject is not within a predetermined luminance range, the focus adjustment means does not perform focus adjustment in the AF mode. (Item 10) The focus adjustment device according to any one of Items 1 to 9, wherein the focus adjustment means performs focus adjustment based on the contrast of the signal obtained from the imaging means, and when the contrast is lower than a preset threshold value, the focus adjustment means does not perform focus adjustment in the AF mode. (Item 11) The focus adjustment device according to Item 1, wherein the first detection means detects the angular velocity of the holding means and detects the rotation angle by integrating the angular velocity over time. (Item 12) The focus adjustment device according to Item 2, wherein the second detection means detects the angular velocity of the turning means and detects the turning drive angle by integrating the angular velocity over time. (Item 13) The focus adjustment device according to any one of Items 1 to 12, wherein the focus adjustment means controls the position of the focus lens based on the correction amount in synchronization with the tracking of the subject by the turning means. (Item 14) The focus adjustment device according to any one of items 1 to 13, the imaging unit, and the holding means An imaging device characterized by comprising the same. (Item 15) The focus adjustment device according to any one of items 1 to 13, the imaging unit, and the holding means, and An imaging system, wherein the imaging unit is detachable from the holding means. (Item 16) The imaging system according to item 15, wherein the optical system is detachable from the imaging means. (Item 17) A detection step of detecting a rotation angle of a holding means including a turning means for holding an imaging unit including an optical system including a focus lens and an imaging means and turning the imaging unit; A focus adjustment step of controlling the position of the focus lens to perform focus adjustment; A determination step of determining whether or not focus adjustment is performed in an AF mode of focusing on a subject based on a signal obtained from the imaging means in the focus adjustment step; When focus adjustment is not performed in the AF mode and the subject is being tracked using the turning means, a calculation step of obtaining a correction amount for correcting a deviation of the alignment position according to a change amount of a distance between the imaging unit corresponding to the rotation angle detected in the detection step and the focused subject; A correction step of correcting the position of the focus lens based on the correction amount A focus adjustment method characterized by comprising the same. (Item 18) A program for causing a computer to function as each means of the focus adjustment device according to any one of items 1 to 13. (Item 19) A computer-readable storage medium storing the program according to item 18.

[0089] The invention is not limited to the above embodiments, and various changes and modifications are possible without departing from the spirit and scope of the invention. Therefore, the claims are attached to disclose the scope of the invention.

Explanation of Signs

[0090] 10: Lens unit, 101: First lens group, 102: Diaphragm, 103: Second lens group, 104: Focus lens, 105: Zoom actuator, 106: Diaphragm actuator, 107: Focus actuator, 108: Zoom drive circuit, 109: Diaphragm drive circuit, 110: Focus drive circuit, 111: Lens control unit, 20: Camera unit, 201: Image sensor, 202: Image sensor drive circuit, 203: Image processing circuit, 204: Camera control unit, 205: Focus detection unit, 206: Subject detection unit, 30: Gimbal unit, 301: Gimbal control unit, 302: Display, 303: Operation unit, 304: Memory, 305: Inertial measurement unit, 306: Y-axis turning mechanism, 307: Pitch-axis turning mechanism, 308: Roll-axis turning mechanism

Claims

1. A holding unit that holds an imaging unit including an optical system including a focus lens and an imaging means, and a first detection means for detecting a rotation angle of the holding unit provided with a turning means for turning the imaging unit; Focus adjustment means for controlling the position of the focus lens to perform focus adjustment; Calculation means for obtaining a correction amount for correcting a deviation of the alignment position according to a change amount of a distance between the imaging unit corresponding to the rotation angle and the subject that has been focus-adjusted when the rotation angle of the holding unit is detected by the first detection means; The focus adjustment means is capable of performing focus adjustment in an AF mode of focusing on the subject based on a signal obtained from the imaging means, The calculation means obtains the correction amount when the focus adjustment is not performed in the AF mode and the subject is being tracked using the turning means, The focus adjustment means controls the position of the focus lens based on the correction amount, and a focus adjustment device characterized by this.

2. Further comprising a second detection means for detecting an angle of turning drive by the turning means, The calculation means obtains the change amount based on the rotation angle of the holding unit, the angle of the turning drive, the length from the center of rotation of the holding unit to the imaging unit, and the distance from the imaging unit to the subject before the holding unit rotates. The focus adjustment device according to claim 1, characterized by this.

3. The rotation angle of the holding unit detected by the first detection means is an off-angle with respect to an axis formed by a line connecting the center of rotation of the holding unit and the subject. The focus adjustment device according to claim 2, characterized by this.

4. The calculation means obtains the change amount using the cosine theorem. The focus adjustment device according to claim 2, characterized by this.

5. The holding unit is a gimbal. The focus adjustment device according to claim 1, characterized by this.

6. The focus adjustment means is capable of performing focus adjustment in an MF mode of controlling the position of the focus lens according to an operation from an operation means for instructing the position of the focus lens, Further comprising switching means for switching between the AF mode and the MF mode, When the MF mode is selected, the focus adjustment means does not perform focus adjustment in the AF mode. The focus adjustment device according to claim 1, characterized by this.

7. When the reliability of the defocus amount obtained based on the signal obtained from the imaging means is lower than a predetermined reliability, the focus adjustment means does not perform focus adjustment in the AF mode, and the focus adjustment apparatus according to claim 1 is characterized in that.

8. The focus adjustment means determines whether to perform control in the AF mode based on shooting conditions, and the focus adjustment apparatus according to claim 1 is characterized in that.

9. When the luminance of the subject is not within a predetermined luminance range, the focus adjustment means does not perform focus adjustment in the AF mode, and the focus adjustment apparatus according to claim 1 is characterized in that.

10. The focus adjustment means performs focus adjustment based on the contrast of the signal obtained from the imaging means, and when the contrast is lower than a preset threshold value, does not perform focus adjustment in the AF mode, and the focus adjustment apparatus according to claim 1 is characterized in that.

11. The first detection means detects the angular velocity of the holding means, and detects the rotation angle by time-integrating the angular velocity, and the focus adjustment apparatus according to claim 1 is characterized in that.

12. The second detection means detects the angular velocity of the turning means, and detects the turning drive angle by time-integrating the angular velocity, and the focus adjustment apparatus according to claim 2 is characterized in that.

13. The focus adjustment means controls the position of the focus lens based on the correction amount in synchronization with the tracking of the subject by the turning means, and the focus adjustment apparatus according to claim 1 is characterized in that.

14. The focus adjustment apparatus according to any one of claims 1 to 13, The imaging unit, The holding means An imaging apparatus characterized by having.

15. The focus adjustment apparatus according to any one of claims 1 to 13, The imaging unit, The holding means, and has An imaging system characterized in that the imaging unit is detachable from the holding means.

16. The optical system is detachable from the imaging means, and the imaging system according to claim 15 is characterized in that.

17. A detection step of detecting the rotation angle of a holding means including a turning means for holding an imaging unit including an optical system including a focus lens and an imaging means and turning the imaging unit; A focus adjustment step of controlling the position of the focus lens to perform focus adjustment; In the focus adjustment step, a determination step of determining whether focus adjustment is performed in an AF mode in which focus is adjusted on a subject based on a signal obtained from the imaging means; When focus adjustment is not performed in the AF mode and the subject is being tracked using the swivel means, a calculation step of obtaining a correction amount for correcting a deviation of the combined image position according to a change amount of the distance between the imaging unit corresponding to the rotation angle detected in the detection step and the subject whose focus has been adjusted; A correction step of correcting the position of the focus lens based on the correction amount; A focus adjustment method characterized by comprising the above steps.

18. A program for causing a computer to function as each means of the focus adjustment device according to any one of Claims 1 to 13.

19. A computer-readable storage medium storing the program according to Claim 18.

Images