Attitude information acquisition device, imaging device, and attitude information acquisition method

Abstract

To provide an attitude information acquisition device capable of acquiring attitude information on a driven part of a device which has an actuator for driving the driven part.SOLUTION: An attitude information acquisition device includes: first acquisition means for acquiring a shake detection result from a shake detection part which is provided at a driven part capable of changing an attitude relative to a base 40 so as to detect a shake applied to the driven part; second acquisition means for acquiring drive information on the driven part; and an attitude information calculation part for calculating attitude information on the driven part on the basis of the shake detection result and the drive information.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a posture information acquisition device for acquiring posture information of a driven part, an imaging device, and a posture information acquisition method. [Background technology]

[0002] Some imaging devices such as video cameras have a function of calculating a relative attitude from the output of a sensor mounted thereon. Patent Document 1 discloses a technique for detecting the movement of an imaging device using an angular velocity sensor, an acceleration sensor, and a geomagnetic sensor, and calculating the attitude of the imaging device by combining these outputs. In this way, the attitude of the imaging device can be calculated by using sensors that detect the movement of the imaging device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-181059 A Summary of the Invention [Problem to be solved by the invention]

[0004] Here, in recent years, there has been an imaging device that has an actuator for rotating an imaging unit and is fixed to a tabletop, a ceiling, or the like for use. In such an imaging device, a user inputs a drive amount to the imaging device, or the drive amount is automatically calculated from information on an imaged video, and a drive target value is determined, and the imaging direction is controlled by driving the actuator according to the target value. However, Patent Document 1 does not take into consideration the acquisition of attitude information in such an imaging device. Moreover, this is not limited to imaging devices, and is also applicable to the case of acquiring attitude information of a driven part that changes its attitude by being driven to rotate relative to a base, or the like.

[0005] An object of the present invention is to provide a posture information acquisition device capable of acquiring posture information of a driven part of a device having an actuator that drives a driven part such as an imaging unit. [Means for solving the problem]

[0006] In order to achieve the above object, a posture information acquisition device according to one aspect of the present invention is characterized in that it is provided on a driven part whose posture can be changed with respect to a base, and includes a first acquisition means for acquiring a vibration detection result from a vibration detection unit that detects vibration applied to the driven part, a second acquisition means for acquiring drive information of the driven part, and a posture information calculation unit for calculating posture information of the driven part based on the vibration detection result and the drive information. Other features of the present invention will be described in the description of the embodiment of the invention. Effect of the Invention

[0007] According to the present invention, it is possible to provide a posture information acquisition device capable of acquiring posture information of a driven part of a device having an actuator that drives a driven part. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram for explaining a pan-tilt camera and its attitude. [Diagram 2] FIG. 1 is a configuration diagram for explaining a pan-tilt camera according to a first embodiment and a second embodiment. [Diagram 3] 4 is a graph for explaining a pan / tilt drive target value and a pan / tilt drive amount. [Figure 4] FIG. 2 is a configuration diagram of an attitude information calculation unit of the pan-tilt camera of the first embodiment. [Diagram 5] 5 is a flowchart showing a posture information calculation process of the pan-tilt camera according to the first embodiment. [Figure 6] FIG. 11 is a configuration diagram for explaining an attitude information calculation unit of the pan-tilt camera according to the second embodiment. [Figure 7]13 is a flowchart showing a posture information calculation process of the pan-tilt camera according to the second embodiment. [Figure 8] FIG. 11 is a configuration diagram for explaining a pan-tilt camera according to a third embodiment. [Figure 9] FIG. 11 is a configuration diagram for explaining an attitude information calculation unit of the pan-tilt camera of the third embodiment. [Figure 10] 13 is a flowchart showing a posture information calculation process of the pan-tilt camera according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the parts common to the first to third embodiments 1 to 3 will be described first, and then the parts specific to each embodiment will be described. Also, in each drawing, the same reference numbers are given to the same members, and duplicated explanations will be omitted. Note that the following embodiments do not limit the invention according to the claims. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily.

[0010] <Common part> A configuration common to the first to third embodiments will be described with reference to Figures 1 to 3. In the first to third embodiments, a pan-tilt camera will be described as an example of a device in which a driven part is driven relative to a base.

[0011] 1 is an external view of a pan-tilt camera 100. The pan-tilt camera 100 includes a base 40, an imaging section 10, a tilt driving section 20, and a pan driving section 30, which are driven sections whose attitude can be changed with respect to the base.

[0012] The imaging unit 10 includes an imaging lens 105 that constitutes an imaging optical system, and an imaging element 106 that captures a subject image formed by the imaging optical system.

[0013] The tilt driving unit 20 is a mechanism that supports the imaging unit 10 to rotate freely in the tilt direction, and includes a stepping motor that is an actuator for rotational drive, and an integrated circuit (IC) that generates a pulse signal for driving the stepping motor. The tilt driving unit 20 sets the number of pulses in the integrated circuit to drive the imaging unit 10 to rotate in the tilt direction.

[0014] The pan driving unit 30 is a mechanism that supports the tilt driving unit 20 to rotate freely in the pan direction, and is composed of a stepping motor, which is an actuator for rotational drive, and an integrated circuit that generates a pulse signal for driving the stepping motor. The pan driving unit 30 sets the number of pulses in the integrated circuit, thereby driving the tilt driving unit 20 and the imaging unit 10 supported by it to rotate in the pan direction.

[0015] The base 40 is a mechanism that rotatably supports the pan drive unit 30, and is fixed to a ceiling, a top plate, a tripod, etc. Also, shooting may be performed while the base 40 is placed on a top plate, etc.

[0016] The pan-tilt camera 100 can rotate the imaging unit 10 in the pan direction and tilt direction and change the attitude of the imaging unit 10 by setting the number of pulses in the integrated circuits of the tilt driver 20 and the pan driver 30.

[0017] The actuators of the tilt drive unit 20 and the pan drive unit 30 are not limited to stepping motors, and voice coil motors or other motors may be used.

[0018] The attitude of the imaging unit 10, which is the driven unit of the pan-tilt camera 100, will be described with reference to FIG. 1. In this embodiment, a coordinate system of three orthogonal axes that rotates in accordance with the movement of the imaging unit 10 is defined, and the coordinate system expresses the attitude of the imaging unit 10 by the rotation angle. In other words, the calculated attitude information is expressed as a relative angle indicating how much the imaging unit 10 has rotated with respect to the coordinate system at a certain reference timing. For example, the X-axis, Y-axis, and Z-axis are defined as shown in FIG. 1(a), and this is expressed as the reference timing T A Coordinate system C in AAfter that, the posture of the imaging unit 10 changes, and at timing T B In X b Axis, Y b axis, Z b Axis-like coordinate system C B In this case, the coordinate system C A From coordinate system C B The rotation angle for converting to is timing T B This is the attitude of the imaging unit 10 in the state shown in FIG.

[0019] The rotation angle of the coordinate system can be expressed, for example, by using Euler angles with the coordinate system at the reference timing as the reference coordinate system, and the present embodiment will also use Euler angles for the explanation. Note that since Euler angles are publicly known information, detailed explanation will be omitted. Also, in FIG. 1, the optical axis direction of the imaging unit 10 corresponds to the Z axis of the coordinate system, but since the calculated attitude is expressed as a relative angle with respect to the reference angle, the coordinate system can be defined arbitrarily. Note that the reference timing may be, for example, when the pan-tilt camera 100 is turned on, or when an instruction to record video is received from a client device, which will be described later. FIG. 2 is a block diagram showing the configuration of the pan-tilt camera 100 according to an embodiment of the present invention. The pan-tilt camera 100 is connected to a client device (not shown) via a network 102, and they can communicate with each other.

[0020] The client device is a device that includes a controller that allows the user to instruct the pan-tilt camera 100 to pan-tilt and record images (either still images or videos), and a monitor that allows the user to check the images captured by the pan-tilt camera 100.

[0021] The network 102 is realized by, for example, the Internet, a wired LAN, a wireless LAN, a WAN, etc. In this embodiment, a case where the imaging device and the client device are connected via a network will be described, but the connection may be made via an interface other than a network, and the method of controlling the pan-tilt camera 100 is not limited to these.

[0022] The communication control unit 101 receives a pan / tilt drive instruction from the client device to the imaging unit 10 via the network 102, and outputs the drive instruction to the pan / tilt drive amount determination unit 103. Specifically, the user sets the rotation angle and drive time of the pan / tilt drive on the client device, and these are input to the pan / tilt drive amount determination unit 103 via the network 102 and the communication control unit 101. Furthermore, when a recording instruction is received from the client device, the communication control unit 101 outputs the recording instruction to the recording control unit.

[0023] The pan / tilt drive amount determination unit 103 determines the pan / tilt drive target value and the pan / tilt drive amount based on the rotation angle and drive time indicated by the drive instruction received from the communication control unit 101. FIG. 3 is a graph showing an example of the pan / tilt drive target value and the pan / tilt drive amount. The pan / tilt drive target value is time-series data showing the target attitude angle of the imaging unit 10. For example, FIG. 3(a) shows the pan / tilt drive target value d(t) when the target angle D and the target time T are input at time t=0. When the target angle D and the target time T are input, the pan / tilt drive amount determination unit 103 first determines the drive start timing t1 and the drive end timing t2 so as to satisfy the target drive time. After that, the pan / tilt drive target value d(t) between the timing t1 and the timing t2 is calculated so that the pan / tilt drive target value reaches the target angle D at the timing t2.

[0024] The pan / tilt drive target value d(t) is calculated, for example, using the following (Equation 1).

[0025]

number

[0026] The calculated pan / tilt drive target value is input to the attitude information calculation unit 200 for calculating the attitude of the imaging unit 10. The pan / tilt drive target value is initialized to 0 at the reference timing for calculating the attitude, and is thereafter accumulated each time a drive instruction is received.

[0027] On the other hand, the pan / tilt drive amount is time-series data representing the drive angular velocity of the actuator. For example, FIG. 3(b) represents the pan / tilt drive amount ω(t) when a target angle D and a target time T are input at timing t0. The pan / tilt drive amount ω(t) is calculated based on a pan / tilt drive target value d(t), and is calculated as an angular velocity by, for example, differentiating the pan / tilt drive target value d(t) with respect to time. The calculated pan / tilt drive amount is input to the pan / tilt drive instruction unit 104.

[0028] The pan / tilt drive instruction unit 104 calculates the number of pulses to be set in the integrated circuits of the tilt drive unit 20 and the pan drive unit 30 based on the pan / tilt drive amount acquired from the pan / tilt drive amount determination unit 103, and outputs the calculated number of pulses to the tilt drive unit 20 and the pan drive unit 30. Each of the tilt drive unit 20 and the pan drive unit 30 sets the number of pulses acquired from the pan / tilt drive support unit 104 in the integrated circuits of the tilt drive unit 20 and the pan drive unit 30. As a result, the actuator is driven at an angle based on the pan / tilt drive amount, and the attitude of the imaging unit 10 with respect to the base 40 changes.

[0029] Although the pan / tilt drive amount determination unit 103 calculates the pan / tilt drive target value based on (Equation 1) and drives the actuator at a constant speed, the method of calculating the pan / tilt drive target value is not limited to this. For example, the pan / tilt drive amount determination unit 103 may hold in advance patterns of pan / tilt drive target values ​​for accelerating and decelerating drive, and may perform drive involving acceleration and deceleration by combining these patterns based on the input rotation angle and drive time.

[0030] In addition, here, the case where the pan-tilt drive amount determination unit 103 determines the pan-tilt drive target value and the pan-tilt drive amount based on a user instruction from a client device has been described, but the pan-tilt drive of the imaging unit 10 is not limited to this. For example, the pan-tilt drive amount determination unit 103 may determine the rotation angle and drive time based on video information stored in the image memory 110 described later instead of the user instruction. This allows the pan-tilt camera 100 to automatically track the subject. Specifically, the pan-tilt drive amount determination unit 103 detects the subject from the input video information, and determines the rotation angle and drive time of the pan-tilt drive from the calculated subject movement amount and subject movement speed. A separate circuit for subject detection may be provided, and the subject detection result may be input to the pan-tilt drive amount determination unit 103.

[0031] The imaging unit 10 has an imaging lens 105, an imaging element 106, and an angular velocity sensor 107. The imaging element 106 captures an image of a subject through the imaging lens 105, and outputs a captured image signal to an image signal processing unit 108. The imaging element 106 is formed of, for example, a CMOS image sensor.

[0032] The image signal processor 108 performs signal processing such as white balance adjustment and gamma correction on the image signal output from the image sensor 106 , and stores the generated video signal in an image memory 110 .

[0033] The image memory 110 is a storage medium for temporarily storing a video signal. The video signal stored in the image memory 110 is transmitted to a client device via a network 102 by a communication control unit 101, or is recorded on a recording medium 112 via a recording control unit 111.

[0034] The recording control unit 111 receives a video recording instruction from the client device via the network 102 by the communication control unit 101, synchronizes the video signal stored in the image memory 110 with the orientation information output by the orientation information calculation unit 200, and records them on the recording medium 112. Note that in the first to third embodiments, a case will be described in which the pan-tilt camera 100 includes the recording medium 112. However, the pan-tilt camera 100 may not include the recording medium 112, and may transmit the video signal and orientation information to the client device via the network 102 and record them on the client device side. The storage medium 112 may also be a portable storage medium (e.g., an SD card) that can be attached to the pan-tilt camera 100.

[0035] The angular velocity sensor 107 is a detection unit that detects the shaking applied to the imaging unit 10 as an angular velocity, and detects angular velocities around three orthogonal axes. The three detected angular velocity signals are input to the attitude information calculation unit 200 and used to calculate the attitude of the imaging unit 10. In this embodiment, in order to simplify the explanation of the attitude calculation, the detection axes of the angular velocity sensor 107 are the x-axis, y-axis, and z-axis. The angular velocity sensor 107 is attached so that the x-axis coincides with the side direction of the imaging unit 10, the y-axis coincides with the top and bottom directions of the imaging unit 10, and the z-axis coincides with the optical axis direction of the imaging unit 10.

[0036] The attitude information calculation unit 200 calculates the attitude of the imaging unit 10 based on the pan / tilt drive target value and the angular velocity signal. In general, the output of an inertial sensor such as an angular velocity sensor or an acceleration sensor contains errors due to electrical noise characteristics and phase delays caused by filter calculations, and the angular velocity signal contains error components especially in the low frequency band. Therefore, if the attitude information is calculated based only on the output of the inertial sensor, the accuracy of the calculated attitude information will decrease.

[0037] On the other hand, the pan-tilt drive target value does not include electrical noise components contained in the inertial sensor output, and since the attitude change of the imaging optical system can be known in advance, no error due to phase delay occurs. However, although information on the pan-tilt drive position of the imaging unit 10 relative to the base 40 is included, a component that changes the attitude of the pan-tilt camera 100 itself due to an external force is not included. Here, the attitude change due to an external force refers to, for example, an attitude change due to the influence of vibrations caused by wind or human movement when the camera is fixed to a building. Therefore, the attitude information calculation unit 200 of the first to third embodiments calculates the attitude of the imaging unit 10 using the angular velocity signal and the pan-tilt drive target value. Details of the calculation method will be described in the first to third embodiments.

[0038] <First embodiment> Fig. 4 is a functional block diagram of the orientation information calculation unit 200 of the pan-tilt camera of the first practical embodiment. Fig. 5 is a flowchart showing the orientation information calculation process by the orientation information calculation unit 200, and represents the process that is repeatedly executed at an arbitrary cycle (for example, an image capture cycle). The orientation information calculation process by the orientation information calculation unit 200 of this embodiment will be described with reference to these drawings.

[0039] In this embodiment, the pan / tilt drive target value output by the pan / tilt drive amount determination unit 103 is used as a framing component, and the angle calculated based on the angular velocity signal detected by the angular velocity sensor 107 is used as a shaking component, and the two are combined to calculate the attitude of the imaging unit 10.

[0040] The posture information calculation unit 200 of this embodiment has an angle conversion unit 201 that converts an angular velocity signal into an angle signal, a high frequency component extraction unit 202 that extracts high frequency components from the angle signal, and an adder 210 that adds the high frequency components of the extracted angle signal and a framing component.

[0041] The angle conversion unit 201 converts the angular velocity signal detected by the angular velocity sensor 107 into an angle signal (ψ G , θ G , φ GThe following (Equation 2) converts the angular velocity signal (ωx, ωy, ωz) output from the angular velocity sensor 107 into an angle signal (ψ G , θ G , φ G ) where (ωx, ωy, ωz) represent the angular velocities around the x-axis, y-axis, and z-axis detected by the angular velocity sensor 107. G , θ G are the rotation angles ψ in the pan direction of the pan / tilt drive target values ​​output by the pan / tilt drive amount determination unit 103. T , the rotation angle θ in the tilt direction T Also, the angle signal φ G represents the rotation angle around the optical axis direction of the imaging unit 10. Here, (ψ G(t) , θ G(t) , φ G(t) ) is the angle signal calculated in this process, and (ψ G(t-1) , θ G(t-1) , φ G(t-1) ) is the angle signal calculated in the previous process.

[0042] Matrix A is a transformation matrix for transforming the angular velocity signal detected by angular velocity sensor 107 into an angle change amount expressed in Euler angles, and is expressed by the following (Equation 3). (θ, φ) represent the attitude angle of image capture unit 10 fed back from the output of adder 210, where θ represents the rotation angle in the tilt direction, and φ represents the rotation angle around the optical axis direction of image capture unit 10.

[0043]

number

[0044]

number

[0045] The method of converting the angular velocity signal into an angle signal is not limited to using this formula, and various known methods, such as pseudo-integration using a low-pass filter or a mathematical integration method, can be used to convert the angular velocity into an angle and add it to the previous angle signal.

[0046] The high frequency component extraction unit 202 extracts the angle signal (ψ G , θ G , φ G The high frequency component extraction unit 202 is configured with, for example, a high pass filter.

[0047] The adder 210 adds the high frequency component of the angle signal (that is, the shaking component) extracted by the high frequency component extraction section 202 to the framing component, thereby synthesizing the shaking component and the framing component to calculate attitude information of the imaging section 10.

[0048] The flow of the posture information calculation process by posture information calculation section 200 of this embodiment will be described with reference to Fig. 5. First, in step S101, it is determined whether the current processing timing is the reference timing for posture calculation described above. If it is determined that it is the reference timing, the initialization processes shown in steps S102, S103, and S104 are executed.

[0049] First, in step S102, the angle conversion unit 201 calculates the integrated angle value (ψ G , θ G , φ G ) is initialized to 0. Next, in step S103, the pan / tilt drive amount determination unit 103 determines the pan / tilt drive target value (ψ T , θ T , 0) is initialized to 0. As a result, in step S104, the posture information (ψ, θ, φ) output from the adder 210 is also initialized to 0. G , θ G , φ G ) and the pan-tilt drive target value (ψ T , θ T , 0) may be initialized in any order, and may be performed sequentially or simultaneously.

[0050] If it is determined in step S101 that the current processing timing is not the reference timing, or if it is determined that the current processing timing is the reference timing and each initialization processing has ended, the process proceeds to step S105. In step S105, the angle conversion unit 201 performs an integral operation on the angular velocity signal detected by the angular velocity sensor 107 to obtain an angle signal (ψ G , θ G , φ G ) and proceed to step S106.

[0051] In step S106, the high-frequency component extraction unit 202 extracts the angle signal (ψ G , θ G , φ G ) and extract the high frequency components contained in the vibration component (ψ Gh , θ Gh , φ Gh ) to the adder 210.

[0052] In step S107, the adder 210 calculates the pan / tilt drive target value (ψ T , θ T , 0) and the fluctuation component (ψ Gh , θ Gh , φ Gh ) is added together. The addition result is output as a posture angle (ψ, θ, φ) to the recording control unit 111. Here, ψ represents the rotation angle in the pan direction, θ represents the rotation angle in the tilt direction, and φ represents the rotation angle in the roll direction.

[0053]

number

[0054] As described above, according to this embodiment, it is possible to detect vibrations caused by the surrounding environment by using the shaking components detected by the angular velocity sensor, while accurately calculating the attitude of the imaging unit by using the pan-tilt drive target value.

[0055] <Second embodiment> Fig. 6 is a functional block diagram of the orientation information calculation unit 200 of the pan-tilt camera of the second practical embodiment. In Fig. 6, the same components as those in Fig. 4 are given the same reference numerals, and their explanation will be omitted. Fig. 7 is a flowchart showing the orientation information calculation process by the orientation information calculation unit 200, and represents the process repeatedly executed at an arbitrary period, as in the first embodiment. In Fig. 7, the same processes as those in Fig. 5 are given the same reference numerals, and their explanation will be omitted. The orientation information calculation process by the orientation information calculation unit 200 of this embodiment will be explained with reference to these drawings.

[0056] In this embodiment, the attitude of the imaging unit 10 is calculated by Kalman filtering based on the pan / tilt drive target value output by the pan / tilt drive amount determination unit 103 and the angular velocity signal detected by the angular velocity sensor 107. Here, the Kalman filter is an example of a filter for estimating the dynamic state of the system from an input value including an error by using a state equation and an observation equation.

[0057] The posture information calculation unit 200 of this embodiment includes a Kalman filter calculation unit 203 to which the angular velocity signal output from the angular velocity sensor 107 and the pan / tilt drive target value output from the pan / tilt drive amount determination unit 103 are input. The Kalman filter calculation unit 203 acquires posture information based on the output signal obtained by Kalman filtering, and outputs the posture information to the recording control unit 111.

[0058] The flow of the posture information calculation process by posture information calculation section 200 of this embodiment will be described with reference to Fig. 7. First, in step S101, it is determined whether the current processing timing is the reference timing for posture calculation described above. If it is determined that it is the reference timing, the initialization processes shown in steps S102, S103, and S111 are executed. The initialization processes in steps S102 and S103 are the same as those in the first embodiment, and therefore will not be described.

[0059] In step S111, the Kalman filter calculation unit 203 is initialized, and the attitude angles (ψ, θ, φ), which are state values ​​of the Kalman filter described later, are initialized to 0. If it is determined in step S101 that the current processing timing is not the reference timing, the process proceeds to step S112. In step S112, the Kalman filter calculation unit 203 calculates the angular velocity signal (ωx, ωy, ωz) output by the angular velocity sensor 107 and the pan / tilt drive target value (ψ T , θ T The two signals (θ, φ) are input to a Kalman filter. The output of the Kalman filter is output as the attitude angle (ψ, θ, φ) to the recording control unit 111. Note that the Kalman filter is publicly known information, so a detailed description will be omitted.

[0060] In this embodiment, assuming that the attitude of the imaging unit 10 is a dynamic state to be estimated, the state equation is expressed by (Equation 5) and the observation equation is expressed by (Equation 6), and the true state is estimated from two inputs each containing different types of noise components. The noise components relative to the true state are that the pan / tilt drive target value does not contain vibration components of the surrounding environment, and the noise components of the angle signal are output fluctuations (including errors such as offset components and drift components) due to the temperature characteristics of the angular velocity sensor 107, etc.

[0061]

number

[0062]

number

[0063] x is a state value representing the true attitude of the imaging unit 10, y is an observation value, u is an input value, v is process noise, and w is observation noise. The state value x is the attitude angle (ψ, θ, φ), and the observation value y is the pan-tilt drive target value (ψ T , θ T, 0), and the input value u is the angular velocity signal (ωx, ωy, ωz). Matrix A is a transformation matrix for converting the angular velocity signal (ωx, ωy, ωz) into an angle change amount expressed in Euler angles, as shown in (Equation 3) above, and (θ, φ) represent the attitude information of the imaging unit 10 fed back from the subsequent stage. The process noise v and observation noise w need to be set taking into consideration the magnitude of the noise components contained in the angular velocity signal and the pan / tilt drive target value. The Kalman filter calculation unit 203 obtains the attitude angle (ψ, θ, φ) by Kalman filtering based on the state equation and observation equation.

[0064] As described above, according to this embodiment, by performing Kalman filtering using the angular velocity signal and the pan / tilt drive target value, it is possible to calculate a more accurate attitude while taking into account the effects of vibration.

[0065] <Third embodiment> In this embodiment, a method of calculating attitude information of the imaging unit 10 in a case where the imaging unit 10 has a function of correcting image blur caused by shaking applied to the imaging unit 10 will be described.

[0066] Fig. 8 is a functional block diagram showing the configuration of a pan-tilt camera 100 in a third practical embodiment. In Fig. 8, the same components as those in Fig. 2 are given the same reference numerals, and the description thereof will be omitted.

[0067] The pan-tilt camera 100 of this embodiment is different from the pan-tilt camera 100 shown in Fig. 2 in that it includes an image blur correction control unit 109 and has a function of electronically correcting image blur. The image blur correction control unit 109 calculates an image blur correction amount for correcting image blur occurring in an image (video) captured by the imaging unit 10 due to shaking applied to the imaging unit 10, and outputs the amount to the image signal processing unit 108 and the attitude information calculation unit 200. The image blur correction control unit 109 receives an instruction from the client device via the network 102 by the communication control unit 101 to turn on or off the image blur correction function, indicating whether or not to perform image blur correction, and switches the calculation method of the image blur correction angle depending on the on / off. When the image blur correction function is on, the image blur correction amount is calculated by integrating the angular velocity signal that is the output of the angular velocity sensor 107. When calculating the image blur compensation amount, the angle signal may be subjected to high-pass filtering to remove low-frequency components, or may be multiplied by a gain to adjust the effectiveness of compensation, and the image blur compensation amount may be calculated based on the output of the angular velocity sensor 107. A method for calculating the image blur compensation amount can be performed using a known technique, and therefore a description thereof will be omitted. On the other hand, when the image blur compensation function is turned off, the image blur compensation amount is fixed to 0.

[0068] The image signal processing unit 108 corrects the image blur by shifting the cut-out position of the image so as to cancel the movement of the image blur. The shift amount of the cut-out position is calculated based on the image blur correction amount output from the image blur correction control unit 109, and is calculated by using, for example, the following (Equation 7) to obtain the image blur correction amount (ψ H , θ H ) to the amount of shift in the cut-out position (H a , H b ) is converted to (ψ H , θ H ) and (H a , H b ) represent the amount of shift in the cut-out position in the horizontal and vertical directions, respectively. Also, f is the focal length of the imaging lens 105, and p is the cell pitch of the imaging element 106.

[0069]

number

[0070] In the present embodiment, the image blur correction means is described as a method of electronically correcting the image by shifting the cropping position of the image, but the image blur correction method is not limited to this. For example, the imaging lens 105 may have a shift lens (not shown) for image blur correction, and the image blur may be corrected by driving the shift lens in a direction perpendicular to the optical axis direction of the imaging lens 105 based on the image blur correction angle output from the image blur correction control unit 109. By driving the shift lens in this way, the relative position between the imaging element 106 and the optical image formed by the imaging lens is shifted, and the image blur can be corrected. In this case, a Hall element that detects the position of the shift lens may be provided in the imaging unit 10, and the image blur correction control unit 109 may back-calculate the actual image blur correction amount from the position of the shift lens detected by the Hall element, and input the correction amount to the attitude information calculation unit 200. Alternatively, the image blur may be corrected by driving the imaging element 106 in a direction perpendicular to the optical axis direction. In this case as well, a Hall element that detects the position of the image sensor 106 may be provided in the imaging unit 10, and the actual amount of image blur correction may be calculated back from the position of the image sensor 106 detected by the Hall element and input to the attitude information calculation unit 200. Also, a plurality of image blur correction methods may be combined.

[0071] When image stabilization is effective, the effects of vibrations due to the surrounding environment do not appear in the recorded image, so if the attitude calculation method described in the first or second embodiment is used, the movement of the image and the calculated attitude information do not match. As described above, in the pan-tilt camera of this embodiment, the image signal and the attitude information are recorded in synchronization with each other, so it is preferable to match the attitude information with the movement of the image after image stabilization. In this embodiment, an attitude calculation method for achieving this purpose will be described.

[0072] Fig. 9 is a functional block diagram of the attitude information calculation unit 200 of the pan-tilt camera of the third practical embodiment. In Fig. 9, the same components as those in Fig. 4 are given the same reference numerals, and their explanations are omitted. Fig. 10 is a flow chart showing the process of calculating attitude information by the attitude information calculation unit 200, and represents the process that is repeatedly executed at an arbitrary period. In Fig. 10, the same processes as those in Fig. 5 are given the same reference numerals, and their explanations are omitted.

[0073] The orientation information calculation unit 200 of this embodiment differs from the orientation information calculation unit 200 of the first embodiment shown in FIG. 4 in that it includes a subtractor 220 that subtracts the output of the image blur correction control unit 109 from the output of the angle conversion unit 201. The subtraction result by the subtractor 220 is output to the high frequency component extraction unit 202. This makes it possible to obtain a shake component after image blur correction that corresponds to an image in which at least a part of the influence of the shake component has been corrected by image blur correction. In this embodiment, the orientation information is obtained using the shake component after image blur correction and the framing component.

[0074] The flow of the attitude information calculation process by the attitude calculation unit 220 of this embodiment will be described with reference to FIG. 10. Steps S101 to S105 are the same as those of the first embodiment, and if it is determined that it is the reference timing, various initialization processes (S102 to S103) are performed and the process proceeds to step S105. Also, if it is determined that it is not the reference timing, the process proceeds to step S105. In step S105, the output of the angular velocity sensor 107 is converted into an angle signal (ψ G , θ G , φ G ) and proceed to step S108.

[0075] In step S108, the subtractor 220 subtracts the angle signal (ψ G , θ G , φ G ), the image blur correction amount (angle) output by the image blur correction control unit 109 is subtracted from the image blur correction amount (angle), the subtraction result is output to the high frequency component extraction unit 202, and the process proceeds to step S106. By subtracting the image blur correction amount in this step, it is possible to obtain the posture angle (ψ, θ, φ) taking image blur correction into consideration.

[0076] Steps S106 and S107 are the same as those in the first embodiment. High-frequency components are extracted from the subtraction result by the subtractor 220 and are converted into fluctuation components (ψ Gh , θ Gh , φ Gh ) and the framing component (ψ T , θ T , 0) to obtain the attitude angle information.

[0077] As described above, according to this embodiment, when an image blur correction function is provided, by calculating posture information based on the angular velocity signal, the pan / tilt drive target value, and the image blur correction angle, it is possible to calculate posture information taking the image blur correction function into consideration.

[0078] (Other embodiments and modifications) Although the present invention has been described in detail above based on the preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-mentioned embodiments may be combined as appropriate. For example, the second and third embodiments may be combined, and the output of the subtractor 220 and the output of the pan / tilt drive amount determination unit 103 may be input to a Kalman filter to obtain attitude information.

[0079] In the above-described first to third embodiments, the pan / tilt drive amount determination unit 103 determines the drive target value (ψ T , θ T, 0) was acquired, and the drive target value was set as the framing component. However, the framing component is not limited to this as long as it is drive information for the base 40 of the imaging unit 10. For example, it may be a value obtained by converting the pan / tilt drive amount acquired by the pan / tilt drive amount determination unit 103 into an angle. In addition, if a position detection sensor (encoder) that detects the pan / tilt drive of the imaging unit 10 relative to the base 40 is provided, the pan / tilt position may be detected based on the output of the position detection sensor and used as the framing component. However, when the detection result of the position detection sensor that detects the pan / tilt drive is used as the framing component, the influence of a phase delay is more likely to occur compared to when information based on a pan / tilt drive instruction (for example, a drive target value) is used as the framing component, so phase compensation may be performed.

[0080] In the above-mentioned first to third embodiments, the attitude information is calculated within the pan-tilt camera 100 body, but may be acquired by another device connected to the network. In that case, the device functions as an attitude information acquisition device. When the attitude information is calculated by another device, the output of the angular velocity sensor and, as a framing component, for example, a drive target value of pan-tilt drive are transmitted to the device via the communication control unit 111. When the device is a client device that transmits a drive instruction to the pan-tilt camera 100, the client device may acquire drive information such as a drive target value based on the drive instruction output to the pan-tilt camera 100. Even when the attitude information is calculated by a device other than the client device, the output of the angular velocity sensor may be received from the pan-tilt camera 100, and drive information such as a drive target value may be received from the client device.

[0081] In addition, in the above-mentioned first to third embodiments, an angular velocity signal is used as the shaking component, but the shaking component is not limited to this, and the output of an acceleration sensor, the output of a geomagnetic sensor, etc. can also be used. When detecting the attitude in three axes, it is sufficient to combine an acceleration sensor (pitch and roll directions) and a geomagnetic sensor (yaw direction).

[0082] In the above-mentioned first to third embodiments, the pan-tilt camera with an integrated lens is described as an example, but the configuration of the imaging device is not limited to this as long as at least one of pan, tilt, and roll can be driven. For example, the present invention can be applied to a pan-tilt camera with an interchangeable lens or a gimbal camera capable of pan-tilt driving. In addition, the present invention can be applied to any purpose, such as a so-called surveillance camera or a camera used for video production. In addition, the present invention can be applied to a shooting system configured by combining a camera with a pan-tilt head or gimbal capable of pan-tilt driving. The pan-tilt driving can be performed in real time based on a user's instruction, or can be performed at a preset time or time interval by a preset drive. In addition, as described above, the pan-tilt driving can be performed by automatically tracking a subject. In addition, the present invention can be applied to a device such as a robot arm whose posture can be changed by rotating the pan-tilt roll relative to a base other than the imaging device.

[0083] Furthermore, the method of acquiring the attitude information is not limited to the above-mentioned first to third embodiments. For example, the phase delay amount of the sway component is acquired by comparing the framing component with the sway component, and the phase delay amount acquired for the sway component is compensated for using a phase compensation filter, thereby reducing the influence of the phase delay contained in the sway component. By acquiring the attitude information based on the acquired phase-compensated sway component, it is possible to acquire the attitude information more accurately than acquiring the attitude information without using the framing component.

[0084] The present invention also includes cases where a software program that realizes the functions of the above-described embodiments is supplied to a system or device having a computer capable of executing the program directly from a recording medium or via wired / wireless communication, and the program is executed.

[0085] Therefore, the program code itself that is supplied to and installed in a computer to realize the functional processing of the present invention also realizes the present invention. In other words, the computer program itself for realizing the functional processing of the present invention is also included in the present invention.

[0086] In this case, so long as the program has the functions of a program, the form of the program is not important, such as object code, a program executed by an interpreter, or script data supplied to an OS.

[0087] The recording medium for supplying the program may be, for example, a hard disk, a magnetic recording medium such as a magnetic tape, an optical / magneto-optical storage medium, or a non-volatile semiconductor memory.

[0088] As a method of supplying the program, a method is also conceivable in which the computer program constituting the present invention is stored in a server on a computer network, and a connected client computer downloads the computer program to program it. [Explanation of symbols]

[0089] 100 Pan Tilt Camera 10. Imaging unit 20 Tilt drive unit 30 Pan drive unit 103 Pan-tilt drive amount determination unit 104 Pan-tilt drive instruction unit 105 Imaging Lens 106 Image sensor 107 Angular rate sensor 108 Image signal processing section 109 Image stabilization control unit 111 Recording control section 201 Angle conversion section 202 High frequency component extraction section 203 Kalman filter calculation section

Claims

1. a first acquisition means for acquiring a vibration detection result from a vibration detection unit provided on a driven part whose attitude can be changed with respect to a base and configured to detect vibration applied to the driven part; A second acquisition means for acquiring drive information of the driven part; and a posture information calculation unit that calculates posture information of the driven part based on the shaking detection result and the drive information.

2. The posture information calculation unit 2. The posture information acquisition device according to claim 1, further comprising: a signal obtained by extracting high frequency components from the vibration detection result; and a posture information of the driven part is calculated based on the drive information.

3. The posture information calculation unit 3. The posture information acquisition device according to claim 2, further comprising: a signal obtained by adding a high-frequency component extracted from the vibration detection result to the drive information, and calculating posture information of the driven part based on the signal.

4. the attitude information calculation unit calculates information about an attitude of the driven part based on an output signal obtained by inputting the drive information and the vibration detection result to a filter; 2. The attitude information acquisition apparatus according to claim 1, wherein the filter is a filter for estimating a state of the system from an input value including an error.

5. 5. The attitude information acquisition apparatus according to claim 4, wherein the filter is a Kalman filter.

6. 2. The attitude information acquisition device according to claim 1, wherein the shake detection result is an angular velocity signal.

7. 2. The attitude information acquisition device according to claim 1, wherein the driven part is driven to perform at least one of panning, tilting and rolling with respect to the base.

8. The posture information acquisition device according to claim 1 , wherein the drive information is information indicating a position of the driven part relative to the base.

9. A posture information acquisition device comprising: the posture information acquisition device according to claim 1 ; the base; and the driven part; The imaging device according to the present invention, wherein the driven portion has an imaging portion including an imaging element that captures an image of light from an imaging optical system.

10. A posture information acquisition device according to any one of claims 1 to 8, The base, the driven part, a control unit for correcting an image blur of an image captured by an imaging unit of the driven unit based on a blur correction amount; the attitude information calculation unit calculates attitude information of the driven part based on the drive information, the shake detection result, and the shake correction amount; The imaging device according to the present invention, wherein the driven portion has an imaging portion including an imaging element that captures an image of light from an imaging optical system.

11. The imaging device described in claim 10, characterized in that the orientation information calculation unit calculates the orientation information of the driven part based on the shake component corresponding to an image in which at least a portion of the influence of the shake component has been corrected, obtained based on the shake detection result and the shake correction amount, and based on the drive information.

12. 12. The imaging apparatus according to claim 11, wherein the orientation information calculation section acquires the shake component by subtracting the shake correction amount from the shake detection result.

13. acquiring a vibration detection result from a vibration detection unit provided on a driven part whose attitude can be changed with respect to a base and configured to detect vibration applied to the driven part; acquiring drive information of the driven part; and calculating attitude information of the driven part based on the shaking detection result and the drive information.

Images