Imaging apparatus, control method for imaging apparatus, and control program

Abstract

To provide an imaging apparatus including a rotation mechanism in a roll direction in addition to a mechanism driven in panning / tilting / zooming directions, which is capable of properly controlling each mechanism according to an actual movement of a subject, even when the subject moves.SOLUTION: An imaging apparatus 10 includes a main body part 11, an imaging element 12, a rotation mechanism 13, a panning direction drive mechanism 14a and a tilting direction drive mechanism 14b, a lens drive mechanism 15, a feature point extraction part 21, a distance / field angle information acquisition part 22, an actual-distance-for-each-pixel calculation part 23, a tracking target movement distance / speed calculation part 24, and a drive control part 25. The drive control part 25 controls at least one of the rotation mechanism 13, the panning direction drive mechanism 14a and the tilting direction drive mechanism 14b, and the lens drive mechanism 15 according to an actual movement distance of a subject calculated in the tracking target movement distance / speed calculation part 24.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to, for example, an imaging device that captures video and the like, and a control method and control program for the imaging device. [Background technology]

[0002] In recent years, advances in image recognition technology have led to the use of imaging devices equipped with an automatic tracking function that pans, tilts, and zooms in accordance with the movement of a subject in a captured image. For example, Patent Document 1 discloses a control device that includes a detection means for detecting an object from an image captured by an imaging device, a control means for controlling a drive device so that the imaging direction of the imaging device is changed in accordance with the movement of the object detected by the detection means, and a determination means for determining movement information of the object detected by the detection means, and the control means controls the drive frequency of the drive device in accordance with the movement information determined by the determination means.

[0003] Such control devices lack the function to rotate in a direction perpendicular to the subject (roll direction) and the method to control the rotation in the roll direction based on the captured image, and therefore cannot rotate the image in accordance with the angle changes that accompany the movement of the subject. Patent Document 2 discloses a configuration in which a gyro sensor is used to level a captured image in an imaging system including an imaging device with a roll mechanism (platform head). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-52231 [Patent Document 2] Japanese Patent Application Laid-Open No. 2013-179431 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the imaging system described in Patent Document 2 has the following problems. In other words, in the imaging system disclosed in the above publication, although the imaging device is equipped with a roll mechanism (camera head) and can level the captured image using a gyro sensor, it is not possible to appropriately control each mechanism, including the roll mechanism, according to the captured image.

[0006] In particular, when a subject moves in the captured video, if an attempt is made to detect operational inputs such as gestures by the subject, it may be difficult to appropriately control each mechanism depending on the positional relationship with the subject. The object of the present disclosure is to provide an imaging device that has a rotation mechanism in the roll direction in addition to mechanisms driven in the pan / tilt / zoom directions, and that is capable of appropriately controlling each mechanism in accordance with the actual movement of the subject even when the subject moves, as well as a control method and a control program for the imaging device. [Means for solving the problem]

[0007] The imaging device according to the present disclosure includes a main body, a lens, an image sensor, a rotation mechanism, an attitude adjustment mechanism, a focus position drive mechanism, a feature point extraction unit, a relative position information acquisition unit, a distance information acquisition unit, a movement distance calculation unit, and a control unit. The lens is contained within the main body and has an optical axis. The image sensor converts light incident from the subject side through the lens into an electrical signal and outputs video data. The rotation mechanism rotates the main body around a roll axis, which serves as the center of rotation when rotating the main body in a direction approximately perpendicular to the subject. The attitude adjustment mechanism adjusts the orientation of the main body in the pan direction and the tilt direction. The focus position drive mechanism controls the focus position of the subject by changing the position of the lens. The feature point extraction unit extracts feature points of the subject included in the video data as a tracking target. The relative position information acquisition unit acquires relative position information of the subject in the video. The distance information acquisition unit acquires information regarding the distance to the subject for each pixel of the image sensor. The movement distance calculation unit calculates the actual movement distance of the tracked target based on the movement distance of the feature point extracted by the feature point extraction unit, the relative position information of the target in the video acquired by the relative position information acquisition unit, and the distance to the target for each pixel acquired by the distance information acquisition unit. The control unit controls at least one of the rotation mechanism, the attitude adjustment mechanism, and the focus position drive mechanism based on the actual movement distance calculated by the movement distance calculation unit. [Effects of the Invention]

[0008] According to the imaging device of the present disclosure, in an imaging device that has a rotation mechanism in the roll direction in addition to mechanisms that are driven in the pan / tilt / zoom directions, even when the subject moves, each mechanism can be appropriately controlled in accordance with the actual movement of the subject. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is an overall perspective view showing a configuration of an imaging device according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a control block diagram of the imaging device of FIG. 1. [Figure 3]2 is a conceptual diagram showing an example of controlling the imaging device of FIG. 1 in the pan / tilt direction by a specific gesture. [Figure 4] 2 is a conceptual diagram showing an example of zoom control performed on the imaging device of FIG. 1 by a specific gesture. [Figure 5] 2 is a conceptual diagram showing an example of controlling the imaging device in FIG. 1 in the roll direction by a specific gesture. [Figure 6] 2 is an explanatory diagram showing a method for calculating the actual movement distance of a subject in the imaging device of FIG. 1; [Figure 7] 2 is an explanatory diagram showing a method for calculating the actual moving distance of a subject when driven in the pan / tilt direction in the imaging device of FIG. 1; [Figure 8] 2 is a diagram showing the relationship between gestures and functions when performing an external operation input to the imaging device of FIG. 1; [Figure 9] 4 is a flowchart showing an example of processing of a control method for the image capture device of FIG. 1 (position control based on the relative movement distance of a tracked feature point); [Figure 10] 4 is a flowchart showing an example of processing (speed control based on the relative movement distance of a tracking target feature point) of a control method for the imaging device of FIG. [Figure 11] FIG. 10 is a control block diagram showing the configuration of an imaging device according to another embodiment of the present disclosure. [Figure 12] 12 is a flowchart showing an example of processing in a control method for the imaging device of FIG. 11 (position control based on the angle of each of the pan / tilt / zoom driving units). [Figure 13] 12 is a flowchart showing an example of processing (speed control based on the angle of each of the pan / tilt / zoom driving units) in a control method for the imaging device of FIG. 11; [Figure 14] FIG. 10 is a control block diagram showing the configuration of an imaging device according to still another embodiment of the present disclosure. [Figure 15] 15 is a flowchart showing an example of processing (position control based on acceleration obtained from a gyro sensor) in a control method for the imaging device of FIG. 14; [Figure 16]15 is a flowchart showing an example of processing (speed control based on acceleration obtained from a gyro sensor) in a control method for the imaging device of FIG. 14; DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, more detailed description than necessary may be omitted. For example, detailed description of well-known matters or redundant description of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. The applicant provides the accompanying drawings and the following description to enable those skilled in the art to fully understand the present disclosure, and does not intend for them to limit the subject matter described in the claims.

[0011] (Embodiment 1) An imaging device according to an embodiment of the present disclosure will be described below with reference to FIGS. (1) Overall Configuration of the Imaging Device 10 An imaging device 10 according to this embodiment is installed, for example, in a photography studio, a conference hall, an event venue, a sports facility, etc., and is controlled by a controller such as a PC (Personal Computer) to perform various types of imaging. As shown in Fig. 1, the imaging device 10 performs imaging while switching the imaging direction among the pan direction, tilt direction, and roll direction. Also, as shown in Fig. 1, the imaging device 10 has multiple optical lenses (including a zoom lens L1 and a focus lens L2) built into the main body 11, and performs imaging by changing the imaging range (zoom range) and focus position by controlling the relative positions of the optical lenses to change.

[0012] Furthermore, the imaging device 10 of this embodiment detects gestures of the subject (person) to be photographed, and is controlled to be driven in the pan / tilt direction, roll direction, and zoom direction according to the movement of the subject (person). Here, the images captured by the imaging device 10 include, for example, both moving images and continuously captured still images.

[0013] As shown in Figures 1 and 2, the imaging device 10 includes a main body 11, an imaging element 12, a rotation mechanism 13, a pan direction drive mechanism (attitude adjustment mechanism) 14a, a tilt direction drive mechanism (attitude adjustment mechanism) 14b, a lens drive mechanism 15, a base 16, a rotation unit 17, a distance measuring device 20, a feature point extraction unit (relative position information acquisition unit) 21, a distance / angle of view information acquisition unit (distance information acquisition unit) 22, a pixel-by-pixel actual distance calculation unit 23, a tracked object movement distance / speed calculation unit (movement distance calculation unit) 24, a drive control unit (control unit) 25, and a drive motor 26.

[0014] The main body 11 is a substantially cylindrical member that houses multiple optical lenses (including a zoom lens L1 and a focus lens L2) and an image sensor 12. As shown in Fig. 1, the main body 11 is driven to rotate in the pan direction by a pan direction drive mechanism 14a, and driven to change its orientation up and down in the tilt direction by a tilt direction drive mechanism 14b. Furthermore, as shown in Fig. 1, the main body 11 is driven to rotate in the roll direction by a rotation mechanism 13.

[0015] This allows the imaging device 10 to take pictures while switching the direction in which it takes pictures. As shown in FIG. 2, the image sensor 12 is provided in the main body 11, and converts light incident thereon through multiple optical lenses including a zoom lens L1 and a focus lens L2 into a signal to generate digital image data of the subject.

[0016] The rotation mechanism 13 rotates the main body 11 around a roll axis 13a shown in Fig. 1. This allows the subject to be photographed while the image is being rotated. 1, the pan direction drive mechanism (attitude adjustment mechanism) 14a drives the main body 11 and the swivel unit 17 to rotate relative to the base unit 16. This allows shooting while changing the shooting direction in the left and right directions relative to the subject.

[0017] 1, tilt direction drive mechanism (attitude adjustment mechanism) 14b drives main body 11 to rotate in the vertical direction relative to swivel unit 17. This makes it possible to take pictures while changing the shooting direction in the vertical direction relative to the subject. The lens driving mechanism (focus position driving mechanism) 15 is contained within the main body 11 and drives a plurality of optical lenses, including the zoom lens L1 and the focus lens L2, to change their relative positions. This allows the focal position relative to the subject to be adjusted and the shooting range (zoom range) including the subject to be changed.

[0018] The base 16 is a member that constitutes the lower portion of the imaging device 10, and is configured so that the swivel unit 17 is swivelably driven relative to the base 16. The swivel unit 17 is an arm-shaped member attached to the base unit 16 in a swivelable manner, and is connected to the base unit 16 at its lower part, and supports the main body unit 11 in a rotatable manner at its upper part.

[0019] 2, the distance measuring device 20 is provided in the main body 11 and is, for example, a phase detection auto focus (PDAF) or a time of flight (ToF) sensor, and measures the distance from the imaging device 10 to a subject. Information on the distance to the subject measured by the distance measuring device 20 is used when the imaging device 10 is operated by a gesture (body movement) of the subject (described later).

[0020] The feature point extraction unit (relative position information acquisition unit) 21 extracts feature points of the subject included in the video data as the tracking target. For example, in the shooting range shown in Fig. 3, the feature point extraction unit 21 extracts the hand at the end of the arm of the subject (person) P1 as the feature point to be tracked. In addition, in order to recognize the relative position of the feature point to be tracked, the feature point extraction unit 21 extracts the center position or linear edge portion of a characteristic object in the background of the subject (person) P1 shown in Fig. 3 as the background feature point.

[0021] Here, among the feature points extracted by the feature point extracting unit 21, the feature point of the hand portion of the subject (person) P1 is set as the means (tracking target) for operating the imaging device 10. The feature point extraction unit 21 can extract, for example, any one of the face, torso, arms, and fingers of the subject (person) as a feature point. As described above, the feature point extraction unit 21 extracts feature points from stationary background parts in order to detect the relative positions in the video of feature points (such as a human hand) set as tracking targets. The background parts from which feature points are extracted include objects with characteristic shapes, linear edge parts, etc. The feature point extraction unit 21 extracts, for example, linear edge parts as a set of feature points using a Hough transform.

[0022] The distance / angle of view information acquisition unit (distance information acquisition unit) 22 acquires the "actual distance (distance information) from the imaging device 10 to the subject" measured by the distance measuring device 20, and also acquires "information on the angle of view at the time of shooting" from the lens driving mechanism 15 and transmits it to the actual distance per pixel calculation unit 23. The pixel-by-pixel actual distance calculation unit 23 calculates the actual distance to the subject for each pixel included in the imaging element 12 based on the information on the actual distance to the subject received from the distance / angle of view information acquisition unit 22 and the information on the angle of view at the time of shooting.

[0023] The tracked object movement distance / speed calculation unit (movement distance calculation unit) 24 calculates the actual movement distance (or speed) of the tracked object (part of the person's hand) based on information on the coordinates of the feature points (feature points of the tracked object and feature points of the background) set on the object in the video from the feature point extraction unit 21 and information on the distance to the object for each pixel calculated by the pixel-by-pixel actual distance calculation unit 23. That is, the tracking target movement distance / speed calculation unit 24 calculates the actual movement distance of the tracking target by eliminating the influence of the movement of the subject in the video and the driving of the imaging device 10 in the pan / tilt, roll and zoom directions.

[0024] In this embodiment, the tracked object moving distance / speed calculation unit 24 may be configured to calculate only the actual moving distance of the tracked object, without calculating the moving speed based on the moving distance. Alternatively, the tracked object moving distance / speed calculation unit 24 may be configured to calculate only the actual moving speed of the tracked object, without calculating the moving distance. The drive control unit (control unit) 25 is provided in each of the above-mentioned rotation mechanism 13, pan direction drive mechanism 14a, tilt direction drive mechanism 14b, and lens drive mechanism 15, and controls the drive motors 26 mounted on each mechanism 13, 14a, 14b, and 15 to control the attitude of the main body unit 11.

[0025] In the imaging device 10 of this embodiment, as shown in FIGS. 3, 4 and 5, in order to perform operations in the pan / tilt direction, the zoom direction and the roll direction from the subject side, the drive control unit 25 controls operations in the pan / tilt direction, the roll direction and the zoom direction in accordance with the actual movement distance (or speed) of the tracked object calculated by the tracked object movement distance / speed calculation unit 24 described above.

[0026] That is, in this embodiment, when the hand portion of the subject (person) P1 is set as the tracking target, the actual movement distance is calculated excluding the influence of the subject's movement and the driving of the imaging device 10 in the pan / tilt direction, roll direction, and zoom direction, thereby accurately detecting the distance that the hand portion, which is the tracking target, has actually moved. The drive motors 26 are provided in the rotation mechanism 13, pan direction drive mechanism 14a, tilt direction drive mechanism 14b, and lens drive mechanism 15, and are driven when the attitude of the main body 11 is changed.

[0027] In FIG. 2, for the sake of convenience, the drive control unit 25 and the drive motor 26 are shown as a single component, but in reality, they are provided as separate components in the rotation mechanism 13, pan direction drive mechanism 14a, tilt direction drive mechanism 14b, and lens drive mechanism 15. The control of the imaging device 10 of this embodiment, which is operated by a gesture of a subject (person), will be described below.

[0028] For example, when the imaging device 10 is driven in the pan direction by a predetermined gesture (moving one hand from left to right) of a preset subject (person), the subject's (person's) hand (tracking target point) and at least one background feature point are set as shown in FIG. 3, and the main body 11 is controlled to rotate in the pan direction (the shooting range slides sideways) according to the movement distance (or speed) of the detected tracking target point.

[0029] The same applies to external operation input control in the tilt direction. Note that the feature points of the background may be set to characteristic objects other than the person shown in Fig. 3, or straight edge portions may be set as a collection of feature points. Also, the feature points of the background may be immobile parts of the subject (person) other than the hand portion to be tracked. Next, when the imaging device 10 is driven in the zoom direction by another gesture of the subject (person) that has been set in advance (spreading both hands to both sides), as shown in FIG. 4, both hands (tracking target points) of the subject (person) and at least one background feature point are set, and it is detected that the detected tracking targets have moved simultaneously in opposite directions, and multiple optical lenses including the zoom lens L1 are moved in the zoom direction according to the moving distance (or speed) to control the imaging range to expand.

[0030] Furthermore, when the imaging device 10 is driven in the roll direction by yet another gesture (moving one arm up and down) of the subject (person) that has been set in advance, as shown in FIG. 5, the subject's (person's) hand and shoulder are set as the tracking target points, and a straight edge is set as the background feature point. A change in the angle of the straight line connecting the two points of the detected tracking target (hand and shoulder) with respect to the straight edge (background feature point) is detected, and the main body 11 is controlled to rotate in the roll direction (the shooting range is rotated) according to the magnitude of that angle.

[0031] In addition, in the drive control in the roll direction shown in Figure 5, control is possible using only information about the angle between the line connecting the two feature points and the line portion of the background, so distance information obtained from the distance measuring device 20 is not used. Furthermore, the extraction of feature points from the video described above can be performed by using, for example, known object recognition technology to extract skeletal feature points and obtain straight line edges from the captured video.

[0032] Here, a method for calculating the actual movement distance of the feature point to be tracked of the subject as viewed from the imaging device 10 will be described below with reference to FIGS. 6, if θview (calculated from the focal length), Lt (calculated from the measurement results of the distance measuring device 20), and x (calculated using trigonometric functions from θview and Lt), the distance to the subject is calculated for each pixel of the image sensor 12, and the actual movement distance of the subject is calculated from the amount of movement in the image. Note that if the amount by which the subject moved in each frame of the image is known, the movement speed of the subject can be calculated.

[0033] In this embodiment, distortion at the edges of the screen due to aberration is not taken into consideration, but actual movement may be calculated using distortion-corrected images as needed. Subsequently, when the imaging device 10 is driven in the pan direction, as shown in FIG. 7, the state before driving in the pan direction is as follows: θv0: angle of view (calculated from the lens focal length), L0: distance to the subject (calculated from the measurement results of the distance measuring device 20), x0: horizontal coordinate of the subject (tracking target feature point), H0: The angle of view θv0 is converted into the vertical / horizontal length of the shooting range at a specific distance a from the camera, and The state after driving in the pan direction is as follows: θv1: angle of view (calculated from the lens focal length), L1: distance to the subject (calculated from the measurement results of the distance measuring device 20), x1: horizontal coordinate of the subject (tracking target feature point), d: coordinate change of the tracked object due to driving in the pan direction, θ Pan: Pan change amount (obtained from encoder / IMU), H1: The angle of view θv1 converted into the vertical / horizontal length of the shooting range at a specific distance a from the camera, Then, based on the actual movement distance of the subject and the change in the angle of view (H0 / H1), the actual movement distance d for each pixel is calculated by the following relational expression:

[0034]

number

[0035] Therefore, when the imaging device 10 is operated externally by the gestures of the subject (person), the actual movement distance of the feature point (person's hand, etc.) to be tracked of the subject is calculated without being affected by changes in the shooting range due to the operation of the imaging device 10, and the imaging device 10 is operated according to the actual movement distance, allowing the operator to intuitively operate the imaging device 10 without feeling any discomfort.

[0036] Here, the switching of the control mode when the imaging device 10 is operated from the outside will be described below with reference to FIG. That is, when starting position control of the image capture device 10 using the movement distance of the tracked target as an input, in order to drive the pan / tilt direction drive axes (pan direction drive mechanism 14a and tilt direction drive mechanism 14b), the control is started when the image capture device 10 detects that the index finger and thumb of one hand are extended, as shown in FIG. 8.

[0037] Similarly, when starting position control of the imaging device 10 using the movement distance of the tracked object as input, if the drive axis in the zoom direction (lens drive mechanism 15) is to be driven, the control is started when the imaging device 10 detects that the index fingers of both hands are extended, as shown in Figure 8. Furthermore, when starting position control of the imaging device 10 using the movement distance of the tracked object as input, if the drive shaft in the roll direction (rotation mechanism 13) is to be driven, the control is started when the imaging device 10 detects that the index finger of one hand is extended, as shown in FIG. 8.

[0038] Next, when starting position control of the image capture device 10 using the movement speed of the tracked target as an input, in order to drive the pan / tilt direction drive axes (pan direction drive mechanism 14a and tilt direction drive mechanism 14b), the control is started when the image capture device 10 detects that the index finger and little finger of one hand are extended, as shown in FIG. 8. Similarly, when starting position control of the imaging device 10 using the movement speed of the tracked object as input, if the drive axis in the zoom direction (lens drive mechanism 15) is to be driven, the control is started when the imaging device 10 detects that the index finger and little finger of both hands are extended, as shown in FIG. 8.

[0039] Furthermore, when starting position control of the image capture device 10 using the movement speed of the tracked object as input, if the roll direction drive shaft (rotation mechanism 13) is to be driven, the control is started when the image capture device 10 detects that the index finger and middle finger of one hand are extended, as shown in FIG. 8. Next, when starting speed control of the image capture device 10 using the movement distance of the tracked target as an input, if the pan / tilt direction drive axes (pan direction drive mechanism 14a and tilt direction drive mechanism 14b) are to be driven, the control is started when the image capture device 10 detects that the five fingers of one hand are spread apart, as shown in FIG. 8.

[0040] Similarly, when starting speed control of the imaging device 10 using the movement distance of the tracked object as input, if the drive axis in the zoom direction (lens drive mechanism 15) is driven, the control is started when the imaging device 10 detects that the five fingers of both hands are spread apart, as shown in Figure 8. Furthermore, when starting speed control of the image capture device 10 using the movement distance of the tracked object as input, if the drive shaft in the roll direction (rotation mechanism 13) is driven, the control is started when the image capture device 10 detects that the index finger, thumb, and middle finger of one hand are extended, as shown in FIG. 8.

[0041] Furthermore, when external operation input to the imaging device 10 is to be terminated, the imaging device 10 detects that five fingers of one hand are closed, as shown in FIG. 8, and the external operation input control is terminated regardless of the control mode at that time. <Method for controlling the imaging device 10> The control method for the imaging device 10 of this embodiment will be described below with reference to the flowcharts shown in FIGS.

[0042] That is, as shown in FIG. 9, in position control in which distance is input, in order to calculate the actual movement distance of the subject (tracked feature point), in step S11, the imaging device 10 starts acquiring video information, and in step S12, the feature point extraction unit 21 performs feature point / straight edge detection processing of the subject by object recognition. Next, in step S13, the feature point extraction unit 21 acquires coordinate information of all feature points and straight line edges in the video.

[0043] Next, in step S14, the feature point extraction unit 21 determines whether or not a specific feature point pattern has been detected. If the feature point extraction unit 21 has detected a specific feature point pattern, the process proceeds to step S15; if not, the process returns to step S12. Next, in step S15, when the shape of the fingers of a hand or the like shown in FIG. 8, which has been registered in advance as a specific feature point pattern, is detected, in step S16, the control mode and the drive axes to be controlled (pan direction drive mechanism 14a, tilt direction drive mechanism 14b, rotation mechanism 13, lens drive mechanism 15) are set, and a timer is started.

[0044] Next, in step S17, input of the motion of the subject (tracking target feature point) and acquisition of the coordinates of the background feature point are started according to the control mode and drive axis to be controlled set in step S16. Next, in step S18, the distance / angle of view information acquisition unit 22 acquires focal length (angle of view) information of the zoom lens L1.

[0045] Next, in step S19, the distance / angle of view information acquisition unit 22 acquires the “distance between the subject and the imaging device 10” from the distance measurement device 20. Next, in step S20, the pixel-by-pixel actual distance calculation unit 23 calculates the actual movement distance (actual movement distance) of the subject (tracking target feature point) for each pixel in the image sensor 12. The processing in steps S18 to S20 is processing for calculating the actual movement distance of the subject (tracking target feature point) from the movement distance of the subject (tracking target feature point) in the video when the distance from the imaging device 10 to the subject changes.

[0046] Next, in step S21, it is determined whether a feature point pattern indicating the end of a gesture action input has been detected in the video. If a feature point pattern has been detected, the process proceeds to step S22; if not, the process returns to step S18. Next, in step S22, since it is determined in step S21 that a feature point pattern indicating the end of motion input has been detected, input of the subject's motion and acquisition of the coordinates of the background feature points are terminated, and the timer is stopped.

[0047] Next, in step S23, the tracked object moving distance / speed calculation unit 24 calculates the moving distance of the subject (tracked object feature point) in the video caused by the operation or movement of the imaging device 10. Next, in step S24, the tracking target moving distance / speed calculation unit 24 calculates the actual moving distance (actual moving distance) or speed of the subject (tracking target feature point) using the moving distance calculated in step S23 and the elapsed time of the timer.

[0048] Next, in step S25, the tracked object moving distance / speed calculation unit 24 inputs the actual moving distance (actual moving distance) or speed of the subject (tracked object feature point) to the drive control unit 25. Next, in step S26, the drive control unit 25 controls the drive in the pan / tilt direction, roll direction, and zoom direction based on the actual movement distance (actual movement distance) or speed of the subject (tracking target feature point) input in step S25.

[0049] Next, in step S27, it is determined whether the drive of each drive shaft is at the mechanical limit position of the device, and if it is not at the limit position, proceed to step S28, and if it is at the limit position, proceed to step S29 to stop the drive, and then proceed to step S28. Next, in step S28, since it was determined in step S27 that the device was not at its mechanical limit position, it is determined whether or not to terminate the moving object recognition in the video. If it is determined that the recognition is to be terminated, the control is terminated, but if it is determined that the recognition is not to be terminated, the process returns to step S12 and the subsequent processes are repeated.

[0050] Next, as shown in FIG. 10, in speed control in which distance is input and speed is output, in order to calculate the actual movement distance of the subject (tracked feature point), in step S31, the imaging device 10 starts acquiring video information, and in step S32, the feature point extraction unit 21 performs feature point / straight edge detection processing of the subject by object recognition. Next, in step S33, the feature point extraction unit 21 acquires coordinate information of all feature points and straight line edges in the video.

[0051] Next, in step S34, the feature point extraction unit 21 determines whether or not a specific feature point pattern has been detected. If the feature point extraction unit 21 has detected a specific feature point pattern, the process proceeds to step S35; if not, the process returns to step S32. Next, in step S35, when the shape of the fingers of a hand or the like shown in FIG. 8, which has been registered in advance as a specific feature point pattern, is detected, in step S36, the control mode and the drive axes to be controlled (pan direction drive mechanism 14a, tilt direction drive mechanism 14b, rotation mechanism 13, lens drive mechanism 15) are set.

[0052] Next, in step S37, input of the motion of the subject (tracking target feature point) and acquisition of the coordinates of the background feature point are started according to the control mode and drive axis to be controlled set in step S36. Next, in step S38, the distance / angle of view information acquisition unit 22 acquires focal length (angle of view) information of the zoom lens L1.

[0053] Next, in step S39, the distance / angle of view information acquisition unit 22 acquires the “distance between the subject and the imaging device 10” from the distance measurement device 20. Next, in step S40, the pixel-by-pixel actual distance calculation unit 23 calculates the actual movement distance (actual movement distance) of the subject (tracking target feature point) for each pixel in the image sensor 12. The processing in steps S38 to S40, similar to steps S18 to S20 described above, is processing for calculating the actual movement distance of the subject (tracking target feature point) from the movement distance of the subject (tracking target feature point) in the video when the distance from the imaging device 10 to the subject changes.

[0054] Next, in step S41, the tracked object moving distance / speed calculation unit 24 calculates the moving distance of the subject (tracked object feature point) in the video caused by the operation or movement of the imaging device 10. Next, in step S42, the tracked object moving distance / speed calculation unit 24 calculates the actual moving distance (real moving distance) of the subject (tracked object feature point). Next, in step S 43 , the tracked object moving distance / speed calculation unit 24 inputs the actual moving distance (real moving distance) of the subject (tracked object feature point) to the drive control unit 25 .

[0055] Next, in step S44, the drive control unit 25 controls the drive in the pan / tilt direction, roll direction, and zoom direction based on the actual movement distance (real movement distance) of the subject (tracking target feature point) input in step S44. Next, in step S45, it is determined whether the drive of each drive shaft is at the mechanical limit position of the device, and if it is not at the limit position, the process proceeds to step S46, and if it is at the limit position, the process proceeds to step S49 to stop the drive, and then the process proceeds to step S46.

[0056] Next, in step S46, since it is determined in step S45 that the device is not at its mechanical limit position, it is determined whether a feature point pattern indicating the end of a gesture action input has been detected in the image. If a feature point pattern is detected, the process proceeds to step S47; if not, the process returns to step S38. Next, in step S47, since it is determined in step S46 that a feature point pattern indicating the end of action input has been detected, the input of the subject's action and the acquisition of the coordinates of the background feature points are terminated.

[0057] Next, in step S48, it is determined whether or not to end the recognition of a moving object in the video. If it is determined that the recognition is to be ended, the control ends immediately, but if it is determined that the recognition is not to be ended, the process returns to step S32 and the subsequent processes are repeated. As a result, in the speed control shown in FIG. 10, compared to the position control shown in FIG. 9, operation acceptance control according to a gesture can be started before the movement of the tracking target feature point of the subject (person) ends.

[0058] <Major features> As shown in FIG. 2, the imaging device 10 of this embodiment includes a main body 11, lenses L1 and L2, an image sensor 12, a rotation mechanism 13, a pan direction drive mechanism 14a, a tilt direction drive mechanism 14b, a lens drive mechanism 15, a feature point extraction unit 21, a distance / angle of view information acquisition unit 22, a pixel-by-pixel actual distance calculation unit 23, a tracked object movement distance / velocity calculation unit 24, and a drive control unit 25. The lenses L1 and L2 are contained within the main body 11 and have an optical axis. The image sensor 12 converts light incident from the subject side through the lenses L1 and L2 into an electrical signal and outputs video data. The rotation mechanism 13 rotates the main body 11 around a roll axis in a direction approximately perpendicular to the subject. The pan direction drive mechanism 14a and the tilt direction drive mechanism 14b adjust the orientation of the main body 11 in the pan direction and the tilt direction. The lens driving mechanism 15 controls the focal position of the subject by changing the positions of the lenses L1 and L2. The feature point extraction unit 21 extracts feature points of the subject included in the video data as a tracking target. The distance and angle of view information acquisition unit 22 acquires relative position information of the subject in the video. The pixel-by-pixel actual distance calculation unit 23 acquires information on the distance to the subject for each pixel of the image sensor. The tracked target movement distance and speed calculation unit 24 calculates the actual movement distance of the tracked target based on the movement distance of the feature point extracted by the feature point extraction unit 21, the relative position information of the subject in the video acquired by the distance and angle of view information acquisition unit 22, and the distance to the subject for each pixel acquired by the pixel-by-pixel actual distance calculation unit 23. The drive control unit 25 controls at least one of the rotation mechanism 13, the pan direction driving mechanism 14a, the tilt direction driving mechanism 14b, and the lens driving mechanism 15 based on the actual movement distance calculated by the tracked target movement distance and speed calculation unit 24.

[0059] This makes it possible, for example, to use object recognition technology to capture feature points or straight line components of a subject or the like in a captured image as a tracking target, and calculate the actual moving distance (or actual speed) of the tracking target (e.g., a hand) based on the amount of change in the tracking target in the image, the distance to the tracking target measured by the distance measuring device 20, and the lens angle of view information, thereby controlling the operation of the imaging device 10 in each of the pan, tilt, zoom, and roll (PTZR) directions.

[0060] That is, in a video captured while moving in each of the pan / tilt / zoom / roll (PTZR) directions, for example, the hand portion of the subject (person) P1 moves within the video due to the influence of the movement of the imaging device 10 in each direction in addition to the actual movement of the hand portion, making it difficult to intuitively operate the imaging device 10 using gestures, etc.

[0061] Therefore, in the imaging device 10 of this embodiment, by detecting changes in the actual distance from the imaging device 10 to the subject, and accurately detecting only changes in the position of the actual tracking target (for example, the hand), the imaging device 10 can be operated by a specified gesture that moves the hand of the subject (person). In other words, by setting the control parameter to the actual movement distance (or actual speed) of the feature point, the operator's operation is not affected by changes in the angle of view or the distance to the subject, and the operator can operate the imaging device 10 intuitively.

[0062] As a result, in the imaging device 10 equipped with mechanisms that are driven in the pan / tilt / zoom / roll directions, when there is movement of the subject, each mechanism can be appropriately controlled in accordance with the actual movement of the subject. [Other embodiments] Although one embodiment of the present disclosure has been described above, the present disclosure is not limited to the above embodiment, and various modifications are possible without departing from the gist of the disclosure.

[0063] (A) In the above embodiment, examples of the imaging device and its control method that realize the present disclosure have been described, but the present disclosure is not limited to this. For example, the present disclosure may be realized as a control program that causes a computer to execute the above-described method for controlling an imaging device.

[0064] This control program is stored in a memory (storage unit) installed in the imaging device, and the CPU reads the control program stored in the memory and causes the hardware to execute each step. More specifically, the CPU reads the control program and executes the above-mentioned steps, thereby achieving the same effect as above. The present disclosure may also be realized as a recording medium storing a control program for an imaging device.

[0065] (B) In the above embodiment, an example has been described in which the configuration of the present disclosure is applied to the imaging device 10, which includes the pan direction drive mechanism 14a, the tilt direction drive mechanism 14b, the rotation mechanism 13, and the lens drive mechanism 15, and changes the shooting direction in the pan direction, tilt direction, roll direction, and zoom direction. However, the present disclosure is not limited to this.

[0066] For example, the configuration of the present disclosure may be applied to an imaging device that changes the shooting direction in the pan direction, tilt direction, and zoom direction, that is, an imaging device that does not have a drive mechanism in the roll direction. (C) In the above embodiment, an example was described in which the actual movement distance for each pixel was calculated by acquiring distance information to the subject measured by the distance measuring device 20. However, the present disclosure is not limited to this.

[0067] For example, as shown in FIG. 11, an imaging device 110 may be configured with a drive encoder 120 instead of a distance measuring device. The drive unit encoder 120 is provided in each of the pan direction drive mechanism 14a, tilt direction drive mechanism 14b, rotation mechanism 13, and lens drive mechanism 15, and detects the direction (pan / tilt), rotation angle (roll), (zoom), etc. (PTRZ position) of each drive motor 26, and outputs them to the tracked object movement distance / speed calculation unit 24.

[0068] This makes it possible to recognize the driving status (movement distance, direction, etc.) of the drive motor 26 of each mechanism without obtaining the distance to the subject obtained by a distance measuring device, and therefore to calculate the actual movement distance of the subject (tracked feature point) for each pixel of the image sensor 12. Here, the control method of the imaging device 110 shown in FIG. 11 will be described below with reference to FIGS.

[0069] That is, as shown in FIG. 12, in position control in which distance or speed is input and position is output, the processing of steps S11 to S16 described in the above embodiment is performed to calculate the actual movement distance of the subject (tracking target feature point). Next, in step S117, input of the motion of the subject (tracking target feature point) and acquisition of the rotation angle of the drive motor 26 are started according to the control mode and drive axis to be controlled set in step S16.

[0070] Next, in steps S18 to S21, the same processes as those in the first embodiment are performed. Next, in step S122, since it is determined in step S21 that a feature point pattern indicating the end of motion input has been detected, input of the subject's motion and acquisition of the rotation angle of drive motor 26 are terminated, and the timer is stopped. Next, in step S123, the tracked object moving distance / speed calculation unit 24 calculates the moving distance of the subject (tracked object feature point) in the video caused by the operation of the drive motor 26 of the imaging device 10.

[0071] The subsequent processes in steps S24 to S29 are the same as those in the above embodiment. As a result, by using the movement amount information acquired from the drive unit encoder 120 provided in each of the pan direction drive mechanism 14a, tilt direction drive mechanism 14b, rotation mechanism 13, and lens drive mechanism 15, the actual movement distance of the subject (tracking target feature point) can be calculated, as in the above-mentioned embodiment 1.

[0072] Similarly, as shown in FIG. 13, in speed control in which distance is input and speed is output, the processing of steps S31 to S36 described in the above embodiment is performed to calculate the actual movement distance of the subject (tracking target feature point). Next, in step S137, input of the motion of the subject (tracking target feature point) and acquisition of the rotation angle of the drive motor 26 are started according to the control mode and drive axis to be controlled set in step S36.

[0073] Next, in steps S37 to S40, the same processes as those in the first embodiment are performed. Next, in step S140, the tracked object moving distance / speed calculation unit 24 calculates the moving distance of the subject (tracked object feature point) in the video caused by the operation of the drive motor 26 of the imaging device 10. The subsequent processes in steps S42 to S46 are the same as those in the above embodiment.

[0074] Next, in step S147, since it is determined in step S46 that a feature point pattern indicating the end of motion input has been detected, input of the motion of the subject and acquisition of the rotation angle of drive motor 26 are terminated. Next, in step S48, it is determined whether or not to end the recognition of a moving object in the video. If it is determined that the recognition is to be ended, the control ends immediately, but if it is determined that the recognition is not to be ended, the process returns to step S32 and the subsequent processes are repeated.

[0075] As a result, in terms of speed control, the same effect as in the above embodiment can be achieved by acquiring the rotation angle of the drive motor 26 of each mechanism of the imaging device 110 instead of the distance information to the subject acquired from the distance measuring device 20. (D) In the above embodiment, an example was described in which the actual movement distance for each pixel was calculated by acquiring distance information to the subject measured by the distance measuring device 20. However, the present disclosure is not limited to this.

[0076] For example, as shown in FIG. 14, the imaging device 210 may be configured with a gyro sensor 220 and a distance-velocity-angle-angular velocity conversion unit 221 provided in each mechanism (pan direction drive mechanism 14a, tilt direction drive mechanism 14b, rotation mechanism 13, and lens drive mechanism 15) instead of a distance measuring device. The gyro sensor 220 is provided in each of the pan direction drive mechanism 14a, tilt direction drive mechanism 14b, rotation mechanism 13, and lens drive mechanism 15, and detects the angular velocity of each mechanism and outputs it to the tracked object movement distance / speed calculation unit 24.

[0077] This allows the acceleration of each mechanism measured by the gyro sensor to be converted into distance, velocity, angle, and angular velocity by the distance / velocity / angle / angular velocity conversion unit 221 and output, without the need to obtain the distance to the subject using a distance measuring device, so that the actual movement distance of the subject (tracking target feature point) for each pixel of the image sensor 12 can be calculated. Here, the control method of the imaging device 210 shown in FIG. 14 will be described below with reference to FIGS.

[0078] That is, as shown in FIG. 15, steps S11 to S16 are the same as those in the above embodiment. Next, in step S217, input of the subject's movement and acquisition of acceleration information from the gyro sensor 220 are started. Next, the processes in steps S18 to S21 are the same as those in the above embodiment.

[0079] Next, in step S222, the input of the subject's movement and the acquisition of acceleration information from the gyro sensor 220 are terminated, and the timer is stopped. Next, in step S223, the movement distance of the tracked target in the video due to the movement and rotation of the image capturing device 210 is calculated. The subsequent processes in steps S24 to S29 are the same as those in the above embodiment.

[0080] As a result, by using the movement acceleration obtained from the gyro sensor 220 provided in each of the pan direction drive mechanism 14a, tilt direction drive mechanism 14b, rotation mechanism 13, and lens drive mechanism 15, the actual movement distance of the subject (tracking target feature point) can be calculated, as in the above-mentioned embodiment 1. Similarly, as shown in FIG. 16, in speed control in which distance is input and speed is output, the processing of steps S31 to S36 described in the above embodiment is performed to calculate the actual movement distance of the subject (tracking target feature point).

[0081] Next, in step S237, input of the motion of the subject (tracking target feature point) and acquisition of the acceleration detected by the gyro sensor 220 are started according to the control mode and drive axis to be controlled set in step S35. Next, in steps S38 to S40, the same processes as those in the first embodiment are performed. Next, in step S241, the tracked object moving distance / speed calculation unit 24 calculates the moving distance of the subject (tracked object feature point) in the video caused by the operation of the drive motor 26 of the imaging device 10.

[0082] The subsequent processes in steps S42 to S46 are the same as those in the above embodiment. Next, in step S247, since it is determined in step S46 that a feature point pattern indicating the end of motion input has been detected, input of the motion of the subject and acquisition of the acceleration detected by gyro sensor 220 are terminated. Next, in step S48, it is determined whether or not to end the recognition of a moving object in the video. If it is determined that the recognition is to be ended, the control ends immediately, but if it is determined that the recognition is not to be ended, the process returns to step S32 and the subsequent processes are repeated.

[0083] As a result, in terms of speed control, instead of distance information to the subject obtained from the distance measuring device 20, acceleration is obtained from the gyro sensor 220 provided in each mechanism of the imaging device 110, thereby achieving the same effect as in the above embodiment. (E) In the above embodiment, an example has been described in which position control (for example, FIG. 9) and speed control (for example, FIG. 10) are performed as control of the imaging device 10. However, the present disclosure is not limited to this.

[0084] For example, the control of the imaging device may be configured to perform only one of position control and speed control. (F) In the above embodiment, an example has been described in which the imaging device 10 is externally operated with the subject's hand in the video, which has been extracted as a feature point, as the tracking target. However, the present disclosure is not limited to this.

[0085] For example, the tracking target may be other parts of the subject, such as the head, arms, shoulders, legs, etc. In addition, a plurality of tracking targets may be combined and controlled so as to operate the imaging device from the outside. (G) In the above embodiment, an example has been described in which the start, end, and switching of the control target of the operation control from the subject for controlling the drive in the pan / tilt direction, zoom direction, and roll direction are performed by detecting a predetermined hand shape, but the present disclosure is not limited to this.

[0086] For example, control of the operation from the subject may be started and ended by operation input from a separately provided remote controller, or the controlled object may be switched. (H) In the above embodiment, an example has been described in which the distance from the imaging device 10 to the subject is measured by the distance measuring device 20, and the actual moving distance (or speed) of the tracked target is calculated. However, the present disclosure is not limited to this.

[0087] For example, if the subject has a known size, the distance from the imaging device to the subject can be recognized based on the size of the subject in the image, so the configuration may not have a distance measuring device. (I) In the above embodiment, an example has been described in which the selection of the control mode, the setting of the drive axis to be controlled, and the timing of starting processing are performed simultaneously. However, the present disclosure is not limited to this.

[0088] For example, the selection of the control mode, the setting of the drive axis to be controlled, and the control of the start of processing may be started separately rather than at the same timing. That is, the control system may be configured so that after detecting a pattern for determining the control mode, a pattern for setting the drive axis is detected, and then a pattern indicating the start of processing is detected. The process of setting the control mode and the drive axis to be controlled may be set in advance, without relying on the detection of a predetermined operation (feature point) pattern by the operator.

[0089] (J) In the above embodiment, an example has been described in which an operation input from a subject for controlling drive in the pan / tilt direction, zoom direction, and roll direction is performed by detecting a predetermined action set in advance, as shown in Figures 3 to 5. However, the present disclosure is not limited to this. For example, the external operation input in each direction may be a movement other than the gestures shown in FIGS. 3 to 5 (for example, a movement of the fingers instead of the arms, or a movement of the whole body).

[0090] <Additional Notes> The above description of the embodiments discloses the following techniques. (Technology 1) The imaging device according to Technology 1 is a main body; a lens contained in the main body and having an optical axis; an imaging element that converts light incident from the subject side through the lens into an electrical signal and outputs video data; a rotation mechanism that rotates the main body around a roll axis that serves as a rotation center when the main body is rotated in a direction substantially perpendicular to the subject; an attitude adjustment mechanism that adjusts the orientation of the main body in a pan direction and a tilt direction; a focus position driving mechanism that changes the position of the lens to control the focus position of the subject; a feature point extraction unit that extracts feature points of a subject included in the video data as a tracking target; a relative position information acquisition unit that acquires relative position information of the subject in the video; a distance information acquisition unit that acquires information about a distance to the subject for each pixel of the imaging element; a movement distance calculation unit that calculates an actual movement distance of the tracked object in accordance with the movement distance of the feature point extracted by the feature point extraction unit, the relative position information of the object in the video acquired by the relative position information acquisition unit, and the distance to the object for each pixel acquired by the distance information acquisition unit; a control unit that controls at least one of the rotation mechanism, the attitude adjustment mechanism, and the focus position drive mechanism in accordance with the actual movement distance calculated by the movement distance calculation unit; It is equipped with:

[0091] (Technology 2) The imaging device according to the second aspect of the present invention is the imaging device according to the first aspect of the present invention, The relative position information acquisition unit acquires, as the relative position information, feature points or straight line components of the background of the subject included in the video. (Technology 3) The imaging device according to the third aspect of the present invention is an imaging device according to the first or second aspect of the present invention, the rotation mechanism, the attitude adjustment mechanism, and the focal position drive mechanism each have a drive unit and an encoder attached to the drive unit and outputting a movement amount; The relative position information acquisition unit acquires, as the relative position information, the movement amount information from the encoders of the rotation mechanism, the attitude adjustment mechanism, and the focus position drive mechanism.

[0092] (Technology 4) The imaging device according to the fourth aspect of the present invention is an imaging device according to any one of the first to third aspects of the present invention, An acceleration sensor is provided in the main body portion and measures the acceleration of the main body portion, The relative position information acquisition unit acquires acceleration information from the acceleration sensor as the relative position information.

[0093] (Technology 5) The imaging device according to technology 5 is an imaging device according to any one of technology 1 to 4, The feature point extraction unit extracts the feature points from any one of a face, a torso, an arm, and a finger of a person. (Technology 6) The imaging device according to Technology 6 is an imaging device according to any one of Technology 1 to Technology 5, The feature point extraction unit extracts the feature points from the background of the subject.

[0094] (Technology 7) The imaging device according to Technology 7 is an imaging device according to any one of Technology 1 to Technology 6, The feature point extraction unit detects straight edges by a Hough transform and extracts the feature points. (Technology 8) The imaging device according to technology 8 is an imaging device according to any one of technology 1 to 7, The relative position information acquisition unit acquires, as the relative position information of the subject, information on a distance to the subject and information on an angle of view when the subject is photographed.

[0095] (Technology 9) The imaging device according to Technology 9 is an imaging device according to any one of Technology 1 to Technology 8, When the control unit detects a specific movement of the subject, it starts controlling at least one of the rotation mechanism, the attitude adjustment mechanism, and the focus position drive mechanism according to the movement distance calculated by the movement distance calculation unit.

[0096] (Technology 10) An imaging device according to Technology 10 is an imaging device according to any one of Technology 1 to Technology 9, When the control unit detects a specific movement of the subject, it terminates control of at least one of the rotation mechanism, the attitude adjustment mechanism, and the focus position drive mechanism in accordance with the movement distance calculated by the movement distance calculation unit.

[0097] (Technology 11) An imaging device according to technology 11 is an imaging device according to any one of technology 1 to technology 10, When the control unit detects a specific movement of the subject, it changes the control targets of the rotation mechanism, the attitude adjustment mechanism, and the focus position drive mechanism according to the movement distance calculated by the movement distance calculation unit.

[0098] (Technology 12) An imaging device according to Technology 12 is an imaging device according to any one of Technology 1 to Technology 11, When the control unit detects that the imaging device is at the limit position in the driving range, it controls at least one of the rotation mechanism, the attitude adjustment mechanism, and the focus position driving mechanism to stop the driving. [Industrial Applicability]

[0099] The imaging device of the present disclosure is an imaging device that has a rotation mechanism in the roll direction in addition to mechanisms that are driven in the pan / tilt / zoom directions, and has the effect of being able to appropriately control each mechanism in accordance with the actual movement of the subject even when the subject moves, and therefore can be widely applied to various imaging devices in which operation input is made by gestures, etc. [Explanation of symbols]

[0100] 10. Imaging device 11 Main body 12 Image sensor 13 Rotation mechanism 13a Roll axis 14a Panning drive mechanism (attitude adjustment mechanism) 14b Tilt direction drive mechanism (attitude adjustment mechanism) 15 Lens drive mechanism (focus position drive mechanism) 16 Base 17 Swivel section 20 Ranging device 21 Feature point extraction unit (relative position information acquisition unit) 22 Distance and angle of view information acquisition unit (distance information acquisition unit) 23 Per-pixel real distance calculation unit 24 Tracking target movement distance and speed calculation unit (movement distance calculation unit) 25 Drive control unit (control unit) 26 Drive motor (drive unit) 110 Imaging device 120 Drive Encoder 122 Field of view information acquisition unit 210 Imaging device 220 Gyro Sensor 221 Distance, speed, angle, and angular velocity conversion section 222 Field of view information acquisition unit L1 Zoom Lens (Optical Lens) L2 Focus lens (optical lens)

Claims

1. a main body; a lens contained in the main body and having an optical axis; an imaging element that converts light incident from the subject side through the lens into an electrical signal and outputs video data; a rotation mechanism that rotates the main body around a roll axis that serves as a rotation center when the main body is rotated in a direction substantially perpendicular to the subject; an attitude adjustment mechanism that adjusts the orientation of the main body in a pan direction and a tilt direction; a focus position driving mechanism that changes the position of the lens to control the focus position of the subject; a feature point extraction unit that extracts feature points of a subject included in the video data as a tracking target; a relative position information acquisition unit that acquires relative position information of the subject in the video; a distance information acquisition unit that acquires information about a distance to the subject for each pixel of the imaging element; a movement distance calculation unit that calculates an actual movement distance of the tracked object in accordance with the movement distance of the feature point extracted by the feature point extraction unit, the relative position information of the object in the video acquired by the relative position information acquisition unit, and the distance to the object for each pixel acquired by the distance information acquisition unit; a control unit that controls at least one of the rotation mechanism, the attitude adjustment mechanism, and the focus position drive mechanism in accordance with the actual movement distance calculated by the movement distance calculation unit; An imaging device comprising:

2. the relative position information acquisition unit acquires, as the relative position information, feature points or straight line components of the background of the subject included in the video; The imaging device according to claim 1 .

3. the rotation mechanism, the attitude adjustment mechanism, and the focal position drive mechanism each have a drive unit and an encoder attached to the drive unit and outputting a movement amount; the relative position information acquisition unit acquires, as the relative position information, information on the movement amounts from the encoders of the rotation mechanism, the attitude adjustment mechanism, and the focus position drive mechanism.

3. The imaging device according to claim 1.

4. An acceleration sensor is provided in the main body portion and measures the acceleration of the main body portion, the relative position information acquisition unit acquires acceleration information from the acceleration sensor as the relative position information; 3. The imaging device according to claim 1.

5. the feature point extraction unit extracts the feature points from any one of a face, a torso, an arm, and a finger of a person; 3. The imaging device according to claim 1.

6. the feature point extraction unit extracts the feature points from the background of the subject; 3. The imaging device according to claim 1.

7. the feature point extraction unit detects straight edges by a Hough transform and extracts the feature points; 3. The imaging device according to claim 1.

8. the relative position information acquisition unit acquires, as the relative position information of the subject, information on a distance to the subject and information on an angle of view when photographing the subject; 3. The imaging device according to claim 1.

9. when the control unit detects a specific movement of the subject, the control unit starts control of at least one of the rotation mechanism, the attitude adjustment mechanism, and the focus position drive mechanism in accordance with the movement distance calculated by the movement distance calculation unit.

3. The imaging device according to claim 1.

10. when the control unit detects a specific movement of the subject, the control unit terminates control of at least one of the rotation mechanism, the attitude adjustment mechanism, and the focus position drive mechanism in accordance with the movement distance calculated by the movement distance calculation unit.

3. The imaging device according to claim 1.

11. When the control unit detects a specific movement of the subject, the control unit changes the control targets of the rotation mechanism, the attitude adjustment mechanism, and the focus position drive mechanism in accordance with the movement distance calculated by the movement distance calculation unit.

3. The imaging device according to claim 1.

12. when the control unit detects that the imaging device is at a limit position in a driving range thereof, the control unit controls at least one of the rotation mechanism, the attitude adjustment mechanism, and the focus position driving mechanism to stop the driving.

3. The imaging device according to claim 1.

13. a main body; a lens contained in the main body and having an optical axis; an imaging element that converts light incident from the subject side through the lens into an electrical signal and outputs video data; a roll axis that serves as a rotation center when the main body is rotated in a direction substantially perpendicular to the subject; a rotation mechanism that rotates the main body around the roll axis; an attitude adjustment mechanism that adjusts the orientation of the main body in a pan direction and a tilt direction; a focus position driving mechanism that changes the position of the lens to control the focus position of the subject; A control method for an imaging device comprising: a feature point extraction step of extracting feature points of a subject included in the video data as tracking targets; a relative position information acquisition step of acquiring relative position information of the subject in the video; a distance information acquisition step of acquiring information about a distance to the subject for each pixel of the imaging element; a movement distance calculation step of calculating an actual movement distance of the tracked object in accordance with the movement distance of the feature point extracted in the feature point extraction step, the relative position information of the object in the video acquired in the relative position information acquisition step, and the distance to the object for each pixel acquired in the distance information acquisition step; a control step of controlling at least one of the rotation mechanism, the attitude adjustment mechanism, and the focus position drive mechanism in accordance with the actual movement distance calculated in the movement distance calculation step; A control method for an imaging device comprising:

14. a main body; a lens contained in the main body and having an optical axis; an imaging element that converts light incident from the subject side through the lens into an electrical signal and outputs video data; a roll axis that serves as a rotation center when the main body is rotated in a direction substantially perpendicular to the subject; a rotation mechanism that rotates the main body around the roll axis; an attitude adjustment mechanism that adjusts the orientation of the main body in a pan direction and a tilt direction; a focus position driving mechanism that changes the position of the lens to control the focus position of the subject; A control program for an imaging device comprising: a feature point extraction step of extracting feature points of a subject included in the video data as tracking targets; a relative position information acquisition step of acquiring relative position information of the subject in the video; a distance information acquisition step of acquiring information about a distance to the subject for each pixel of the imaging element; a movement distance calculation step of calculating an actual movement distance of the tracked object in accordance with the movement distance of the feature point extracted in the feature point extraction step, the relative position information of the object in the video acquired in the relative position information acquisition step, and the distance to the object for each pixel acquired in the distance information acquisition step; a control step of controlling at least one of the rotation mechanism, the attitude adjustment mechanism, and the focus position drive mechanism in accordance with the actual movement distance calculated in the movement distance calculation step; A control program that causes a computer to execute a control method for an imaging apparatus comprising:

15. a main body; a lens contained in the main body and having an optical axis; an imaging element that converts light incident from the subject side through the lens into an electrical signal and outputs video data; a roll axis that serves as a rotation center when the main body is rotated in a direction substantially perpendicular to the subject; a rotation mechanism that rotates the main body around the roll axis; a feature point extraction unit that extracts at least two feature points of a subject included in the video data as tracking targets and extracts straight line portions of a background of the subject; a rotation angle calculation unit that calculates a rotation angle of the rotation mechanism according to an amount of change in an angle of a straight line connecting the feature points extracted by the feature point extraction unit with respect to a straight line portion of the background; a control unit that controls the rotation mechanism in accordance with the rotation angle calculated by the rotation angle calculation unit; An imaging device comprising:

16. a main body; a lens contained in the main body and having an optical axis; an imaging element that converts light incident from the subject side through the lens into an electrical signal and outputs video data; a rotation mechanism that rotates the main body around a roll axis that serves as a rotation center when the main body is rotated in a direction substantially perpendicular to the subject; A control method for an imaging device comprising: a feature point extraction step of extracting at least two feature points of a subject included in the video data as tracking targets and extracting straight line portions of a background of the subject; a rotation angle calculation step of calculating a rotation angle of the rotation mechanism according to an amount of change in an angle of a straight line connecting the feature points extracted in the feature point extraction step with respect to a straight line portion of the background; a control step of controlling the rotation mechanism in accordance with the rotation angle calculated in the rotation angle calculation step; A control method for an imaging device comprising:

17. a main body; a lens contained in the main body and having an optical axis; an imaging element that converts light incident from the subject side through the lens into an electrical signal and outputs video data; a rotation mechanism that rotates the main body around a roll axis that serves as a rotation center when the main body is rotated in a direction substantially perpendicular to the subject; A control program for an imaging device comprising: a feature point extraction step of extracting at least two feature points of a subject included in the video data as tracking targets and extracting straight line portions of a background of the subject; a rotation angle calculation step of calculating a rotation angle of the rotation mechanism according to an amount of change in an angle of a straight line connecting the feature points extracted in the feature point extraction step with respect to a straight line portion of the background; a control step of controlling the rotation mechanism in accordance with the rotation angle calculated in the rotation angle calculation step; A control program that causes a computer to execute a control method for an imaging apparatus comprising:

18. a main body; a lens contained in the main body and having an optical axis; an imaging element that converts light incident from the subject side through the lens into an electrical signal and outputs video data; an attitude adjustment mechanism that adjusts the orientation of the main body in a pan direction and a tilt direction; a focus position driving mechanism that changes the position of the lens to control the focus position of the subject; a feature point extraction unit that extracts feature points of a subject included in the video data as a tracking target; a relative position information acquisition unit that acquires relative position information of the subject in the video; a distance information acquisition unit that acquires information about a distance to the subject for each pixel of the imaging element; a movement distance calculation unit that calculates an actual movement distance of the tracked object in accordance with the movement distance of the feature point extracted by the feature point extraction unit, the relative position information of the object in the video acquired by the relative position information acquisition unit, and the distance to the object for each pixel acquired by the distance information acquisition unit; a control unit that controls at least one of the attitude adjustment mechanism and the focal position drive mechanism in accordance with the actual movement distance calculated by the movement distance calculation unit; An imaging device comprising:

19. a main body; a lens contained in the main body and having an optical axis; an imaging element that converts light incident from the subject side through the lens into an electrical signal and outputs video data; a roll axis that serves as a rotation center when the main body is rotated in a direction substantially perpendicular to the subject; an attitude adjustment mechanism that adjusts the orientation of the main body in a pan direction and a tilt direction; a focus position driving mechanism that changes the position of the lens to control the focus position of the subject; A control method for an imaging device comprising: a feature point extraction step of extracting feature points of a subject included in the video data as tracking targets; a relative position information acquisition step of acquiring relative position information of the subject in the video; a distance information acquisition step of acquiring information about a distance to the subject for each pixel of the imaging element; a movement distance calculation step of calculating an actual movement distance of the tracked object in accordance with the movement distance of the feature point extracted in the feature point extraction step, the relative position information of the object in the video acquired in the relative position information acquisition step, and the distance to the object for each pixel acquired in the distance information acquisition step; a control step of controlling at least one of the attitude adjustment mechanism and the focus position drive mechanism in accordance with the actual movement distance calculated in the movement distance calculation step; A control method for an imaging device comprising:

20. a main body; a lens contained in the main body and having an optical axis; an imaging element that converts light incident from the subject side through the lens into an electrical signal and outputs video data; a roll axis that serves as a rotation center when the main body is rotated in a direction substantially perpendicular to the subject; an attitude adjustment mechanism that adjusts the orientation of the main body in a pan direction and a tilt direction; a focus position driving mechanism that changes the position of the lens to control the focus position of the subject; A control program for an imaging device comprising: a feature point extraction step of extracting feature points of a subject included in the video data as tracking targets; a relative position information acquisition step of acquiring relative position information of the subject in the video; a distance information acquisition step of acquiring information about a distance to the subject for each pixel of the imaging element; a movement distance calculation step of calculating an actual movement distance of the tracked object in accordance with the movement distance of the feature point extracted in the feature point extraction step, the relative position information of the object in the video acquired in the relative position information acquisition step, and the distance to the object for each pixel acquired in the distance information acquisition step; a control step of controlling at least one of the attitude adjustment mechanism and the focus position drive mechanism in accordance with the actual movement distance calculated in the movement distance calculation step; A control program that causes a computer to execute a control method for an imaging apparatus comprising:

Images