Control device, imaging mobile device, and control method

Abstract

Reduces image shake caused by high-frequency vibrations of moving objects. [Solution] The control device 5 controls an imaging mobile body which includes a mobile body 1 having a drive unit 6, an imaging means 2 mounted on the mobile body for performing imaging, and a vibration damping means 3 for reducing image shake caused by changes in the position of the mobile body. When imaging, the control device 5 performs a first process of reducing the drive frequency of the drive unit compared to before imaging, and a second process of controlling the drive unit so that the mobile body moves toward the position before imaging after imaging if the amount of change in the position of the mobile body during imaging exceeds a first predetermined amount that can reduce image shake by the vibration damping means.

Description

Technical Field

[0001] The present invention relates to the control of a moving body capable of imaging.

Background Art

[0002] A moving body such as a drone may be equipped with a camera for aerial photography or the like. Patent Document 1 discloses a moving body provided with a vibration-proof (gimbal) mechanism for reducing image blur in imaging with a camera.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a drone, high-frequency vibration occurs when the propeller is driven. In contrast, since the vibration-proof device mainly reduces image blur caused by low-frequency vibration such as the swaying of the drone during flight, image blur due to high-frequency vibration remains in the captured image.

[0005] The present invention provides a control device capable of reducing image blur caused by high-frequency vibration of a moving body.

Means for Solving the Problems

[0006] One aspect of the present invention is a control device (control method) that controls an imaging mobile body which includes a mobile body having a drive unit, an imaging means mounted on the mobile body for performing imaging, and a vibration damping means for reducing image shake caused by changes in the position of the mobile body. The control device is characterized by performing, during imaging, a first process (first step) of reducing the drive frequency of the drive unit compared to before imaging, and a second process (second step) of controlling the drive unit so that the mobile body moves toward the position before imaging after imaging if the amount of change in the position of the mobile body during imaging exceeds a first predetermined amount by which image shake can be reduced by the vibration damping means.

[0007] Furthermore, another aspect of the present invention is a control device (control method) characterized in that, in a first process (first step) during imaging, the drive frequency of the drive unit is reduced from that before imaging, and the drive frequency is reduced so that the amount of position change of the moving body due to the reduction in the drive frequency does not exceed a first predetermined amount that can reduce image shake by vibration damping means. An imaging moving body equipped with the above control device also constitutes another aspect of the present invention. [Effects of the Invention]

[0008] According to the present invention, it is possible to reduce image shake caused by high-frequency vibrations of a moving object. [Brief explanation of the drawing]

[0009] [Figure 1] Schematic diagram and block diagram of the drone imaging system of Example 1. [Figure 2] A diagram showing the operation during continuous imaging in Example 1. [Figure 3] A diagram showing perspective correction in Example 1. [Figure 4] A flowchart showing the continuous imaging process in Example 1. [Figure 5] Block diagram of the drone imaging system in Example 2. [Modes for carrying out the invention]

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. [Examples]

[0011] Figure 1(a) shows a schematic of an imaging drone system including an imaging drone (imaging mobile body) in which a camera 2 as an imaging means and a gimbal system 3 as a vibration damping means are mounted on a drone 1, which is an unmanned aerial vehicle, and a drone control device 4. Figure 1(b) shows the electrical configuration of the imaging drone system. Note that the mobile body is not limited to a drone, but may be other mobile bodies such as a manned aircraft, automobile, or ship. Also, the vibration damping means is not limited to a gimbal system provided on the drone, but may be optical vibration damping means that move the lens or image sensor within the camera, or electronic vibration damping means that shift the cropping range within the captured image. Furthermore, the gimbal system and the vibration damping means within the camera may be used in combination. In addition, in this embodiment, a camera 2 in which the lens unit is integrally fixed to the camera body is used, but a camera in which the lens unit can be attached to and detached from the camera body may also be used.

[0012] In the drone 1 shown in Figures 1(a) and 1(b), 1a represents multiple propellers, 5 represents the flight control unit as a control device, 6 represents the propeller drive unit, 7 represents the positioning unit, and 8 represents the drone memory. In the camera 2, 9 represents the camera control unit, 10 represents the imaging optical system, 11 represents the image sensor, 12 represents the image processing unit, 13 represents the image synthesis unit within the image processing unit 12, and 14 represents the camera memory. In the gimbal system 3, 15 represents the gimbal control unit, 16 represents the shake sensor, and 17 represents the gimbal drive unit. In the drone operating device 4, 18 represents the operation control unit, 19 represents the drone operating unit, 20 represents the camera operating unit, and 21 represents the display unit.

[0013] The propeller drive unit 6 provided on the drone 1 includes a propeller 1a for flying the drone 1 in response to a flight control signal from the flight control unit 5, and a motor (not shown) for rotating the propeller 1a.

[0014] The positioning unit 7 acquires positional information indicating the three-dimensional coordinates of the drone 1's current position by communicating with a GNSS (Global Navigation Satellite System), such as GPS (Global Positioning System). By using the positional information acquired by the positioning unit 7, it is possible to fly the drone 1 to various target locations or to maintain a hovering state at a fixed position.

[0015] The drone memory 8 records position information acquired by the positioning unit 7, and also records information about the drone 1 in advance, such as the total weight of the drone 1 including the camera 2 and gimbal system 3, and the maximum correction angle of the gimbal system 3. The flight control unit 5 uses the information recorded in the drone memory 8 to generate flight control signals for the motor drive unit 6.

[0016] In camera 2, the imaging optical system 10 forms an image from the subject. The image sensor 11 is a photoelectric conversion element such as a CCD sensor or CMOS sensor, and performs photoelectric conversion (imaging) of the subject image formed by the imaging optical system 10. The image processing unit 12 has an A / D converter, a white balance adjustment circuit, a gamma correction circuit, and an interpolation calculation circuit, and generates an image from the imaging signal from the image sensor 11. The image processing unit 12 also performs compression processing of images and sound.

[0017] The image synthesis unit 13 performs image synthesis processing to generate a composite image when an imaging mode for image synthesis is selected by the camera operation unit 20 of the drone operation device 4. Image synthesis modes include, for example, an HDR (High Dynamic Range) imaging mode that synthesizes multiple images with different exposures to generate a composite image with a wide dynamic range. There is also a high-resolution imaging mode that synthesizes multiple images obtained by imaging while moving the image sensor 11 in a direction perpendicular to the optical axis in units smaller than the pixel pitch to generate a high-resolution composite image.

[0018] The camera memory 14 records compressed captured images, composite images, and audio (hereinafter collectively referred to as image information).

[0019] The camera control unit 9 controls imaging by the imaging device 11. Further, when imaging operations such as zooming, focusing, and aperture adjustment are performed by the camera operation unit 20 of the drone operation device 4, the camera control unit 9 controls the driving of a zoom lens, a focus lens, an aperture, etc. (not shown) of the imaging optical system 10. Furthermore, the camera control unit 9 transmits image information to the drone operation device 4 and causes the display unit 21 to display an image corresponding to the image information.

[0020] The gimbal unit 17 in the gimbal system 3 includes a gimbal mechanism that holds the camera 2 rotatably (movable) in the pitch (vertical) direction, yaw (horizontal) direction, and roll direction, and three motors that drive the gimbal mechanism to rotate the camera 2 in the above three directions. The shake sensor 16 is constituted by a vibration gyro or the like and detects rotational shake in the pitch direction, yaw direction, and roll direction among the shakes applied to the drone 1 (that is, the camera 2). The gimbal control unit 15 controls the three motors of the gimbal unit 17 so as to obtain a correction angle corresponding to the magnitude of the rotational shake detected by the shake sensor 16. Thereby, image shake in the pitch direction, yaw direction, and roll direction is corrected (reduced). The gimbal system 3 is optimized to correct image shake caused by low-frequency shake such as the sway accompanying the flight of the drone 1.

[0021] In the drone operation device 4, the drone operation unit 19 and the camera operation unit 20 are operated by a user (operator) to remotely control the drone 1 and the camera 2. When the operation control unit 18 detects a user's operation on the drone operation unit 19, it transmits a drone operation signal to the flight control unit 5 of the drone 1. The flight control unit 5 controls the flight of the drone 1 by controlling the motor drive unit 6 based on the received drone operation signal and the position information from the positioning unit 7.

[0022] Furthermore, when the operation control unit 18 detects a user operation on the camera operation unit 20, it transmits a camera operation signal to the camera control unit 9 of the camera 2. Based on the received camera operation signal, the camera control unit 9 controls the operation of the imaging optical system 10, the image sensor 11, and the image processing unit 12. As described above, the display unit 21 displays an image corresponding to the image information transmitted from the camera 2 so that the user can view it.

[0023] The flight control unit 5, camera control unit 9, and gimbal control unit 15 are interconnected and can communicate with each other, allowing for control of the drone 1's flight and gimbal's movement in sync with the camera 2's image capture timing. The camera control unit 9 can also record metadata, including image stabilization information such as the correction angle of the gimbal system 3, in the camera memory 14 along with the image information.

[0024] Next, using Figures 2(a) and 2(b), the control of the rotation speed of the propeller 1a of the drone 1 (the drive speed of the propeller drive unit 6) and the flight state during each imaging and imaging preparation period in continuous imaging to acquire multiple (n) images in this embodiment will be explained. Multiple images are acquired to generate the composite image described above. The rotation speed of the propeller 1a corresponds to the drive frequency of the propeller drive unit 6, and the vibrations generated in the drone 1 can be said to be vibrations with a frequency corresponding to the drive frequency of the propeller drive unit 6. The drive frequency of the propeller drive unit 6 may also correspond to the drive frequency of a motor not shown.

[0025] Figure 2(a) shows the range h in the direction of gravity (vibration-damping range: hereinafter referred to as the correctable range) over which image shake can be corrected by the gimbal system 3. As will be described later, in this embodiment, the propeller rotation speed of the drone 1 is reduced from the time before imaging (imaging preparation period) for each image taken during continuous imaging. This reduction in propeller rotation speed causes a change in position (descent) of the drone 1. If the amount of change in position of the drone 1 is within the correctable range h, the gimbal system 3 can correct the image shake.

[0026] Figure 2(b) shows the imaging, propeller speed, and flight status from the 1st to the nth image when centering is performed on drone 1 during continuous imaging. Centering is the control of the propeller speed to move drone 1, which has changed position from the starting position of continuous imaging, toward the starting position (in the direction of approaching the starting position). Specifically, this includes returning to or approaching the starting position within the correctable range, and moving from outside the correctable range to within the correctable range. Figure 2(c) shows the imaging, propeller speed, and flight status from the 1st to the nth image when centering is not performed on drone 1.

[0027] In this embodiment, it is determined whether or not to center the drone 1 during continuous imaging. When the propeller (motor) 1a of the drone 1 is driven at a high rotational speed, the resulting high-frequency vibrations cause high-frequency image shake in the multiple images obtained during continuous imaging to generate a composite image, which cannot be corrected by the gimbal system 3. Therefore, by reducing the propeller rotational speed to a predetermined rotational speed in the low-frequency range where image shake correction by the gimbal system 3 is possible, only during each imaging (exposure) in continuous imaging, the occurrence of high-frequency image shake can be suppressed.

[0028] However, by setting the propeller rotation speed to below a predetermined speed, the drone 1 will experience a change in position (descent) due to gravity. In this embodiment, it is determined (selected) whether or not to perform centering based on the amount of this change in position.

[0029] In Figure 2(a), 201 represents the subject. When performing continuous imaging, the total imaging time required from the start of the first imaging to the end of the nth imaging can be calculated from the set shutter speed and number of imaging cycles for a single imaging cycle. Using the calculated total imaging time, the amount of change in position (descent) Δx of the drone 1 during the total imaging time (i.e., during continuous imaging) can be determined when the propeller rotation speed is reduced to a predetermined rotation speed or less with each imaging cycle of continuous imaging.

[0030] Furthermore, as shown in Figure 2(a), when the gimbal unit 17 is driven to its maximum correction angle, the distance from the camera 2 to the subject at the center of its imaging field of view is f, and the maximum correction angle at which the gimbal unit 17 can correct image shake is θ. max In this case, the pitch-direction correction range h in which the gimbal unit 17 can correct image vibration is calculated by the following equation (1).

[0031] h = 2 × f × sin(θ) max / twenty one) Note that the method for calculating the correctable range h is not limited to this; it may also be calculated using the distance from camera 2 to the subject at the center of the imaging angle when camera 2 and the subject are facing each other directly. If Δx > h when comparing the position change amount Δx with the correctable range h, image shake that cannot be corrected by the gimbal system 3 will remain in the captured image, so it is necessary to center the drone 1 during the imaging preparation period between images in continuous imaging.

[0032] In contrast, when Δx ≤ h, continuous imaging is completed within the range h in which image shake correction by the gimbal system 3 is possible, so there is no need to perform centering. Performing centering increases the total imaging time, and there is a risk that the accuracy of position information will decrease due to sensor drift of the positioning unit 7, etc. For this reason, it is desirable to set the number of centering operations to the minimum necessary. In this embodiment, therefore, it is determined whether or not to perform centering of the drone 1 based on the result of comparing the position change amount Δx with a predetermined amount (first predetermined amount H1 described later) set according to the correctable range h.

[0033] In Figure 2(b), during the imaging preparation period before continuous imaging begins, drone 1 is in a hovering state and its propeller rotation speed is constant. When the first imaging begins, the propeller rotation speed is reduced to reduce high-frequency vibrations applied to camera 2 (first process). As a result, drone 1 descends. When the first imaging is completed and the preparation period for the second imaging begins, the propeller rotation speed is increased to center drone 1 (second process). Once centering is complete, the second imaging begins, and the propeller rotation speed is reduced again.

[0034] The above control is repeated until the nth image is taken. After the nth image is taken, drone 1 is put into a hovering state.

[0035] Centering does not have to be performed after every single image; it may be performed after at least one image in a series of images. For example, if centering is performed after every m images, the centering should be performed during the preparation period between the (a × m)th image (a = 1, 2, 3, ...) and the (a × m + 1)th image.

[0036] In Figure 2(c), during the imaging preparation period before continuous imaging begins, Drone 1 is in a hovering state, similar to Figure 2(b). When the first imaging begins, the propeller rotation speed decreases, causing Drone 1 to descend. Since centering is not performed, Drone 1 continues to descend during the second imaging preparation period, and the second imaging is performed. Drone 1 continues to descend until the nth imaging is completed, after which it returns to a hovering state. Note that Drone 1 may be kept in a hovering state during the imaging preparation period when centering is not performed.

[0037] Thus, in this embodiment, by determining whether to perform centering based on the amount of position change Δx of the drone 1 during the total imaging time of continuous imaging, it is possible to perform continuous imaging for image synthesis while suppressing the decrease in accuracy of the positioning unit 7 due to the length of the total imaging time.

[0038] Next, the image synthesis process in this embodiment will be explained using Figures 3(a) and 3(b). Figure 3(a) shows the position change of the drone 1 (camera 2) in the direction of gravity and the operation of the gimbal system 3 during continuous imaging. Figure 3(b) shows the images obtained by imaging at two different positions of the drone 1. 301 indicates the subject, and 302 indicates the first position of the drone 1 where the camera 2 is directly facing the subject 301 in the direction of gravity. 303 indicates the second position to which the drone 1 has descended from the first position 302. The second position 303 is a position within the range where image shake correction by the gimbal system 3 is possible and indicates the position where the nth image is taken. 304 shows the image obtained by imaging at the first position 302, and 305 shows the image obtained by imaging at the second position 303. 306 indicates the center of each image.

[0039] At the second position 303, the camera 2's orientation is changed diagonally upward by the gimbal system 3 in order to obtain an image 305 with the same center 306 as the image 304 obtained at the first position 302. Perspective distortion occurs in the image 305 obtained at the second position 303, depending on the orientation of the camera 2 (i.e., the relative position between the subject 301 and the drone 1). Therefore, when combining these images 305 and 306, it is necessary to correct the perspective distortion of the image 306 (hereinafter referred to as perspective correction).

[0040] The flowchart in Figure 4 shows the processes (control methods) that the flight control unit 5 and camera control unit 9, each composed of a computer, execute according to the program in this implementation. Here, as mentioned above, the process shown is for when continuous imaging for image synthesis by camera 2 starts from a hovering state at the first position (starting position) of drone 1.

[0041] In step S4001, the camera control unit 9 determines the shutter speed for each image during continuous imaging based on the user's input of the shutter speed.

[0042] Next, in step S4002, the camera control unit 9 determines the number of images to be composited (number of composite images), that is, the number of images to be captured in continuous imaging, based on the user's input of the number of images to be composited.

[0043] Next, in step S4003, the flight control unit 5 receives information on shutter speed and the number of composite images from the camera control unit 9, and calculates (acquires) the amount of position change Δx of the drone 1 during the total imaging time obtained from these. Specifically, the amount of position change Δx during the total imaging time is calculated based on the total weight of the drone 1 recorded in the drone memory 8 mentioned above and the amount of decrease in propeller rotation speed from the hovering state to below a predetermined rotation speed in the low frequency range mentioned above.

[0044] The amount of decrease in propeller rotation speed at this time can be set according to the shutter speed of camera 2. For example, when the shutter speed is short, even if the amount of position change is large, the amount of image shake included in the captured image tends to be small, so it is preferable to set a large amount of decrease in propeller rotation speed. On the other hand, when the shutter speed is long, if the amount of position change is large, the amount of image shake included in the captured image will also be large, so it is preferable to set the amount of decrease in propeller rotation speed to be smaller than when the shutter speed is short. In this case, the amount of decrease in propeller rotation speed may be set to be less than or equal to a threshold set according to the shutter speed as its maximum allowable value. Alternatively, instead of the amount of decrease in propeller rotation speed, the amount of position change Δx may be calculated using the reduced propeller rotation speed (target rotation speed).

[0045] Next, in step S4004, the flight control unit 5 compares the amount of position change Δx of the drone 1 during the total imaging time with a first predetermined amount H1 set according to the correction range h provided by the gimbal system 3. The first predetermined amount H1 may be the same as the amount of position change corresponding to the correction range h, or it may be set to be slightly smaller (for example, 90%). If the amount of position change Δx ≤ the first predetermined amount H1, the process in step S4005 is performed, and if the amount of position change Δx > the first predetermined amount H1, the process in step S4010 is performed.

[0046] In step S4005, the camera control unit 9 determines whether or not the user has given an instruction to start continuous imaging. If an instruction to start is given, the process in step S4006 is performed; otherwise, the determination in this step is repeated.

[0047] In step S4006 (first step), the flight control unit 5 reduces the propeller rotation speed in the propeller drive unit 6 by the amount of reduction described above in step S4003.

[0048] Next, in step S4007, the camera control unit 9 performs one image capture in continuous imaging.

[0049] Next, in step S4008, the camera control unit 9 determines whether continuous imaging has finished. If continuous imaging has finished, this process is terminated. If continuous imaging has not finished, the process in step S4009 is performed.

[0050] In step S4009, the flight control unit 5 compares the actual position change amount Δxr of the drone 1 during continuous imaging, acquired by at least one of the positioning unit 7 and the vibration sensor 16, with a second predetermined amount H2 set according to the correctable range h. This comparison is performed taking into account the actual position change of the drone 1 due to external factors such as strong winds. The second predetermined amount H2 may be the same as the first predetermined amount H1 mentioned above, or it may be set to a different value from the first predetermined amount H1 (for example, 90% of the first predetermined amount H1). If the actual position change amount Δxr is greater than or equal to the second predetermined amount H2, this process is terminated. Otherwise, the process returns to step S4006, and the propeller rotation speed that was reduced in the previous step S4006 is maintained (third process).

[0051] Meanwhile, in step S4010, the camera control unit 9 determines whether or not the user has given an instruction to start continuous imaging. If an instruction to start is given, the process in step S4011 is performed; otherwise, the determination in this step is repeated.

[0052] In step S4011, the flight control unit 5 records the position information (three-dimensional coordinates) of the drone 1 acquired by the positioning unit 7 into the drone memory 8.

[0053] Next, in step S4012 (first step), the flight control unit 5 reduces the propeller rotation speed in the propeller drive unit 6 by the amount of reduction described above in step S4003.

[0054] Next, in step S4013, the camera control unit 9 performs one image capture in continuous imaging.

[0055] Next, in step S4014, the camera control unit 9 determines whether continuous imaging has finished. If continuous imaging has finished, this process is terminated. If continuous imaging has not finished, the process in step S4015 is performed.

[0056] In step S4015, the flight control unit 5 compares the actual position change amount Δxr of the drone 1 during continuous imaging, acquired by at least one of the positioning unit 7 and the vibration sensor 16, with a second predetermined amount H2, similar to step S4009. If the actual position change amount Δxr during continuous imaging becomes greater than or equal to the second predetermined amount H2 due to external factors or the like, the continuous imaging is stopped and this process is terminated; otherwise, the process in step S4016 is performed.

[0057] In step S4016, the flight control unit 5 determines whether or not to perform centering. That is, if centering is to be performed after every predetermined number of imaging sessions (1 or m times), it determines whether or not to perform centering after the current imaging session. If centering is to be performed, the process in step S4017 is carried out; otherwise, the process returns to step S4012. In this case, step S4012 maintains the propeller rotation speed that was reduced in the previous step S4012.

[0058] In step S4017 (second step), the flight control unit 5 increases the propeller rotation speed in the propeller drive unit 6 to center the drone 1 so that it moves (ascends) toward the first position indicated by the position information recorded in step S4011. Then, it returns to the process of step S4012 and decreases the propeller rotation speed for the next image capture.

[0059] As explained above, in this embodiment, centering is performed only when the amount of change in the position Δx of the drone 1 due to the decrease in propeller rotation speed during continuous imaging exceeds the correctable range h. Therefore, by performing centering, it is possible to perform continuous imaging with reduced image shake while suppressing the effects of sensor drift in the positioning unit 7 caused by the increased total imaging time required for continuous imaging. [Examples]

[0060] Figure 5 shows the electrical configuration of the drone imaging system in Embodiment 2. The drone 1, camera 2, and gimbal system 3 in this embodiment are the same as in Embodiment 1. In this embodiment, the drone control device 4' as an external device includes an image processing unit 501 including an image synthesis unit 502 and a control device memory 503.

[0061] The operation control unit 18 of the drone operation device 4' receives image information (multiple captured images generated by continuous imaging) recorded in the camera memory 14 of the camera 2 from the camera control unit 9 and inputs it to the image processing unit 501. The image synthesis unit 502 within the image processing unit 501 performs image synthesis processing to combine the multiple captured images and generate a composite image.

[0062] The operation control unit 18 records the composite image in the operation device memory 503 and displays it on the display unit 21.

[0063] The image synthesis unit may be located in an external device such as a personal computer, separate from camera 2.

[0064] As in this embodiment, by performing image synthesis processing outside of camera 2, there is no need to perform image synthesis processing within camera 2. As a result, the power consumption of the drone imaging system, including camera 2, can be reduced, enabling longer aerial photography sessions.

[0065] In the above embodiments, we described a case where a flight control unit (control device) mounted on the drone controls the flight state during continuous imaging (such as reducing propeller rotation speed or centering). In contrast, a control device acting as a personal computer not mounted on the drone, or a control device mounted within a drone operating device, may perform the above control via communication with the drone.

[0066] In addition, the above embodiments described the control of the propeller rotation speed during continuous imaging to obtain multiple composite images. However, similar control of the propeller rotation speed may be performed during video imaging.

[0067] Furthermore, in each of the above embodiments, the case was described in which the propeller rotation speed is reduced to a rotation speed in the low frequency range where image shake correction by the gimbal system is possible during continuous imaging, and centering is performed when the amount of change in the drone's position exceeds a first predetermined amount corresponding to the correctable range. In contrast, even if the propeller rotation speed is reduced to such a low frequency range as described above, if reducing it even slightly can reduce the image shake caused by the propeller rotation, it is sufficient to reduce the propeller rotation speed so that the amount of change in the drone's position does not exceed the first predetermined amount. Also, even if hovering is possible at a propeller rotation speed that does not generate high-frequency vibrations that are difficult to correct for image shake, the propeller rotation speed may be reduced during continuous imaging. In that case, in addition to suppressing image shake caused by high-frequency vibrations of the moving object, power consumption for propeller driving can be reduced.

[0068] Furthermore, while the above embodiments described drones using a propeller-driven system, drones using a system that flaps wings, such as an ornithopter, may also be used. In such a system, the frequency at which the wings flap can be controlled by changing the drive frequency of the drive unit. The driving system of the drone is not limited to the above; any driving system that generates vibration in the drone is acceptable. For example, while a gimbal system corrects image shake in the low-frequency range, such control may also be performed if the camera has built-in optical or electronic image stabilization means that can correct image shake in a frequency range higher than the low-frequency range. Note that reducing the propeller rotation speed so as not to exceed a first predetermined amount is also a process performed in steps S4004 to S4005~S4007 as described in Example 1 (Figure 4).

[0069] The above embodiments include the following configuration.

[0070] (Composition 1) A control device for controlling an imaging mobile body, which includes a mobile body having a drive unit, imaging means mounted on the mobile body for performing imaging, and vibration damping means for reducing image shake caused by changes in the position of the mobile body, During the imaging process, a first process is performed to reduce the drive frequency of the drive unit compared to before the imaging; A control device characterized by performing a second process in which, if the amount of change in the position of the moving body during imaging exceeds a first predetermined amount that can reduce image shake by the vibration damping means, the drive unit controls the moving body to move toward the position before imaging after imaging. (Configuration 2) The control device according to configuration 1, characterized in that the second process is performed when the amount of position change acquired before imaging exceeds a first predetermined amount. (Composition 3) The control device according to configuration 2, characterized in that, before imaging, the amount of position change is acquired based on the time required for imaging, the weight of the imaging moving body, and the amount of decrease in the driving frequency or the driving frequency after the decrease. (Composition 4) A control device according to any one of configurations 1 to 3, characterized in that the second process is performed when the amount of position change from the start to the end of continuous imaging, in which the imaging is performed multiple times, exceeds a first predetermined amount. (Composition 5) The control device according to configuration 4, characterized in that the first process is performed for each imaging in the multiple imaging, and the second process is performed after at least one imaging in the multiple imaging. (Composition 6) A control device according to any one of configurations 1 to 5, characterized in that the amount of reduction of the drive frequency in the second process is set according to the shutter speed in the imaging. (Composition 7) A control device according to any one of configurations 1 to 6, characterized in that, when the amount of position change is smaller than the first predetermined amount, a third process is performed to maintain the drive frequency reduced by the first process even after imaging. (Composition 8) A control device according to any one of configurations 1 to 7, characterized in that if the actual amount of positional change of the moving body during the imaging exceeds a second predetermined amount that can reduce image shake by the vibration damping means, subsequent imaging is stopped. (Composition 9) A control device for controlling an imaging mobile body, which includes a mobile body having a drive unit, imaging means mounted on the mobile body for performing imaging, and vibration damping means for reducing image shake caused by changes in the position of the mobile body, During the imaging process, a first process is performed to reduce the drive frequency of the drive unit compared to before the imaging. A control device characterized in that, in the first process, the drive frequency is reduced so that the amount of change in the position of the moving body due to the decrease in the drive frequency does not exceed a first predetermined amount by which the vibration damping means can reduce image shake. (Composition 10) The aforementioned moving object is an aircraft that flies by rotating its propellers. The control device according to any one of configurations 1 to 9, characterized in that the drive frequency is the rotational speed of the propeller. (Composition 11) The control device according to any one of configurations 1 to 10, characterized in that the vibration isolation means is at least one of a vibration isolation means provided on the moving body for movably holding the imaging device and a vibration isolation means provided within the imaging device. (Composition 12) A control device as described in any one of configurations 1 to 11, The aforementioned moving body, The imaging means, An imaging moving body characterized by having the aforementioned vibration damping means.

[0071] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0072] The embodiments described above are merely representative examples, and various modifications and changes can be made to each embodiment when implementing the present invention. [Explanation of symbols]

[0073] 1. Drone (mobile device) 2 cameras 3. Gimbal System 5. Flight Control Unit (Control Device)

Claims

1. A control device for controlling an imaging mobile body, which includes a mobile body having a drive unit, imaging means mounted on the mobile body for performing imaging, and vibration damping means for reducing image shake caused by changes in the position of the mobile body, During the imaging process, a first process is performed to reduce the drive frequency of the drive unit compared to before the imaging. A control device characterized by performing a second process in which, if the amount of change in the position of the moving body during imaging exceeds a first predetermined amount that can reduce image shake by the vibration damping means, the drive unit controls the moving body to move toward the position before imaging after imaging.

2. The control device according to claim 1, characterized in that the second process is performed when the amount of position change acquired before imaging exceeds a first predetermined amount.

3. The control device according to claim 2, characterized in that, before imaging, the amount of position change is acquired based on the time required for imaging, the weight of the imaging moving body, and the amount of decrease in the driving frequency or the driving frequency after the decrease.

4. The control device according to claim 1, characterized in that the second process is performed when the amount of position change from the start to the end of continuous imaging, in which the imaging is performed multiple times, exceeds a first predetermined amount.

5. The control device according to claim 4, characterized in that the first process is performed for each imaging in the multiple imaging, and the second process is performed after at least one imaging in the multiple imaging.

6. The control device according to claim 1, characterized in that the amount of reduction of the drive frequency in the second process is set according to the shutter speed in the imaging.

7. The control device according to claim 1, characterized in that, when the amount of position change is smaller than a first predetermined amount, a third process is performed to maintain the drive frequency reduced by the first process even after imaging.

8. The control device according to claim 1, characterized in that if the actual amount of change in position of the moving body during the imaging exceeds a second predetermined amount that can reduce image shake by the vibration damping means, subsequent imaging is stopped.

9. A control device for controlling an imaging mobile body, which includes a mobile body having a drive unit, imaging means mounted on the mobile body for performing imaging, and vibration damping means for reducing image shake caused by changes in the position of the mobile body, During the imaging process, a first process is performed to reduce the drive frequency of the drive unit compared to before the imaging. A control device characterized in that, in the first process, the drive frequency is reduced so that the amount of change in the position of the moving body due to the decrease in the drive frequency does not exceed a first predetermined amount by which the vibration damping means can reduce image shake.

10. The aforementioned moving object is an aircraft that flies by rotating its propellers. The control device according to claim 1 or 9, characterized in that the drive frequency is the rotational speed of the propeller.

11. The control device according to claim 1 or 9, characterized in that the vibration isolation means is at least one of the vibration isolation means provided on the moving body for movably holding the imaging device and vibration isolation means provided within the imaging device.

12. A control device according to claim 1 or 9, The aforementioned moving body, The imaging means, An imaging moving body characterized by having the aforementioned vibration damping means.

13. A control method for controlling an imaging mobile body, which includes a mobile body having a drive unit, imaging means mounted on the mobile body for performing imaging, and vibration damping means for reducing image shake caused by changes in the position of the mobile body, During the imaging process, a first process is performed to reduce the drive frequency of the drive unit compared to before the imaging. A control method characterized by comprising: a second step of controlling the drive unit so that, when the amount of change in the position of the moving body during imaging exceeds a first predetermined amount that can reduce image shake by the vibration damping means, the moving body moves toward the position before imaging after imaging.

14. A control method for controlling an imaging mobile body, which includes a mobile body having a drive unit, imaging means mounted on the mobile body for performing imaging, and vibration damping means for reducing image shake caused by changes in the position of the mobile body, The imaging process includes a first process that reduces the drive frequency of the drive unit compared to before the imaging. A control method characterized in that, in the first step, the drive frequency is reduced so that the amount of change in the position of the moving body due to the decrease in the drive frequency does not exceed a first predetermined amount by which the vibration damping means can reduce image shake.

15. A program characterized by causing a computer to perform processing according to the control method described in claim 13 or 14.

Images