Method for correcting angle error of gimbal mounted on drone

Abstract

The present invention relates to a method for correcting an angle error generated in a gimbal, consisting of: calculating an angle error value of the gimbal on the basis of an adjustment value checked when rotation of the gimbal is controlled while identifying and tracking a specific target in a vertically downward image captured by a camera mounted on a drone; and using the calculated angle error value for correction.

Description

How to compensate for the angle error of a gimbal mounted on a drone

[0001] The present invention relates to a method for correcting an angle error of a gimbal mounted on a drone, and more specifically, to a method configured to calculate and use an angle error value of a gimbal for correction based on an adjustment value confirmed when controlling rotation of a gimbal while confirming and tracking a specific target in a vertically downward image captured by a camera mounted on a drone, so that an angle error occurring in a gimbal can be corrected.

[0002]

[0003] Generally, a gimbal on which a camera is mounted is a filming device used to correct shaking in the captured video.

[0004] In particular, gimbals equipped with cameras can be easily seen on drones that experience a lot of shaking due to flight, and by using these gimbals to adjust the shooting angle of the camera, the additional effect of being able to secure video information from a wider range of angles can be obtained.

[0005] In this way, control of the gimbal mounted on the drone is for securing more precise image information, and in relation to this, inventions such as “Gimbal control method using screen touch method” in Korean Patent Publication No. 10-2122755 and “Method for obtaining image information with improved precision” in Korean Patent Publication No. 10-2402949 have been proposed and published.

[0006] That is, the “gimbal control method using a screen touch method” of the above-mentioned Korean Patent Publication No. 10-2122755 proposes an invention regarding a method of detecting a location on the screen where a touch occurs when an image is displayed, calculating pixel data and angle data accordingly, and generating an angle control value according to the calculated pixel data and angle data so that the gimbal can be controlled to face a location selected by the user.

[0007] In addition, the “Method for Obtaining Image Information with Improved Precision” of the above-mentioned Korean Patent Publication No. 10-2402949 proposes an invention regarding a method for calculating coordinate information of a shooting target point using location information of a drone, attitude information of a camera generating image information, and altitude information of the ground acquired in advance, and tagging the calculated coordinate information to the image information, thereby enabling precise coordinate information to always be obtained regardless of the topography of the shooting target point even in an environment where a 3D mapping program cannot be used.

[0008] At this time, since the above-mentioned prior inventions are aimed at obtaining precise coordinate information based on the camera's attitude information and precise gimbal control to enable the user-selected location to be photographed, it can be said that a countermeasure against gimbal angular error is absolutely necessary.

[0009] That is, the small gimbal mounted on the drone uses an ultra-small IMU sensor for attitude control, but due to the characteristics of this ultra-small IMU sensor, sensor errors occur depending on the usage environment.

[0010] For example, if a sensor error occurs due to drift caused by temperature changes and accumulated rotation, and the angle of the aligned gimbal becomes misaligned, the acquisition of precise image information for a specific target or the calculation of the target's location through image analysis will inevitably become inaccurate.

[0011] Therefore, it is necessary to check the sensor error that occurred in the ultra-small IMU sensor and align the gimbal angle accordingly, but the above-mentioned prior inventions have a problem in that they do not propose such a correction means, and in reality, many difficulties arise in aligning the gimbal angle.

[0012] For the above reasons, there is a need for an invention regarding a means for checking a sensor error occurring in an ultra-small IMU sensor and aligning a gimbal angle according to the error value, thereby enabling the acquisition of precise image information targeting a specific target or the accurate calculation of the target's location through image analysis.

[0013]

[0014] The small gimbal mounted on the drone uses an ultra-small IMU sensor for attitude control. However, due to the characteristics of this ultra-small IMU sensor, sensor errors may occur depending on the usage environment.

[0015] As a result, an angle error of the gimbal may occur, which may result in inaccurate acquisition of precise image information targeting a specific target or inaccurate calculation of the target's location through image analysis. Therefore, the present invention aims to provide a solution to this problem.

[0016]

[0017] The present invention aims to achieve the above-mentioned purpose,

[0018] A method for correcting an angle error of a gimbal mounted on a drone is proposed, comprising: a video shooting step for adjusting the angle of a gimbal while the drone is in stationary flight so that a camera mounted on the gimbal takes a vertically downward video; a target confirmation step for confirming a specific target located at the center point of the video being shot; a first adjustment value confirmation step for confirming a first adjustment value by automatic adjustment of the gimbal that occurs when the gimbal is rotated around the yaw axis while tracking the identified target in the video; a second adjustment value confirmation step for confirming a second adjustment value by automatic adjustment of the gimbal that occurs when the gimbal is rotated in the opposite direction to before around the yaw axis while tracking the target in the video; an error calculation step for calculating an angle error value of the gimbal using the confirmed first and second adjustment values; and a correction step for correcting an angle error of the gimbal based on the calculated angle error value.

[0019] At this time, in the first adjustment value confirmation step, the first roll axis adjustment value (dRoll1) and the first pitch axis adjustment value (dPitch1) of the gimbal, which are the first adjustment values ​​generated in the process of rotating the gimbal around the yaw axis, are confirmed.

[0020] In the above second adjustment value confirmation step, the second roll axis adjustment value (dRoll2) and the second pitch axis adjustment value (dPitch2) of the gimbal, which are the second adjustment values ​​generated in the process of rotating the gimbal in the opposite direction to the previous one with the rotation amount doubled based on the yaw axis, are confirmed.

[0021] In the above error calculation step, the first and second adjustment values, , It is characterized in that it is configured to calculate the angular error value (dRoll, dPitch) of the gimbal by applying the formula.

[0022]

[0023] A method for correcting an angle error of a gimbal mounted on a drone according to the present invention is as follows:

[0024] Because it is configured to accurately calculate and use the gimbal's angle error value for correction based on the primary and secondary adjustment values ​​confirmed when controlling the gimbal rotation while confirming and tracking a specific target for the vertically downward image taken by the camera mounted on the drone,

[0025] This has the effect of allowing the gimbal's angular error to be precisely corrected,

[0026] This enables the acquisition of precise image information targeting a specific target, and additionally has the effect of eliminating or minimizing errors in calculating the target's location through image analysis.

[0027]

[0028] Figure 1 is a basic configuration diagram of a method for correcting an angle error of a gimbal mounted on a drone according to the present invention.

[0029] Figure 2 is an example comparing the shooting direction vertically downward when there is no angular error in the gimbal and the shooting direction vertically downward when there is an angular error in the gimbal.

[0030] Figures 3a and 3b are screen configuration diagrams showing the image center point confirmed when the gimbal is rotated under rotation control without causing an angular error.

[0031] Figure 4 is a screen configuration diagram showing the image center point confirmed when the gimbal, which has an angular error, is rotated.

[0032] Figure 5 is an additional configuration diagram of a method for correcting an angle error of a gimbal mounted on a drone according to the present invention.

[0033]

[0034] A method for correcting an angle error of a gimbal mounted on a drone according to the present invention is as follows:

[0035] The present invention is characterized by comprising: a video shooting step for adjusting the angle of a gimbal while the drone is in stationary flight so that a camera mounted on the gimbal takes a vertically downward video; a target confirmation step for confirming a specific target located at the center point of the video being shot, which is the center of the video; a first adjustment value confirmation step for confirming a first adjustment value by automatic adjustment of the gimbal that occurs when the gimbal is rotated around the yaw axis while tracking the identified target in the video; a second adjustment value confirmation step for confirming a second adjustment value by automatic adjustment of the gimbal that occurs when the gimbal is rotated in the opposite direction to before around the yaw axis while tracking the target in the video; an error calculation step for calculating an angle error value of the gimbal using the confirmed first and second adjustment values; and a correction step for correcting an angle error of the gimbal based on the calculated angle error value.

[0036]

[0037] The present invention relates to a method for correcting an angle error of a gimbal mounted on a drone,

[0038] The present invention is characterized by comprising: a video shooting step (S100) of adjusting the angle of a gimbal while the drone is in stationary flight so that a camera mounted on the gimbal takes a vertically downward video; a target confirmation step (S110) of confirming a specific target located at the center point of the video being shot, which is the center of the video; a first adjustment value confirmation step (S120) of confirming a first adjustment value by automatic adjustment of the gimbal that occurs when the gimbal is rotated around the yaw axis while tracking the identified target; a second adjustment value confirmation step (S130) of confirming a second adjustment value by automatic adjustment of the gimbal that occurs when the gimbal is rotated in the opposite direction to before around the yaw axis while tracking the target; an error calculation step (S140) of calculating an angle error value of the gimbal using the confirmed first and second adjustment values; and a correction step (S150) of correcting an angle error of the gimbal based on the calculated angle error value.

[0039]

[0040] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0041] First, as illustrated in FIG. 1, the present invention is characterized by including an image capturing step (S100) of adjusting the angle of a gimbal while the drone is in stationary flight so that a camera mounted on the gimbal captures an image in a vertical downward direction.

[0042] That is, the implementation of the present invention is based on the premise that a rotary wing drone equipped with a camera and equipped with a gimbal is flying stationary in the sky.

[0043] Therefore, a control signal for hovering flight must be transmitted to a drone in flight, and when the flight control unit equipped with the drone receives a control signal for hovering flight through the communication unit equipped with the drone, the drone must immediately control the drone to hover in a horizontal state.

[0044] That is, when the communication unit equipped in the drone receives a control signal for hovering flight, the drone currently in motion can be controlled to hover in a horizontal state, and the drone already in hovering flight can be controlled to maintain its current state or to become horizontal to the ground from an inclined state.

[0045] Next, a control signal for attitude control must be transmitted to the gimbal mounted on the drone, and the gimbal control unit equipped on the gimbal must immediately control the attitude of the gimbal so that the camera can shoot vertically downward when receiving the control signal for attitude control.

[0046] And a control signal for shooting must also be transmitted to the camera mounted on the gimbal, and the shooting control unit equipped on the camera must control the camera to maintain a state of shooting real-time video or to shoot real-time video when receiving the control signal for shooting.

[0047] At this time, the control signal targeting the gimbal and the control signal targeting the camera may be received by the drone's communication unit or by the gimbal and the camera, respectively, depending on the configuration of the drone including the gimbal and the camera.

[0048] As a result, in the above image capturing step (S100), a vertically downward image based on a stationary flying drone or camera can be captured, and real-time image information can be generated, and the generated image information can be immediately transmitted to a ground image analysis device and used for image analysis.

[0049] However, when a sensor error occurs in the ultra-small IMU sensor used for gimbal attitude control, an angle error occurs in the gimbal, and as shown in Fig. 2, a problem occurs in which the real-time image being captured is not an accurate vertically downward image. Therefore, the gimbal angle error must be corrected according to each step described below.

[0050]

[0051] That is, as illustrated in FIG. 1, the present invention is characterized by including a target identification step (S110) of identifying an image center point that is the center of an image being captured and identifying a specific target located at the image center point.

[0052] In this way, in the target confirmation step (S110), image analysis targeting image information generated and transmitted in real time is performed in a ground-based image analysis device, thereby confirming the image center point that is the center of the real-time image and confirming a specific target located at the image center point.

[0053] At this time, the image center point can be identified using the resolution information of a camera or a display device that displays real-time images, and the target can be identified by specifying a shape that corresponds to an actual terrain feature or object corresponding to the image center point on the screen, and from then on, image tracking occurs in real time by continuously tracking the specified shape.

[0054] That is, the image analysis device can select an area corresponding to a specific ratio of pixel numbers as an image center point based on resolution information, extract features of terrain features or objects, etc., which are displayed in whole or in part in that area, and perform real-time image tracking based on the extracted feature information.

[0055] However, in a situation where there is no specific target corresponding to the center point of the image or it is difficult to extract features, the image analysis device can extract features of a color that are distinct from the surrounding areas in the relevant area, and can perform real-time image tracking based on the extracted feature information.

[0056] In addition, the image analysis device can divide a real-time image into at least four sectors in a situation where there is no specific target corresponding to the image center point or it is difficult to extract features, extract feature points from each divided sector, and calculate the distance on the screen for each feature point based on the image center point.

[0057] Accordingly, real-time image tracking targeting the image center point that can be identified using a number of feature points can be performed, thereby solving the problem of situations in which there is no specific target in the image or feature extraction is difficult, as described above.

[0058]

[0059] In addition, as illustrated in FIG. 1, the present invention is characterized by including a first adjustment value confirmation step (S120) for confirming a first adjustment value by automatic adjustment of the gimbal that occurs when the gimbal is rotated around the yaw axis while tracking a specific target confirmed in the target confirmation step (S110).

[0060] That is, as shown in FIGS. 3a and 3b, in a normal state where no angular error occurs due to the sensor error of the ultra-small IMU sensor in the gimbal, even if the gimbal rotates in the yaw axis direction, the position of the target in the image remains the same as before, so there is no need to adjust the angle of the gimbal for the purpose of image tracking of the target.

[0061] However, when the image tracking function is turned off and the gimbal is rotated in the yaw axis direction in an error state where an angular error occurs due to a sensor error in the gimbal, a phenomenon occurs in which the position of the target located at the center of the image deviates from the center of the image, as shown in Fig. 4.

[0062] For this reason, in order to track a specific target located at the center of the image, the angle of the gimbal must be adjusted. (At this time, Fig. 3 is a drawing for comparison with Fig. 2b, so it shows the center of the image being tilted to one side of the image, but the center of the image on the actual screen is always located at the center of the image.)

[0063] Therefore, it is possible to check the first adjustment value by automatic adjustment of the gimbal that occurs when the gimbal is controlled to rotate at a certain angle in the yaw axis direction while tracking the target by video, and the checked first adjustment value can be used for the purpose of correcting the angle error of the gimbal.

[0064] In addition, as illustrated in FIG. 1, the present invention is characterized by including a second adjustment value confirmation step (S130) for confirming a second adjustment value by automatic adjustment of the gimbal that occurs when the gimbal is controlled to rotate in the opposite direction to the previous direction based on the yaw axis while tracking a target by image.

[0065] That is, in the second adjustment value confirmation step (S130), the gimbal rotates in the opposite direction and at twice the amount of rotation compared to the rotation of the gimbal that occurred in the first adjustment value confirmation step (S120).

[0066] For example, if the gimbal rotated 45° counterclockwise in the first adjustment value confirmation step (S120), the gimbal rotates 90° clockwise in the second adjustment value confirmation step (S130).

[0067] Accordingly, based on the initial alignment state of the gimbal in the above video shooting step (S100), the same type of rotation occurs in both directions, and the effect of being able to accurately calculate the gimbal's angle error value using the first and second adjustment values ​​confirmed in the process is generated.

[0068] At this time, the first adjustment value is the first roll axis adjustment value (dRoll1) and the first pitch axis adjustment value (dPitch1) based on the initial alignment state of the gimbal and the yaw axis of the gimbal in the video shooting step (S100), and the second adjustment value is the second roll axis adjustment value (dRoll2) and the second pitch axis adjustment value (dPitch2) based on the initial alignment state of the gimbal and the yaw axis of the gimbal.

[0069]

[0070] In addition, as illustrated in FIG. 1, the present invention is characterized in that it comprises an error calculation step (S140) for calculating an angle error value of a gimbal using a first adjustment value confirmed in a first adjustment value confirmation step (S120) and a second adjustment value confirmed in a second adjustment value confirmation step (S130), and a correction step (S150) for correcting an angle error of the gimbal based on the calculated angle error value.

[0071] That is, the image analysis device is configured to check the first roll axis adjustment value (dRoll1) and the first pitch axis adjustment value (dPitch1) of the gimbal, which are the first adjustment values ​​generated in the process of rotating the gimbal around the yaw axis in the first adjustment value verification step (S120), and to check the second roll axis adjustment value (dRoll2) and the second pitch axis adjustment value (dPitch2) of the gimbal, which are the second adjustment values ​​generated in the process of rotating the gimbal around the yaw axis in the opposite direction to before but by twice the rotation amount in the second adjustment value verification step (S130), and the checked first and second adjustment values ​​are then applied to a formula to calculate the angular error value (dRoll, dPitch).

[0072] ,

[0073] Then, a control signal is generated based on the calculated angular error value (dRoll, dPitch) and transmitted to the gimbal. The gimbal control unit that receives this controls the attitude of the gimbal so that the angular error can be corrected.

[0074] Next, the gimbal control unit changes the settings so that the posture of the gimbal axis, which is confirmed in a state of error correction based on the control signal, becomes the initial alignment state, so that the result of the error correction can be maintained constant.

[0075] Accordingly, all problems with angular errors in the gimbal are resolved, enabling the acquisition of precise image information targeting a specific target, and the effect of eliminating or minimizing errors in calculating the target's position through image analysis is achieved. This effect can continue until an angular error in the gimbal occurs again due to a drift phenomenon, etc.

[0076]

[0077] In addition, as illustrated in FIG. 5, the present invention may be configured to further include a re-verification step (S160) of re-confirming and tracking a specific target in an image captured in a state where the gimbal's angular error is corrected, and re-confirming the gimbal's adjustment value generated by controlling the gimbal's rotation, and a re-calibration step (S170) of re-correcting the gimbal's error using the adjustment value if the re-confirmed gimbal's adjustment value is greater than or equal to a reference value.

[0078] That is, in general, the problem of angular error occurring in the gimbal can be resolved as the above correction step (S150) is completed, but additional errors in the gimbal angle may occur due to system errors or drift phenomena during the implementation of the present invention.

[0079] Therefore, after the above correction step (S150) is completed, it is necessary to verify the result, and in the re-verification step (S160) for this, confirmation of the image center point, confirmation and tracking of the image target, and rotation control of the gimbal are performed again so that the gimbal adjustment value can be confirmed again, and when the re-verified gimbal adjustment value is higher than the reference value, a re-calibration step (S170) can be performed to correct the gimbal error again.

[0080] At this time, it is desirable that the reference value used in the above recalibration step (S170) be applied differently depending on the current altitude of the drone, which can be checked in real time using the GPS mounted on the drone.

[0081] For example, when the drone's current altitude is less than 50M, the first reference value is applied, when it is more than 50M but less than 100M, the second reference value is applied, and when it is more than 100M but less than 150M, the third reference value is applied, and so on. Different reference values ​​can be applied depending on the current altitude.

[0082] This is because the higher the current altitude of the drone, the greater the error in the results due to the gimbal angle error.

[0083]

[0084] The embodiments introduced above are provided as examples so that the technical idea of ​​the present invention can be sufficiently conveyed to a person having ordinary skill in the art to which the present invention pertains, and the present invention is not limited to the embodiments described above and may be embodied in other forms.

[0085] Additionally, identical reference numbers throughout the specification represent identical components.

[0086]

[0087] A method for correcting an angle error of a gimbal mounted on a drone according to the present invention is as follows:

[0088] This has the effect of enabling precise correction of angular errors in the process of controlling the attitude of a gimbal mounted on a drone, thereby enabling the acquisition of precise image information targeting a specific target, and additionally enabling errors to be eliminated or minimized in calculating the position of a target through image analysis, so it has sufficient potential for industrial use.

Claims

1. A video shooting step (S100) in which the angle of the gimbal is adjusted while the drone is in stationary flight so that the camera mounted on the gimbal takes a video in a vertical downward direction; A target identification step (S110) for identifying a specific target located at the center point of the image being filmed; A first adjustment value confirmation step (S120) for confirming the first adjustment value by automatic adjustment of the gimbal when the gimbal is controlled to rotate around the yaw axis while tracking a confirmed target by video; A second adjustment value confirmation step (S130) for confirming the second adjustment value by automatic adjustment of the gimbal when the gimbal is controlled to rotate in the opposite direction to the previous direction based on the yaw axis while tracking the target by video; An error calculation step (S140) for calculating the angle error value of the gimbal using the confirmed first and second adjustment values; and A method for correcting an angle error of a gimbal mounted on a drone, characterized in that it comprises a correction step (S150) for correcting an angle error of the gimbal based on a calculated angle error value.

2. In paragraph 1, The image analysis device used in the above target confirmation step (S110) is In situations where there is no specific target corresponding to the center point of the image or where feature extraction is difficult, A method for correcting an angle error of a gimbal mounted on a drone, characterized in that a real-time image is divided into four or more sectors, feature points are extracted from each divided sector, and the distance on the screen for each feature point based on the center point of the image is calculated, thereby enabling real-time image tracking to be performed targeting the center point of the image identified based on a plurality of feature points.

3. In paragraph 1, In the first adjustment value confirmation step (S120), It is configured to check the first roll axis adjustment value (dRoll1) and the first pitch axis adjustment value (dPitch1) of the gimbal, which are the first adjustment values ​​generated in the process of rotating the gimbal around the yaw axis. In the above second adjustment value confirmation step (S130), It is configured to check the second roll axis adjustment value (dRoll2) and the second pitch axis adjustment value (dPitch2) of the gimbal, which are secondary adjustment values ​​that occur in the process of rotating the gimbal in the opposite direction to the previous one with the rotation amount doubled based on the yaw axis. In the above error calculation step (S140), The first and second adjustment values, , A method for correcting an angle error of a gimbal mounted on a drone, characterized in that it is configured to calculate an angle error value (dRoll, dPitch) of the gimbal by applying the formula.

4. In paragraph 1, A re-verification step (S160) in which a specific target is re-confirmed and tracked in an image captured in a state in which the angle error of the gimbal is corrected by the above-mentioned correction step (S150), and the gimbal adjustment value generated by controlling the rotation of the gimbal is re-confirmed; and, A method for correcting an angle error of a gimbal mounted on a drone, characterized in that it further comprises a re-calibration step (S170) for correcting the error of the gimbal again using the adjustment value if the re-confirmed adjustment value of the gimbal is greater than or equal to a reference value.

Images