Tracking and Guiding System About Moving Object Based Image And Method Therefor

Abstract

The present invention relates to an image-based ground moving object tracking guidance system and method, and more specifically, to an image-based ground moving object tracking guidance system and method that uses a gimballess camera attached to the lower part of an unmanned aerial vehicle to guide the tracking of a ground moving object that is stationary or moving at a constant speed. According to the image-based ground moving object tracking guidance system and method of the present invention, when a gimballess unmanned aerial vehicle tracks a ground moving object moving at a stationary or constant speed based on image information, the tracking target is guided to be located at the origin of the camera FOV (Field Of View), so that when guiding the tracking of the ground moving object, the tracking target does not move out of the camera FOV (Field Of View), thereby enabling stable tracking, surveillance, and reconnaissance.

Description

Technology Field

[0001] The present invention relates to an image-based ground moving object tracking guidance system and method, and more specifically, to an image-based ground moving object tracking guidance system and method that uses a gimballess camera attached to the lower part of an unmanned aerial vehicle to guide the tracking of a ground moving object that is stationary or moving at a constant speed. Background Technology

[0002] Unmanned aerial vehicles are aircraft operated remotely or automatically without a pilot on board, and are utilized in various civilian and military fields.

[0003] In particular, vertical take-off and landing (VTOL) unmanned aerial vehicles, such as multicopters and tiltrotors, have the advantage of being easy to operate even in confined take-off and landing spaces and capable of hovering, so the method of attaching cameras to VTOL unmanned aerial vehicles is being utilized in various fields such as surveillance / reconnaissance, target striking, and aerial photography.

[0004] While unmanned aerial vehicles can be operated manually by a remote pilot to perform the aforementioned missions, active research is being conducted on methods to automatically track stationary and moving ground targets based on image information to improve operational convenience and reduce the burden of control.

[0005] Among conventional methods for tracking stationary and moving ground targets based on image information, the most commonly used method is to calculate the relative position and velocity between the tracking target and the unmanned aerial vehicle using image information, and to control the aircraft so that the relative position and relative velocity in the horizontal direction converge to zero while maintaining the current altitude.

[0006] However, when applying this technique to an Under Actuated System, such as a multicopter, where the attitude angle and linear acceleration are interdependent in the equation of motion, there is a limitation in that when tracking a ground object moving at a constant speed, the center of the camera is directed toward a point other than the center of the tracking target, so the ground object may move out of the camera's FOV (Field Of View) during tracking guidance.

[0007] For the reasons mentioned above, a method of attaching a gimbal to a camera to control the camera's tilt angle so that a moving object on the ground does not go out of the camera's FOV (Field Of View) is widely used, but there are disadvantages such as increased cost, increased weight, and system complexity when attaching a gimbal, and there are also limitations in that it is often impossible to mount a gimbal on small unmanned aerial vehicles with a small payload. Prior art literature

[0008] Republic of Korea Registered Patent Publication No. 10-2300349 The problem to be solved

[0009] The present invention aims to solve the aforementioned problems and has as its technical objective the improvement of tracking guidance performance of a moving object on the ground by tracking and guiding the unmanned aerial vehicle so that the center of a camera mounted on the lower part of the unmanned aerial vehicle without a gimbal is oriented toward the center of a target that is stationary or moving on the ground. means of solving the problem

[0010] The image-based ground moving object tracking guidance system according to the present invention for achieving the aforementioned technical problem is an image-based ground moving object tracking guidance system that uses image information from a camera attached to the lower part without a gimbal to guide an unmanned aerial vehicle to track a ground moving object that is stationary or moving at a constant speed, and includes an image-based state estimation unit that calculates the relative position and relative speed of a tracking target using image information, a position command compensation unit that calculates a position compensation command of the unmanned aerial vehicle so that the tracking target is positioned at the center of the camera's FOV (Field-Of-View), a tracking capability determination unit that determines whether stable tracking is possible for the tracking target, a guidance command generation unit that calculates a speed command to track the position compensation command calculated by the position command compensation unit, and a guidance command transmission unit that transmits the speed command calculated by the guidance command generation unit to a flight control computer.

[0011] More specifically, the position command compensation unit may include an attitude angle change observer caused by a disturbance, and is characterized by calculating a BIBO (Bounded Input Bounded Output) stable position compensation command for the disturbance.

[0012] More specifically, the position command compensation unit includes a low-pass filter and is characterized by being able to increase the relative stability of the induction command generation unit by reducing the cutoff frequency of the low-pass filter.

[0013] More specifically, the tracking capability determination unit calculates the expected maximum attitude angle information during the tracking process using the relative position and relative velocity information calculated by the image-based state estimation unit and the unmanned aerial vehicle attitude angle information, and determines whether stable tracking guidance is possible by determining whether the expected maximum attitude angle exceeds the maximum attitude angle limit range during tracking guidance.

[0014] More specifically, the tracking capability determination unit is characterized by determining whether stable tracking guidance is possible by determining whether the tracking target does not go outside the FOV (Field Of View) during tracking guidance based on the relative position in the guidance geometry, the camera angle of view, and the expected maximum attitude angle during the tracking process.

[0015] More specifically, the induction command generation unit generates an acceleration command such that the position error and velocity error become zero, and is characterized by generating a velocity command by integrating the acceleration command.

[0016] A video-based ground moving object tracking guidance method according to the present invention for achieving the aforementioned technical problem is a video-based ground moving object tracking guidance method that uses video information from a camera attached to the lower part without a gimbal to guide an unmanned aerial vehicle to track a ground moving object that is stationary or moving at a constant speed, comprising: (a) a step of calculating the relative position and relative speed of a tracking target based on video information; (b) a step of calculating a position compensation command such that the center of the camera in the guidance geometry is positioned at the center of the tracking target; (c) a step of determining whether the target can be tracked using the tracking target relative position information calculated in step (a) and the position compensation command information calculated in step (b); (d) a step of generating an unmanned aerial vehicle speed command such that the center of the camera in the guidance geometry is positioned at the center of the tracking target using the relative position and relative speed information calculated in step (a) and the position compensation command calculated in step (b), when it is determined that tracking is possible in step (c); and (e) a guidance command transmission step of transmitting the speed command generated in step (d) to a flight control computer.

[0017] More specifically, step (b) is characterized by including an attitude angle change observer caused by a disturbance and calculating a BIBO (Bounded Input Bounded Output) stable position compensation command for the disturbance.

[0018] More specifically, step (b) includes a low-pass filter having a time delay, and is characterized by being able to increase the relative stability of the induction command generation unit by reducing the cutoff frequency of the low-pass filter.

[0019] More specifically, step (c) is characterized by calculating the expected maximum attitude angle information during the tracking process using the relative position and relative velocity information and the unmanned aerial vehicle attitude angle information calculated in step (a), and determining whether stable tracking guidance is possible by determining whether the expected maximum attitude angle exceeds the maximum attitude angle limit range during tracking guidance.

[0020] More specifically, step (c) is characterized by including a process of determining whether stable tracking guidance is possible by determining whether the tracking target does not go outside the FOV during tracking guidance based on the relative position in the guidance geometry, the camera angle of view, and the expected maximum attitude angle during the tracking process.

[0021] More specifically, step (d) is characterized by generating an acceleration command such that the position error and the velocity error become zero, and integrating the acceleration command to generate a velocity command. Effects of the invention

[0022] According to the image-based ground moving object tracking guidance system and method of the present invention, when a gimballess unmanned aerial vehicle tracks a ground moving object moving at a stationary or constant speed based on image information, the tracking target is guided to be located at the origin of the camera FOV (Field Of View), so that when guiding the tracking of the ground moving object, the tracking target does not move out of the camera FOV (Field Of View), thereby enabling stable tracking, surveillance, and reconnaissance. Brief explanation of the drawing

[0023] FIG. 1 illustrates the configuration of an image-based ground moving object tracking guidance system according to one embodiment of the present invention. FIG. 2 illustrates a flowchart of an image-based ground moving object tracking guidance method according to one embodiment of the present invention. FIG. 3(a) shows a tracking guidance geometry according to an image-based ground moving object tracking guidance system and method according to an embodiment of the present invention, and FIG. 3(b) shows the relative position range of an unmanned aerial vehicle that can be tracked in the tracking guidance geometry according to an image-based ground moving object tracking guidance system and method according to an embodiment of the present invention. FIG. 4 illustrates a flowchart of a tracking capability determination unit and a tracking capability determination step among an image-based ground moving object tracking guidance system and method according to an embodiment of the present invention. FIG. 5 shows a control flow diagram of a guidance command generation unit and a guidance command generation step among an image-based ground moving object tracking guidance system and method according to an embodiment of the present invention. FIG. 6 shows an embodiment applying the present invention. Specific details for implementing the invention

[0024] The present invention is capable of various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention.

[0025] Hereinafter, an image-based ground moving object tracking guidance system and a method according to an embodiment of the present invention will be described in detail with reference to the attached drawings.

[0027] FIG. 1 illustrates the configuration of an image-based ground moving object tracking guidance system according to one embodiment of the present invention.

[0028] As illustrated in FIG. 1, the image-based ground moving object tracking guidance system (100) according to the present invention is an image-based ground moving object tracking guidance system that uses image information from a camera attached to the lower part without a gimbal to guide an unmanned aerial vehicle to track a ground moving object that is stationary or moving at a constant speed, and includes an image-based state estimation unit (110) that calculates the relative position and relative speed of a tracking target using image information, a position command compensation unit (120) that calculates a position compensation command of the unmanned aerial vehicle so that the tracking target is positioned at the center of the camera's FOV (Field-Of-View), a tracking capability determination unit (130) that determines whether stable tracking is possible for the tracking target, a guidance command generation unit (140) that calculates a speed command to track the position compensation command calculated by the position command compensation unit (120), and a guidance command transmission unit (150) that transmits the speed command calculated by the guidance command generation unit (140) to a flight control computer.

[0029] FIG. 2 illustrates a flowchart of an image-based ground moving object tracking guidance method according to one embodiment of the present invention.

[0030] As illustrated in FIG. 2, the image-based ground moving object tracking guidance method according to the present invention is an image-based ground moving object tracking guidance method that uses image information from a camera attached to the lower part without a gimbal to guide an unmanned aerial vehicle to track a ground moving object that is stationary or moving at a constant speed, comprising: (a) a step of calculating the relative position and relative velocity of a tracking target based on image information (S210); (b) a step of calculating a position compensation command such that the center of the camera in the guidance geometry is positioned at the center of the tracking target (S220); (c) a step of determining whether the target can be tracked using the tracking target relative position information calculated in step (a) and the position compensation command information calculated in step (b) (S230); (d) a step of generating an unmanned aerial vehicle velocity command such that the center of the camera in the guidance geometry is positioned at the center of the tracking target using the relative position and relative velocity information calculated in step (a) and the position compensation command calculated in step (b) when it is determined that tracking is possible in step (c) (S240); and (e) a guidance command transmission that transmits the velocity command generated in step (d) to a flight control computer. Includes step (S250).

[0031] FIG. 3(a) shows a tracking guidance geometry according to an image-based ground moving object tracking guidance system and method according to an embodiment of the present invention, and FIG. 3(b) shows the relative position range of an unmanned aerial vehicle that can be tracked in the tracking guidance geometry according to an image-based ground moving object tracking guidance system and method according to an embodiment of the present invention.

[0032] With reference to FIG. 3, the position command compensation unit (120) and the position compensation command calculation step (S220) of the image-based ground moving object tracking guidance system and method according to one embodiment of the present invention in the configuration described above will be explained in detail.

[0033] As illustrated in FIG. 3, the position command compensation unit (120) and the position compensation command calculation step (S220) are the positions on the surface where the camera center is oriented. a. Location of ground tracking targets The position of the drone to be located at Location compensation command that compensates Calculate.

[0034] At this time, position compensation command The method of calculating is based on the following [Mathematical Formula 1] and the unmanned aerial vehicle acceleration command in the NED coordinate system calculated in the guidance command generation unit (140) and the guidance command generation step (S240). Calculate acceleration commands for the UAV's longitudinal and lateral axes by taking the rotation matrix, and the attitude angle command to generate the corresponding acceleration commands based on [Equation 2] below. , Calculate . Here, the transfer function number representing the response of the UAV autopilot is Multiplying by gives the UAV's attitude angle for acceleration command estimation , You can obtain.

[0035]

[0036]

[0037] In addition, more specifically, the position command compensation unit (120) may include an attitude angle change observer that is generated by a disturbance, and is characterized by calculating a BIBO (Bounded Input Bounded Output) stable position compensation command for the disturbance.

[0038] More specifically, step (b) is characterized by including an attitude angle change observer caused by a disturbance and calculating a BIBO (Bounded Input Bounded Output) stable position compensation command for the disturbance.

[0039] In other words, the attitude angle of the unmanned aerial vehicle , UAV attitude angle for acceleration command estimation , If you subtract, the attitude angle of the UAV generated to offset the disturbance , can be calculated. At this time, , It is not interdependent with acceleration commands, is BIBO stable against disturbances, and If, , It has the characteristic of being. Therefore, , When designing a position compensation command using [this method], it has the characteristic of calculating a position command that is BIBO stable against disturbances without causing inductive instability due to interaction with acceleration commands.

[0040] In addition, more specifically, the position command compensation unit (120) includes a low-pass filter and is characterized by being able to increase the relative stability of the induction command generation unit by reducing the cutoff frequency of the low-pass filter.

[0041] In addition, more specifically, step (b) includes a low-pass filter having a time delay, and is characterized in that the relative stability of the induction command generation unit can be increased by reducing the cutoff frequency of the low-pass filter.

[0042] That is, a position compensation command can be calculated according to the following [Equation 3] derived based on Figure 3 (a), and the high-frequency components of the calculated position compensation command can be removed by passing the low-pass filter through the low-pass filter. At this time, there is a characteristic that the relative stability margin of the inductor increases as the cutoff frequency of the low-pass filter decreases.

[0043]

[0044] FIG. 4 illustrates a flowchart of a tracking capability determination unit and a tracking capability determination step among an image-based ground moving object tracking guidance system and method according to an embodiment of the present invention.

[0045] With reference to FIG. 4, the tracking capability determination unit (130) and the tracking capability determination step (S230) of the image-based ground moving object tracking guidance system and method according to one embodiment of the present invention in the above configuration will be described in detail.

[0046] As illustrated in FIG. 4, the traceability determination unit (130) and the traceability determination step (S230) determine whether traceability is possible in the order illustrated.

[0047] First, if the tracking target is not detected in the image-based state estimation unit (110) and the image-based state estimation step (S110), or if it is determined that the estimated state information is invalid even if detected, tracking is impossible ( It is judged as ).

[0048] In addition, detect the tracking target, and if the estimated state information is valid, the maximum predicted attitude angle occurring during the tracking guidance process based on the following [Equation 4] ( , Calculate ) and the calculated value is the maximum attitude angle limit ( , If it exceeds ), it is also untraceable ( It is judged as ).

[0049]

[0050] In addition, maximum attitude angle limit ( , If it does not exceed ), the traceable range ( Calculate ) using [Mathematical Formula 5], and if the relative position of the UAV to the tracking target exceeds the corresponding range, tracking is impossible ( It is determined as ), and otherwise traceable( It is judged as ).

[0051]

[0052] That is, the tracking capability determination unit (130) of the image-based ground moving object tracking guidance system (100) according to one embodiment of the present invention calculates the expected maximum attitude angle information during the tracking process using the relative position and relative speed information calculated by the image-based state estimation unit (110) and the unmanned aerial vehicle attitude angle information, and determines whether stable tracking guidance is possible by determining whether the expected maximum attitude angle exceeds the maximum attitude angle limit range during tracking guidance.

[0053] In addition, more specifically, the tracking capability determination unit (130) is characterized by determining whether stable tracking is possible by determining whether the tracking target does not go outside the FOV (Field Of View) during tracking guidance based on the relative position in the guidance geometry, the camera angle of view, and the expected maximum attitude angle information during the tracking process.

[0054] That is, step (c) of the image-based ground moving object tracking guidance method according to one embodiment of the present invention is characterized by calculating the expected maximum attitude angle information during the tracking process using the relative position and relative velocity information calculated in step (a) and the unmanned aerial vehicle attitude angle information, and determining whether stable tracking guidance is possible by determining whether the expected maximum attitude angle exceeds the maximum attitude angle limit range during tracking guidance.

[0055] In addition, more specifically, step (c) is characterized by including a process of determining whether stable tracking guidance is possible by determining whether the tracking target does not go outside the FOV during tracking guidance based on the relative position in the guidance geometry, the camera angle of view, and the expected maximum attitude angle during the tracking process.

[0056] FIG. 5 shows a control flow diagram of a guidance command generation unit and a guidance command generation step among an image-based ground moving object tracking guidance system and method according to an embodiment of the present invention.

[0057] With reference to FIG. 5, the guidance command generation unit (140) and the guidance command generation step (S240) of the image-based ground moving object tracking guidance system and method according to one embodiment of the present invention in the configuration described above will be described in detail.

[0058] First, a position error is calculated by adding a position compensation command to the relative position information calculated in the image-based state estimation unit (110) and the image-based state estimation step (S210), and the position error and the velocity error are each multiplied by a control gain, and then the two values ​​are added to calculate an acceleration command. At this time, the control gain can be an energy-optimal control gain that guarantees stability by designing a linear quadratic controller (LQR) for the state-space equation [Equation 6] below, and finally, the calculated acceleration command is integrated to generate a velocity command.

[0059]

[0060] That is, the guidance command generation unit (140) of the image-based ground moving object tracking guidance system (100) according to one embodiment of the present invention generates an acceleration command that makes the position error and the velocity error zero, and generates a velocity command by integrating the acceleration command.

[0061] In addition, step (d) of the image-based ground moving object tracking guidance method according to one embodiment of the present invention is characterized by generating an acceleration command that makes the position error and the velocity error zero, and integrating the acceleration command to generate a velocity command.

[0062] FIG. 6 shows an embodiment applying the present invention.

[0063] Referring to FIG. 6, this figure compares the value of the position the camera is pointing toward when tracking and guiding a ground moving object moving at a constant speed of 5 m / s, measured by a conventional tracking guidance method, with the value measured by the ground moving object tracking guidance system and method according to the present invention. The conventional tracking guidance method Although the center of the camera's FOV is oriented toward the rear of the tracked target, an embodiment of the present invention It can be seen that this enables stable tracking guidance by aiming at the center of the tracking target.

[0065] Although the present invention has been described above with reference to the embodiments illustrated in the drawings, this is merely for the purpose of explaining the invention, and those skilled in the art will understand that various modifications or equivalent embodiments are possible from the detailed description of the invention.

[0066] Therefore, the true scope of rights of the present invention must be determined by the technical concept of the patent claims. Explanation of the symbols

[0067] 110: Image-based state estimation unit 120: Location Command Compensation Unit 130: Traceability determination unit 140: Induction command generation unit 150: Induction command transmitter

Claims

Claim 1 An image-based ground object tracking guidance system that uses image information from a camera attached to the bottom without a gimbal to guide an unmanned aerial vehicle to track a ground object moving at a stationary or constant speed, comprising: an image-based state estimation unit that calculates relative position and relative speed information between a tracking target and the unmanned aerial vehicle using the image information; a position command compensation unit that calculates a position compensation command for the unmanned aerial vehicle so that the tracking target is positioned at the center of the camera's FOV (Field-Of-View); a tracking capability determination unit that determines whether stable tracking of the tracking target is possible; a guidance command generation unit that calculates a speed command to track the position compensation command calculated by the position command compensation unit; and a guidance command transmission unit that transmits the speed command calculated by the guidance command generation unit to a flight control computer. A video-based ground moving object tracking guidance system comprising: a tracking capability determination unit that calculates an expected maximum attitude angle information during the tracking process of the tracking target using information on relative position and relative velocity calculated by the video-based state estimation unit and attitude angle information of the unmanned aerial vehicle, and determines whether stable tracking guidance of the tracking target is possible by determining whether the expected maximum attitude angle exceeds a maximum attitude angle limit range during tracking guidance; and a guidance command generation unit that calculates a position error by adding a position compensation command calculated by the position command compensation unit to the relative position information calculated by the video-based state estimation unit, calculates an acceleration command by multiplying the position error and the velocity error by their respective control gains and adding the two values, generates an acceleration command such that the position error and the velocity error become zero, and generates a velocity command by integrating the acceleration command. Claim 2 An image-based ground moving object tracking guidance system according to claim 1, wherein the position command compensation unit includes an attitude angle change observer generated by a disturbance and calculates a BIBO (Bounded Input Bounded Output) stable position compensation command for the disturbance. Claim 3 An image-based ground moving object tracking guidance system according to claim 1, wherein the position command compensation unit includes a low-pass filter having a time delay and can increase the relative stability of the guidance command generation unit by reducing the cutoff frequency of the low-pass filter. Claim 4 delete Claim 5 An image-based ground moving object tracking guidance system according to claim 1, wherein the tracking capability determination unit determines whether stable tracking guidance is possible by determining whether the tracking target does not move outside the FOV during tracking guidance based on the relative position in the guidance geometry, the camera angle of view, and the expected maximum attitude angle information during the tracking process. Claim 6 delete Claim 7 A video-based ground object tracking guidance method that uses video information from a camera attached to the bottom without a gimbal to guide an unmanned aerial vehicle to track a ground object moving at a stationary or constant speed, comprising: (a) a step of calculating the relative position and relative velocity of a tracking target based on video information; (b) a step of calculating a position compensation command such that the center of the camera in the guidance geometry is positioned at the center of the tracking target; (c) a step of determining whether the target is trackable using the tracking target relative position information calculated in step (a) and the position compensation command information calculated in step (b); (d) a step of generating an unmanned aerial vehicle speed command such that the center of the camera in the guidance geometry is positioned at the center of the tracking target using the relative position and relative velocity information calculated in step (a) and the position compensation command calculated in step (b), if it is determined that tracking is possible in step (c); and (e) a guidance command transmission step of transmitting the speed command generated in step (d) to a flight control computer. A method for tracking and guiding an image-based ground moving object, comprising: a step (c) using relative position and relative velocity information calculated in step (a) and attitude angle information of the unmanned aerial vehicle to calculate expected maximum attitude angle information during the tracking process of the tracking target, and determining whether stable tracking guidance of the tracking target is possible by determining whether the expected maximum attitude angle exceeds the maximum attitude angle limit range during tracking guidance; and a step (d) calculating a position error by adding a position compensation command calculated in step (b) to the relative position information calculated in step (a), calculating an acceleration command by multiplying the position error and the velocity error by their respective control gains and adding the two values, generating an acceleration command such that the position error and the velocity error become zero, and generating a velocity command by integrating the acceleration command. Claim 8 In claim 7, the above step (b) includes a step of calculating an attitude angle change caused by a disturbance, and is characterized by calculating a BIBO stable position compensation command for the disturbance. Image-based ground moving object tracking guidance method. Claim 9 In claim 7, the above step (b) includes a step of calculating a position compensation command having only low-frequency characteristics using a low-pass filter, and is characterized in that the relative stability of the guidance command generation unit can be increased by reducing the cutoff frequency of the low-pass filter. Claim 10 delete Claim 11 A method for tracking a ground-based moving object based on claim 7, wherein step (c) includes a process of determining whether stable tracking guidance is possible by determining whether the tracking target does not move outside the FOV during tracking guidance based on the relative position in the guidance geometry, the camera angle of view, and the expected maximum attitude angle information during the tracking process. Claim 12 delete

Images