Target tracking moving image shooting system of unmanned aerial vehicle

Abstract

The invention relates to the technical field of unmanned aerial vehicle image shooting, and discloses an unmanned aerial vehicle target tracking moving image shooting system, which comprises the steps of acquiring image information, generating a plurality of shooting targets according to the acquired image information and preset feature information, performing comparison and determination according to the plurality of generated shooting targets, and after a tracking target is determined through comparison, performing target tracking according to the tracking target. Feeding back the determined information to a ground control station to control the unmanned aerial vehicle to track and shoot the tracked target, determining the position coordinates of the target in the image by an algorithm after determining the tracked and shot target, transmitting the coordinate information to a target tracking module 5, and tracking the target by the target tracking module 5 according to the received position coordinates of the target. Parameters of Kalman filtering and particle filtering algorithms are initialized, along with movement of the unmanned aerial vehicle and movement of the target, the target tracking module 5 continuously updates a position prediction value of the target and corrects the prediction value according to actually collected image data, and therefore it is ensured that the unmanned aerial vehicle can conduct real-time correction tracking according to the movement track of the target.

Description

Technical Field

[0001] The invention belongs to the technical field of unmanned aerial vehicle image shooting, and in particular relates to an unmanned aerial vehicle target tracking moving image shooting system. Background Art

[0002] With the rapid development of drone technology, it has been widely used in aerial photography, monitoring, surveying and mapping, etc. However, in many scenarios, drones are required to automatically track specific targets and take clear images of their movement, such as tracking athletes in sports events and tracking individual animals in wildlife monitoring.

[0003] Traditional drone photography often requires manual control of the drone to aim at the target, which not only requires high skills from the operator, but also makes it difficult to ensure stable and accurate tracking of the target for a long time, and is prone to problems such as target loss or shaking or blurring of the shooting image.

[0004] In view of this, the present invention is proposed. Summary of the invention

[0005] In order to solve the above technical problems that traditional drone photography often requires manual control of the drone to aim at the target, which not only requires high skills of the operator, but also makes it difficult to ensure long-term stable and accurate tracking of the target, and is prone to target loss or jitter and blur of the shooting picture, the basic concept of the technical solution adopted by the present invention is:

[0006] A UAV target tracking moving image shooting system, the system method is as follows:

[0007] S1: collect image information;

[0008] S2: generating a plurality of shooting targets according to the collected image information and preset feature information;

[0009] S3: performing comparison and determination according to the generated multiple shooting targets;

[0010] S4: After comparing and determining the tracking target, the determination information is fed back to the ground control station to control the UAV to track and photograph the tracking target.

[0011] As a preferred embodiment of the present invention, the following specific steps are included:

[0012] First, the drone is started to collect images of the on-site environment. The collected image data is input into the image recognition algorithm for processing, and then the target is identified through the target recognition module. Then the target is determined by the target determination block. If the target is identified, the algorithm will determine the position coordinates of the target in the image and pass the coordinate information to the target tracking module. The target tracking module initializes the parameters of the Kalman filter and particle filter algorithms based on the received target position coordinates. As the drone moves and the target moves, the target tracking module continuously updates the target position prediction value and corrects the prediction value based on the actual collected image data. During the shooting process, the gimbal stabilization module can also automatically adjust the shooting parameters according to factors such as ambient light and target movement speed to ensure clear images. Finally, the determined tracking target is fed back to the ground control station through the tracking instruction sending module. Finally, the ground control station performs overall control and parameter modification to complete automatic tracking of the tracking target.

[0013] As a preferred embodiment of the present invention, the following modules are included:

[0014] The image acquisition module is used to capture and shoot the on-site environment, so as to facilitate subsequent comparison and analysis with the target image in the target database;

[0015] A shooting target generation module, used to synthesize the collected image information and preset feature information to generate a shooting target;

[0016] The target recognition module analyzes the image and uses the labeled data to train the convolutional neural network. By adjusting the network parameters and optimizing the algorithm, the model can achieve a higher target recognition accuracy.

[0017] A target determination module, used to determine the identified shooting target and the generated shooting target;

[0018] Target tracking module, in which Kalman filter and extended Kalman filter algorithms are implemented; according to the target's motion characteristics and actual application scenarios, the parameters in the algorithm, such as the state transfer matrix, observation matrix, etc., are determined; through simulation and actual testing, the algorithm is optimized to ensure that it can accurately predict and track the target's motion trajectory;

[0019] The gimbal stabilization module develops a feedback control system based on the PID control algorithm. According to the data of the attitude sensor and the target position information transmitted by the target tracking module, the control parameters of the gimbal motor are calculated. Then, by continuously adjusting the PID parameters (proportional coefficient, integral time constant, differential time constant), the gimbal can quickly and stably adjust the camera's attitude.

[0020] The tracking command sending module is used to send the determined tracking target command feedback to the ground control station.

[0021] As a preferred embodiment of the present invention, based on the specific process of the above-mentioned image acquisition module and the shooting target generation module, the image acquisition module is used to collect and summarize the external information of the large environment, and then the collected and summarized environmental information is used to generate the shooting target through the shooting target generation module, so as to facilitate the subsequent comparison and analysis with the target image in the target database.

[0022] As a preferred embodiment of the present invention, based on the specific processes of the target recognition module and the target determination module, the target recognition module performs deep algorithm learning and recognition on the collected image information and the target image in the target database, thereby improving the accuracy and speed of recognition, and then the recognized target is analyzed and determined by the target determination module.

[0023] As a preferred embodiment of the present invention, based on the specific process of the target tracking module, the linear motion of the target is predicted and corrected by the Kalman filter algorithm, so as to ensure that the image will not be blurred during the acquisition process. At the same time, the particle filter algorithm can accurately estimate the position of the target during the nonlinear motion process. Through this hybrid filtering algorithm, high-precision target tracking can be achieved in different motion states.

[0024] As a preferred embodiment of the present invention, the specific recognition process based on the target recognition module is as follows:

[0025] After identification and comparison, the tracking target will be determined and then fed back to the ground control station through the tracking command sending module to control the tracking and parameter modification of the overall process;

[0026] If the recognition comparison is uncertain, the image will be captured again and compared with the target image feature information in the target database for analysis.

[0027] As a preferred embodiment of the present invention, based on the specific process of the above-mentioned gimbal stabilization module, it is used to quickly and stably adjust the posture of the camera on the drone to adapt to the motion trajectory of different targets and ensure that the tracked target is clearer.

[0028] As a preferred embodiment of the present invention, based on the specific process of the above-mentioned tracking instruction sending module, it is used to send the characteristic information of the determined tracking target to the ground control station, and then the ground control station makes the final confirmation and uncertainty, so as to further determine the accuracy of the tracking target.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] The present invention firstly connects the drone to the ground control station through a mobile network, then performs image acquisition through the drone, and then generates multiple shooting targets by combining the images collected by the drone with preset characteristic image information, and detects the posture change of the drone camera in real time through the sensor in the gimbal stabilization module. When the vibration or tilt of the drone is detected, the posture of the camera is adjusted by controlling the rotation of the motor to ensure that the camera is always aimed at the target and remains stable. After the generation of the shooting target is completed, a comparison can be performed to determine the target to be tracked and shot. The algorithm determines the position coordinates of the target in the image and transmits the coordinate information to the target tracking module. The target tracking module initializes the parameters of the Kalman filter and the particle filter algorithm according to the received target position coordinates. As the drone moves and the target moves, the target tracking module continuously updates the target position prediction value, and corrects the prediction value according to the actually collected image data, so as to ensure that the drone can perform real-time correction tracking according to the trajectory of the target movement, and ensure that the shot image is clearer. Finally, the shooting target determined to be tracked is fed back to the ground control station through the tracking instruction sending module, and finally the ground control station performs overall control and parameter modification, so as to complete automatic tracking and shooting of the tracking target.

[0031] The specific implementation modes of the present invention are further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In the attached picture:

[0033] Figure 1 A schematic diagram of the interaction between a UAV and a ground control station in a UAV target tracking and moving image shooting system;

[0034] Figure 2 It is a schematic diagram of the overall process of a UAV target tracking moving image shooting system;

[0035] Figure 3 A schematic diagram of the image analysis process of a UAV target tracking mobile image shooting system Figure 2 ;

[0036] Figure 4 Schematic diagram of the image analysis process of a UAV target tracking mobile image shooting system Figure 2 ;

[0037] Figure 5 The module diagram of a UAV target tracking moving image shooting system is shown in FIG.

[0038] In the figure: 1. Image acquisition module; 2. Shooting target generation module; 3. Target recognition module; 4. Target determination module; 5. Target tracking module; 6. Pan / tilt stabilization module; 7. Tracking instruction sending module. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The following embodiments are used to illustrate the present invention.

[0040] Embodiment 1:

[0041] like Figures 1 to 5 As shown, a UAV target tracking mobile image shooting system, the system method is as follows:

[0042] S1: collect image information;

[0043] S2: Generate multiple shooting targets according to the collected image information and preset feature information;

[0044] S3: performing comparison and determination according to the generated multiple shooting targets;

[0045] S4: After comparing and confirming the tracking target, the confirmation information is fed back to the ground control station to control the UAV to track and shoot the tracking target.

[0046] Further, the following specific steps are included:

[0047] First, the drone is started to collect images of the on-site environment. The collected image data is input into the image recognition algorithm for processing. Then, the target is identified by the target recognition module 3, and then the target is determined by the target determination block 4. If the target is identified, the algorithm will determine the position coordinates of the target in the image and pass the coordinate information to the target tracking module 5. The target tracking module 5 initializes the parameters of the Kalman filter and particle filter algorithms according to the received target position coordinates. As the drone moves and the target moves, the target tracking module 5 continuously updates the target position prediction value and corrects the prediction value according to the actual collected image data. During the shooting process, the gimbal stabilization module 6 can also automatically adjust the shooting parameters according to factors such as ambient light and target movement speed to ensure clear shooting images. Finally, the determined shooting target is fed back to the ground control station through the tracking instruction sending module 7. Finally, the ground control station performs overall control and parameter modification to complete automatic tracking and shooting of the tracking target.

[0048] Furthermore, the following modules are included:

[0049] Image acquisition module 1, used to capture and shoot the scene environment, so as to facilitate subsequent comparison and analysis with the target image in the target database;

[0050] The shooting target generation module 2 is used to synthesize the collected image information and the preset feature information to generate the shooting target;

[0051] Target recognition module 3 analyzes the image and uses the labeled data to train the convolutional neural network. By adjusting the network parameters and optimizing the algorithm, the model can achieve a higher target recognition accuracy.

[0052] The target determination module 4 is used to determine the identified shooting target and the generated shooting target;

[0053] Target tracking module 5, in which Kalman filter and extended Kalman filter algorithms are implemented; according to the motion characteristics of the target and the actual application scenario, the parameters in the algorithm, such as the state transfer matrix, the observation matrix, etc., are determined, and the algorithm is optimized through simulation and actual testing to ensure that it can accurately predict and track the motion trajectory of the target;

[0054] The pan-tilt stabilization module 6 develops a feedback control system based on the PID control algorithm. According to the data of the attitude sensor and the target position information transmitted by the target tracking module 5, the control parameters of the pan-tilt motor are calculated. Then, by continuously adjusting the PID parameters (proportional coefficient, integral time constant, differential time constant), the pan-tilt can quickly and stably adjust the attitude of the camera.

[0055] The tracking instruction sending module 7 is used to send instructions for the determined tracking target and feed them back to the ground control station.

[0056] Furthermore, based on the specific process of the above-mentioned image acquisition module 1 and the shooting target generation module 2, the image acquisition module 1 is used to collect and summarize the external information of the large environment, and then the collected and summarized environmental information is used to generate a shooting target through the shooting target generation module 2, so as to facilitate subsequent comparison and analysis with the target image in the target database.

[0057] Based on the specific processes of the target recognition module 3 and the target determination module 4, the target recognition module 3 performs deep algorithm learning and recognition on the collected image information and the target image in the target database, thereby improving the accuracy and speed of recognition, and then the target determination module 4 analyzes and determines the identified target.

[0058] Based on the specific process of the target tracking module 5, the linear motion of the target is predicted and corrected by the Kalman filter algorithm, thereby ensuring that the image will not be blurred during the acquisition process. At the same time, the particle filter algorithm can accurately estimate the position of the target during nonlinear motion. Through this hybrid filtering algorithm, high-precision target tracking can be achieved in different motion states.

[0059] Specific recognition process based on target recognition module 3:

[0060] After identification and comparison, the tracking target will be determined and then fed back to the ground control station through the tracking instruction sending module 7 to control the tracking and parameter modification of the overall process;

[0061] If the recognition comparison is uncertain, the image will be captured again and compared with the target image feature information in the target database for analysis.

[0062] Based on the specific process of the above-mentioned gimbal stabilization module 6, it is used to quickly and stably adjust the posture of the camera on the drone to adapt to the movement trajectory of different targets, ensuring that the tracked target is clearer.

[0063] Based on the specific process of the tracking instruction sending module 7, the characteristic information of the determined tracking target is sent to the ground control station, and then the ground control station makes the final confirmation and uncertainty to further determine the accuracy of the tracking target.

[0064] The implementation principle of a drone target tracking mobile image shooting system of this embodiment is as follows: first, the drone is interactively connected with the ground control station through a mobile network, and then the drone is used to collect images. After that, the images collected by the drone and the preset characteristic image information are used to generate multiple shooting targets. The sensor in the gimbal stabilization module 6 detects the posture change of the drone camera in real time. When the vibration or tilt of the drone is detected, the posture of the camera is adjusted by controlling the rotation of the motor to ensure that the camera is always aimed at the target and remains stable. After the generation of the shooting target is completed, a comparison can be performed to determine the target to be tracked and shot. The algorithm determines the position coordinates of the target in the image and transmits the coordinate information to the target tracking module 5. The target tracking module 5 initializes the parameters of the Kalman filter and the particle filter algorithm according to the received target position coordinates. As the drone moves and the target moves, the target tracking module 5 continuously updates the target position prediction value and corrects the prediction value according to the actually collected image data, thereby ensuring that the drone can perform real-time correction tracking according to the trajectory of the target movement, ensuring that the captured image is clearer, and finally the determined tracking shooting target is fed back to the ground control station through the tracking instruction sending module 7. Finally, the ground control station performs overall control and parameter modification to complete automatic tracking and shooting of the tracking target.

Claims

1. A UAV target tracking moving image shooting system, characterized in that: The system method is as follows: S1: collect image information; S2: generating a plurality of shooting targets according to the collected image information and preset feature information; S3: performing comparison and determination according to the generated multiple shooting targets; S4: After comparing and determining the tracking target, the determination information is fed back to the ground control station to control the UAV to track and photograph the tracking target.

2. The unmanned aerial vehicle target tracking moving image shooting system according to claim 1, characterized in that: The specific steps include: First, the drone is started to collect images of the on-site environment. The collected image data is input into the image recognition algorithm for processing. Then, the target is identified through the target recognition module (3). Then, the target is determined through the target determination block (4). If the target is identified, the algorithm will determine the position coordinates of the target in the image and transmit the coordinate information to the target tracking module (5). The target tracking module (5) initializes the parameters of the Kalman filter and particle filter algorithms according to the received target position coordinates. As the drone moves and the target moves, the target tracking module (5) continuously updates the target position prediction value and corrects the prediction value according to the actual collected image data. During the shooting process, the gimbal stabilization module (6) can also automatically adjust the shooting parameters according to factors such as ambient light and target movement speed to ensure clear shooting images. Finally, the determined shooting target is fed back to the ground control station through the tracking instruction sending module (7). Finally, the ground control station performs overall control and parameter modification, thereby completing automatic tracking and shooting of the tracking target.

3. The unmanned aerial vehicle target tracking moving image shooting system according to claim 1, characterized in that: Includes the following modules: An image acquisition module (1) is used to capture and photograph the on-site environment, thereby facilitating subsequent comparative analysis with target images in a target database; A shooting target generation module (2) is used to synthesize the collected image information and preset feature information to generate a shooting target; The target recognition module (3) analyzes the image and uses the labeled data to train the convolutional neural network. By adjusting the network parameters and optimizing the algorithm, the model can achieve a higher target recognition accuracy. A target determination module (4), used to determine the identified shooting target and the generated shooting target; A target tracking module (5) is used to implement Kalman filtering and extended Kalman filtering algorithms in the target tracking module (5); according to the motion characteristics of the target and the actual application scenario, the parameters in the algorithm, such as the state transfer matrix, the observation matrix, etc., are determined, and the algorithm is optimized through simulation and actual testing to ensure that it can accurately predict and track the motion trajectory of the target; The pan-tilt stabilization module (6) develops a feedback control system based on the PID control algorithm, calculates the control parameters of the pan-tilt motor according to the data of the attitude sensor and the target position information transmitted by the target tracking module (5), and then continuously adjusts the PID parameters (proportional coefficient, integral time constant, differential time constant) so that the pan-tilt can quickly and stably adjust the attitude of the camera; The tracking instruction sending module (7) is used to send instructions for the determined tracking target and feed them back to the ground control station.

4. The unmanned aerial vehicle target tracking moving image shooting system according to claim 3, characterized in that: Based on the specific process of the above-mentioned image acquisition module (1) and the shooting target generation module (2), the image acquisition module (1) is used to collect and summarize the external information of the large environment, and then the collected and summarized environmental information is used to generate a shooting target through the shooting target generation module (2), so as to facilitate subsequent comparison and analysis with the target image in the target database.

5. The unmanned aerial vehicle target tracking moving image shooting system according to claim 3, characterized in that: Based on the specific process of the target recognition module (3) and the target determination module (4), the target recognition module (3) performs deep algorithm learning and recognition on the collected image information and the target image in the target database, thereby improving the accuracy and speed of recognition, and then the target determination module (4) analyzes and determines the recognized target.

6. The unmanned aerial vehicle target tracking moving image shooting system according to claim 3, characterized in that: Based on the specific process of the target tracking module (5), the linear motion of the target is predicted and corrected by the Kalman filter algorithm, thereby ensuring that the image will not be blurred during the acquisition process. At the same time, the particle filter algorithm can accurately estimate the position of the target during the nonlinear motion process. Through this hybrid filtering algorithm, high-precision target tracking can be achieved under different motion states.

7. The unmanned aerial vehicle target tracking moving image shooting system according to claim 5, characterized in that: The specific recognition process based on the target recognition module (3) is as follows: After identification and comparison, the tracking target will be determined and then fed back to the ground control station through the tracking instruction sending module (7) to control the tracking and parameter modification of the overall process; If the recognition comparison is uncertain, the image will be captured again and compared with the target image feature information in the target database for analysis.

8. The unmanned aerial vehicle target tracking moving image shooting system according to claim 3, characterized in that: Based on the specific process of the above-mentioned gimbal stabilization module (6), the camera on the drone can be quickly and stably adjusted to adapt to the movement trajectory of different targets, ensuring that the target being tracked and photographed is clearer.

9. The unmanned aerial vehicle target tracking moving image shooting system according to claim 3, characterized in that: Based on the specific process of the tracking instruction sending module (7), the characteristic information of the determined tracking target is sent to the ground control station, and then the ground control station makes the final determination and uncertainty, so as to further determine the accuracy of the tracking target.

Images