Target identification tracking system based on unmanned aerial vehicle
By integrating data processing, object detection and visual positioning technology on drones, combining lightweight neural networks and monocular visual positioning algorithms, the shortcomings of traditional algorithms in target recognition and tracking are solved, and high-precision and fast targets are achieved Identification and tracking, suitable for a variety of application scenarios.
Patent Information
- Application Number
- CN202411348975.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2024-11-12
AI Technical Summary
Traditional target recognition and tracking algorithms have insufficient ability to express target features, are slow and have low accuracy, and have failed to effectively solve the continuous tracking and recognition problems of targets in dynamic environments.
A drone-based target recognition and tracking system is designed, using data processing and expansion modules, object detection and tracking algorithm modules, visual positioning and control modules, information sampling and image processing modules, intelligent analysis and data management modules, combined with lightweight The order of magnitude neural network, large convolutional kernel attention mechanism, border regression loss function, monocular visual positioning algorithm and PID closed-loop control technology to achieve high-precision recognition and tracking of goals.
It realizes high-precision identification and tracking of targets such as faces, vehicles, license plates, etc. The system can track targets stably for a long time in a dynamic environment, improves the expressive ability and recognition speed of target characteristics, and is suitable for criminal case investigation and traffic accident handling and other fields.
Smart Images

Figure CN118941998A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep learning technology, and in particular to a target recognition and tracking system based on an unmanned aerial vehicle. Background Art
[0002] Object recognition is a key task that uses computer technology to simulate human visual functions. It aims to give computers the ability to perceive, identify and understand the surrounding environment from one or more images or videos. Its research focuses on the comprehensive perception and understanding of the three-dimensional environment of the objective world. As an important branch of computer vision technology, object recognition is not only the detection and classification of objects (such as people, vehicles, etc.) in the field of view, but also covers the analysis and understanding of their behavior. The object recognition process is usually divided into several steps: first, the target is located in the image through object detection technology, and then specific recognition is performed, that is, the category, identity and other information of the target are determined, and finally, the target's motion, position and other characteristics are combined to analyze its behavior or state. However, traditional object recognition and tracking algorithms, and deep learning methods, are insufficient in expressing target features, slow and low in accuracy.
[0003] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention
[0004] In view of the problems in the related art, the present invention proposes a target recognition and tracking system based on a drone to overcome the above-mentioned technical problems existing in the existing related art.
[0005] To this end, the specific technical solution adopted by the present invention is as follows:
[0006] A target recognition and tracking system based on a drone, the target recognition and tracking system based on a drone comprises: a data processing and expansion module, a target detection and tracking algorithm module, a visual positioning and control module, an information sampling and image processing module, and an intelligent analysis and data management module;
[0007] The data processing and expansion module is used to standardize the images taken by the drone and to expand and normalize the original data;
[0008] The target detection and tracking algorithm module is used to detect and track targets based on a lightweight neural network model, introduce a large convolution kernel attention mechanism, and use a bounding box regression loss function;
[0009] The visual positioning and control module is used to use the monocular visual positioning algorithm and PID closed-loop control technology to achieve real-time positioning and stable flight of the UAV;
[0010] The information sampling and image processing module is used to sample the video taken by the drone at intervals and perform image enhancement preprocessing;
[0011] The intelligent analysis and data management module is used to perform semantic analysis on detected targets and store structured data in the database.
[0012] Furthermore, the images taken by the drone are standardized, and the original data are expanded and normalized, including:
[0013] The collection and annotation of drone images are completed by using existing dataset subsets, network collection and manual annotation in turn;
[0014] The image data is cut, noise is removed, color is adjusted, and rotation and translation are performed to expand data diversity;
[0015] The images are processed into a standardized standard data set through mean processing, normalization and size adjustment.
[0016] Furthermore, based on the lightweight neural network model, a large convolution kernel attention mechanism is introduced, and the bounding box regression loss function is used for target detection and tracking, including:
[0017] Design a target detection model based on a lightweight neural network model;
[0018] Introducing a large convolution kernel attention mechanism to improve the target detection model’s ability to capture contextual information and enhance the recognition accuracy of target features;
[0019] The bounding box regression loss function is used to optimize the bounding box regression by taking into account the scale, angle and position errors of the object.
[0020] Furthermore, the monocular vision positioning algorithm and PID closed-loop control technology are used to achieve real-time positioning and stable flight of the UAV, including:
[0021] The TLD algorithm is used to track the target for a long time to ensure the continuous tracking of the target in a dynamic environment;
[0022] The visible light image of the UAV is used to obtain the image coordinates of the target based on the monocular vision positioning algorithm, and the position of the target in the geographic coordinate system is calculated by combining the UAV flight altitude and camera viewing angle parameters;
[0023] Combined with the gimbal and fuselage status parameters of the drone, a PID closed-loop controller is used to adjust the flight status and gimbal angle to ensure that the target is always in the ideal position.
[0024] Furthermore, the TLD algorithm is used to track the target for a long time to ensure the continuous tracking of the target in a dynamic environment, including:
[0025] Determine whether the video sequence shot by the drone is the first frame image. If it is the first frame image, manually select the initial target frame. Otherwise, start filtering according to the tracking target motion model and state update equation in the update formula;
[0026] Through artificial fish swarm particle filtering technology, the area where the target is located is screened out and input into the detection block to further confirm the existence of the target;
[0027] If the detection block passes through the variance classifier, random forest classifier and nearest neighbor classifier in sequence, it is determined that the area contains a target. At the same time, the fusion features of the color histogram and directional gradient histogram of the filter area are extracted, the similarity with the initial target is calculated as the particle weight, and the maximum a posteriori probability criterion is used to update the tracking area.
[0028] Furthermore, the update formula is:
[0029]
[0030] In the formula, M represents the target state;
[0031] M i Represented as the target state at time i;
[0032] (x, y) represents the center coordinates of the target tracking area;
[0033] k represents the width of the target tracking area;
[0034] h represents the height of the target tracking area;
[0035] α is the scale transformation factor;
[0036] A and B are both expressed as particle update coefficients;
[0037] C represents the external interference term;
[0038] Z represents Gaussian noise.
[0039] Furthermore, the image coordinates of the target are obtained through the visible light image of the drone based on the monocular vision positioning algorithm, and the position of the target in the geographic coordinate system is calculated in combination with the drone flight altitude and camera viewing angle parameters, including the following steps:
[0040] The forward-view image acquired by the UAV is converted from the image coordinate system to a parallel projection plane;
[0041] If the drone camera has been calibrated, the internal parameters can be used directly to calculate the position; if it has not been calibrated, the internal parameters of the camera can be calculated through multi-point correspondence;
[0042] Based on multiple feature points on the ground target object, a geometric reference is constructed, and the geometric features are converted into projections on the image plane under the current viewing angle through affine transformation;
[0043] The linear least squares method is used to estimate the parameters of the affine transformation to calculate the geometric relationship between the drone's view and the ground objects, and the target image coordinates in the current frame are obtained;
[0044] Through geometric information analysis and optimization algorithms, the precise position and posture parameters of the drone camera are iteratively solved to obtain the specific position and posture of the drone at the current time point;
[0045] The solution result of the current frame is used as the initial condition of the next frame, and the relative position of the drone is continuously tracked in combination with the constraint relationship between the previous and next frames.
[0046] Furthermore, the linear least squares method is used to estimate the parameters of the affine transformation to calculate the geometric relationship between the drone's perspective and the ground object, and the target image coordinates in the current frame are obtained:
[0047] Get the landmark point dataset of the ground target object in the current frame;
[0048] Using all the landmark data, the linear least square method is used to estimate the parameters of the preliminary affine transformation to obtain the initial affine transformation parameters;
[0049] In the feature point data set, two groups of landmark points are removed pair by pair, and the linear least squares method is reapplied using the remaining landmark point data to calculate new affine transformation parameters;
[0050] After removing two sets of landmarks each time, calculate the difference between the newly obtained affine transformation parameters and the initial estimate, and record the difference value calculated each time;
[0051] The process of removing landmarks and calculating differences is repeated until the preset maximum number of iterations is reached;
[0052] Analyze all recorded difference values and select a set of affine transformation parameters with the smallest difference as the final estimate to ensure the best affine transformation model. Use the affine transformation model to calculate the geometric relationship between the drone's perspective and ground objects to obtain the target image coordinates in the current frame.
[0053] Furthermore, the precise position and posture parameters of the drone camera are iteratively solved through the optimization algorithm to obtain the specific position and posture of the drone at the current time point, including:
[0054] Obtain past location information and camera data from the drone flight records, divide them into training sets and test sets, and calculate the initial posture fitness function value;
[0055] Set the upper and lower limits of the camera pose parameters, determine the dimensions of the pose parameters, and preset the initial population size and maximum number of iterations;
[0056] Initialize the pose parameter population of the drone camera according to the preset population size and pose parameter dimension;
[0057] Generate the reverse solution through geometric information analysis, and calculate the fitness function value of each pose parameter in the initial population and the reverse population;
[0058] Select the posture solution with the best fitness from the initial population and the reverse population as the new initial population, and continuously update the position and value of the best fitness;
[0059] According to the posture update formula, the posture parameter configuration of the drone camera is continuously adjusted to gradually approach the optimal solution;
[0060] Repeat the geometric analysis and optimization until the maximum number of iterations is reached, ending the optimization process;
[0061] The optimal posture parameters obtained in the iterative process are applied to the drone camera model to obtain the specific position and posture of the drone at the current moment.
[0062] Furthermore, the pose update formula is:
[0063] H(n+1)=H(n)|sin(l1)|+l2 sin(l1)U;
[0064] Where H(n+1) represents the position vector of the pose parameter configuration in the next iteration;
[0065] H(n) is the position vector of the pose parameter configuration in the current iteration;
[0066] n is represented by the golden section number;
[0067] l1 is represented by a random number;
[0068] l2 is represented as a random number;
[0069] U is denoted as the adjustment factor;
[0070] H represents the posture parameter configuration.
[0071] The beneficial effects of the present invention are:
[0072] 1. The face data, vehicle data, and license plate information detected by the present invention can be compared with the data in the public security system to screen out key personnel and vehicles, and the intelligent algorithm adopted by the system realizes the autonomous recognition and positioning of multiple mobile targets such as people, vehicles and ships. Compared with the traditional target tracking algorithm, the system has better expression ability for target features, fast speed and high accuracy. The human and vehicle features after the data is stored in the database have broad and urgent application needs for tracking pedestrians under different cameras, as well as in criminal case investigation, vehicle traffic accident handling, vehicle traffic accident hit-and-run, automatic recording of illegal vehicles and other fields.
[0073] 2. The present invention utilizes target tracking technology to obtain the position information of the tracking object to generate control information, which can autonomously control the flight of the UAV. In addition, the real-time position information of the tracking object obtained through target tracking can be used to autonomously control the camera to realize the automatic zoom function of the UAV, while ensuring that the target is always in the center of the picture, thereby realizing the entire automation process of the UAV from target discovery to tracking.
[0074] 3. The present invention can track the target stably for a long time in the dynamic video sequence shot by the drone through the TLD algorithm. By analyzing the video frame by frame, the system can determine whether it is the first frame image and select the appropriate initial target frame. At the same time, the target motion model and state update equation are used for particle filtering to ensure that the target can be continuously tracked when the target moves or the environment changes. The artificial fish school particle filtering technology is used to screen the target area, thereby effectively reducing background interference and making the detection block more accurate.
[0075] 4. The present invention uses an optimization algorithm to repeatedly iterate and adjust the camera pose parameters, gradually approaching the optimal solution. This sophisticated optimization process significantly improves the estimation accuracy of the drone camera pose, ensuring stable operation in a dynamic environment. Using the method of geometric information analysis and reverse solution generation, the system can quickly evaluate and screen the best pose parameters. The optimization algorithm selects the best solution between the initial population and the reverse population, continuously updates and approaches the optimal pose configuration, thereby reducing the cumbersome calculation steps in the traditional method and achieving efficient pose optimization. BRIEF DESCRIPTION OF THE DRAWINGS
[0076] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0077] Figure 1 is a principle block diagram of a target recognition and tracking system based on a drone according to an embodiment of the present invention;
[0078] Figure 2 is a technical block diagram of drone target tracking and real-time detection according to an embodiment of the present invention;
[0079] Figure 3 is a block diagram of a monocular vision positioning algorithm based on a drone according to an embodiment of the present invention;
[0080] Figure 4 4 is a block diagram of a PID closed-loop control technology according to an embodiment of the present invention.
[0081] In the figure:
[0082] 1. Data processing and expansion module; 2. Target detection and tracking algorithm module; 3. Visual positioning and control module; 4. Information sampling and image processing module; 5. Intelligent analysis and data management module. DETAILED DESCRIPTION
[0083] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention and are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in the field should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.
[0084] According to an embodiment of the present invention, a target recognition and tracking system based on a drone is provided.
[0085] The present invention is further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to an embodiment of the present invention, the target recognition and tracking system based on a drone comprises: a data processing and expansion module 1, a target detection and tracking algorithm module 2, a visual positioning and control module 3, an information sampling and image processing module 4, and an intelligent analysis and data management module 5;
[0086] Data processing and expansion module 1 is used to standardize the images (pedestrians, vehicles, license plates, etc.) taken by the drone, and to expand and normalize the original data;
[0087] Object detection and tracking algorithm module 2 is used to detect and track objects based on a lightweight neural network model, introduce a large convolution kernel attention mechanism, and use a bounding box regression loss function;
[0088] Specifically, a neural network that has been trained in practice is used to perform face, vehicle, and license plate recognition detection, and the detected faces, vehicles, and license plates are marked. The system detects and identifies pedestrians, vehicles, and license plates in the video images sent back by the drone. The image quality is evaluated. It can be directly recognized under normal lighting conditions. The backlit or low-light images are first enhanced; the blurred parts of the image are reconstructed with super-resolution to achieve a recognizable effect. For moving targets such as people, vehicles, and ships, autonomous recognition, positioning, and continuous tracking can be achieved. The target will always be in the center of the picture and maintain a suitable ratio for viewing. In specific applications, the target tracking technology is used to obtain the position information of the tracking object and generate control information to autonomously control the flight of the drone. In addition, the real-time position information of the tracking object obtained by target tracking is used to autonomously control the camera to ensure that the target is always in the center of the picture. In summary, the entire drone's entire automation process from target discovery to tracking is realized.
[0089] Visual positioning and control module 3, used to realize real-time positioning and stable flight of the UAV by using monocular visual positioning algorithm and PID closed-loop control technology;
[0090] The information sampling and image processing module 4 is used to perform interval sampling on the video taken by the drone and perform image enhancement preprocessing;
[0091] Specifically, in continuous video frames, several key frames are processed, and the interval sampling method is used to obtain information. In continuous video frames, the effective information contained in the images of adjacent frames is mostly similar. Therefore, in the actual application process, it is generally necessary to downsample the video and directly process the key frames. In this system, the interval sampling method of the video is used to achieve fast and effective information acquisition. In order to better utilize the algorithm for intelligent recognition of image data, the data that may have characteristics such as blur, dimness, and fogging are enhanced and input into the algorithm after processing. This system starts image enhancement preprocessing according to the actual situation of the image data, and sends the preprocessed image data to the algorithm to obtain results with higher accuracy.
[0092] The intelligent analysis and data management module 5 is used to perform semantic analysis on the detected targets and store the structured data into a database.
[0093] Specifically, the system uses video structuring technology to describe people in a structured manner, and provides various structured feature attribute information of pedestrians, including clothing and decorative features: tops, pants, skirts and dresses, shoes, hats, sunglasses, scarves, belts; carrying features: shoulder bags, backpacks, handbags, trolley cases, umbrellas; human features: hair, face. After structuring, typical character feature data can be selected for data storage, and the corresponding knowledge graph can be associated to establish a rich database to accumulate original data for subsequent applications. Re-identification after data storage is a very important task in video surveillance and human-computer interaction. Its main purpose is to match pedestrians captured by different cameras to each other, so as to play a role in tracking and monitoring. Video structuring technology is used to describe vehicles in a structured manner, and various structured feature attribute information of vehicles is provided, including:
[0094] Vehicle type characteristics: sedan, bus, truck, tractor, special transport vehicle, etc.;
[0095] Vehicle color characteristics: red, orange, yellow, green, cyan, blue, purple, etc.;
[0096] Vehicle behavior characteristics: vehicle movement characteristics, such as movement direction, movement speed, etc.;
[0097] Vehicle violation characteristics: incorrect lane change, sudden stop, running a red light, incorrect obstruction of view outside the vehicle, etc.;
[0098] The comprehensive features of each part can effectively detect abnormal behaviors in surveillance videos and reduce traffic accidents. After structuring, typical vehicle data can be selected for storage, and the corresponding knowledge graph can be associated to enrich the database, so as to accumulate original data for subsequent applications. At present, with the rapid growth of car-related and driving-related cases, as well as the continuous deepening of vehicle structuring and smart city applications, it is necessary to extract more diverse vehicle information. In addition to the license plate number, the vehicle's brand, body color, vehicle brand, vehicle type, vehicle features, etc. are also required to support rapid retrieval based on vehicle appearance features, such as: annual inspection labels, pendants, tissue boxes, sun visors, etc. These features have broad and urgent application needs in the fields of criminal case investigation, traffic accident handling, hit-and-run, and automatic recording of illegal vehicles.
[0099] Preferably, the images taken by the drone are standardized, and the raw data are expanded and normalized, including:
[0100] The collection and annotation of drone images are completed by using existing dataset subsets, network collection and manual annotation in turn;
[0101] The image data is cut, noise is removed, color is adjusted, and rotation and translation are performed to expand data diversity;
[0102] The images are processed into a standardized standard data set through mean processing, normalization and size adjustment.
[0103] Preferably, based on a lightweight neural network model, a large convolution kernel attention mechanism is introduced, and a bounding box regression loss function is used for target detection and tracking, including:
[0104] Design a target detection model based on a lightweight neural network model;
[0105] Introducing a large convolution kernel attention mechanism to improve the target detection model’s ability to capture contextual information and enhance the recognition accuracy of target features;
[0106] The bounding box regression loss function is used to optimize the bounding box regression by taking into account the scale, angle and position errors of the object.
[0107] Specifically, based on the lightweight neural network model YOLOX-Tiny, a large convolution kernel attention module is introduced, and the SIoU border regression loss function is added to build the model foundation. The neural network is trained based on the standard data set. The trained network can recognize pedestrians, faces, vehicles and license plates in video images. Figure 2 As shown in the figure, based on the lightweight neural network model YOLOX-Tiny, efficient target detection is achieved; a large convolution kernel attention module is introduced to capture a wider range of contextual information and improve the model's ability to recognize target features; the SIoU bounding box regression loss function is used to further improve the accuracy and robustness of target detection by considering the scale, angle and position errors of the target bounding box, and a stable tracking effect can be maintained even in complex environments and lighting conditions.
[0108] Preferably, the monocular vision positioning algorithm and PID closed-loop control technology are used to achieve real-time positioning and stable flight of the UAV, including:
[0109] The TLD algorithm is used to track the target for a long time to ensure the continuous tracking of the target in a dynamic environment;
[0110] The visible light image of the UAV is used to obtain the image coordinates of the target based on the monocular vision positioning algorithm, and the position of the target in the geographic coordinate system is calculated by combining the UAV flight altitude and camera viewing angle parameters;
[0111] Combined with the gimbal and fuselage status parameters of the drone, a PID closed-loop controller is used to adjust the flight status and gimbal angle to ensure that the target is always in the ideal position.
[0112] Specifically, the model adopts the TLD algorithm in the single-target long-term tracking algorithm, integrating the monocular vision positioning algorithm based on the visible light image of the UAV and the PID closed-loop control technology based on the state parameters of the UAV gimbal and fuselage, to achieve fast and accurate calculation of the target's geographic coordinates, and accurately adjust the flight state and gimbal angle of the UAV so that the target always remains in the ideal position, achieving high-precision target tracking effects in complex environments and dynamic flight conditions, and building a system based on this. The TLD algorithm is currently a relatively advanced algorithm for long-term target tracking. It is divided into the following three modules: tracker, detector, and learner. During the tracking process, the system decomposes the task into different steps for execution. Each independent module is used to perform different tasks. The three modules work simultaneously to complete the target tracking task together; such as Figure 3 As shown in the figure, the monocular visual positioning algorithm based on the visible light image of the drone improves the accuracy and real-time performance of positioning by quickly obtaining the geographic coordinates of the target. Target positioning solves the problem of finding the location of the target of interest in an image. The system uses the positioning in the form of (x, y, width, height). In addition, there are four vertices of the rectangular target to correct the deformation through perspective change later. A deep learning method based on convolutional neural network is used to achieve target positioning. The implementation steps are feature extraction of convolution layer, target regression of full-connected layer, definition of Euclidean distance loss or SIoU border regression loss function, and minimization of loss function. Using the visible light image obtained by the drone camera, the image coordinates of the target are extracted through image processing and feature matching technology. Combined with the flight altitude of the drone and the viewing angle parameters of the camera, the algorithm can calculate the position of the target in the actual geographic coordinate system to meet the needs of real-time positioning.
[0113] Specifically, Figure 4 As shown in the figure, PID closed-loop control technology is used, based on the state parameters of the drone gimbal and fuselage, and the deviation between the coordinates of the target in the image and the ideal position is used to adjust the flight state and gimbal angle of the drone through proportional, integral and differential controllers to ensure that the target always remains in the ideal position, achieve high-precision target tracking effects in complex environments and dynamic flight conditions, and ensure the continuity and stability of data acquisition.
[0114] Preferably, the TLD algorithm is used to track the target for a long time to ensure the continuous tracking of the target in a dynamic environment, including:
[0115] Determine whether the video sequence shot by the drone is the first frame image. If it is the first frame image, manually select the initial target frame. Otherwise, start filtering according to the tracking target motion model and state update equation in the update formula;
[0116] Through artificial fish swarm particle filtering technology, the area where the target is located is screened out and input into the detection block to further confirm the existence of the target;
[0117] If the detection block passes through the variance classifier, random forest classifier and nearest neighbor classifier in sequence, it is determined that the area contains a target. At the same time, the fusion features of the color histogram and directional gradient histogram of the filter area are extracted, the similarity with the initial target is calculated as the particle weight, and the maximum a posteriori probability criterion is used to update the tracking area.
[0118] Specifically, the TLD algorithm (i.e., tracking learning detection algorithm) is an improved TLD algorithm based on artificial fish swarm particle filtering. When faced with tracking difficulties such as illumination changes, partial occlusion, background clutter, and in-plane rotation, the TLD algorithm is prone to drift, resulting in tracking failure, and its tracking performance needs to be improved. Therefore, an artificial fish swarm particle filter tracker is used instead of a pyramid optical flow tracker, and the color histogram features and the directional gradient histogram features are fused to establish a target appearance model, and the image pyramid multi-scale idea is introduced for scale matching to improve the robustness of target tracking. Then, the target area is predicted through the particle filtering process, and the global scan of the TLD algorithm detection block is improved to a local scan, eliminating a large number of non-target areas and improving the detection efficiency of the detection module.
[0119] Preferably, the update formula is:
[0120]
[0121] In the formula, M represents the target state;
[0122] M i Represented as the target state at time i;
[0123] (x, y) represents the center coordinates of the target tracking area;
[0124] k represents the width of the target tracking area;
[0125] h represents the height of the target tracking area;
[0126] α is the scale transformation factor;
[0127] A and B are both expressed as particle update coefficients;
[0128] C represents the external interference term;
[0129] Z represents Gaussian noise.
[0130] Preferably, obtaining the image coordinates of the target through the visible light image of the drone based on a monocular vision positioning algorithm, and calculating the position of the target in the geographic coordinate system in combination with the drone flight altitude and camera viewing angle parameters includes the following steps:
[0131] The forward-view image acquired by the UAV is converted from the image coordinate system to a parallel projection plane;
[0132] If the drone camera has been calibrated, the internal parameters can be used directly to calculate the position; if it has not been calibrated, the internal parameters of the camera are calculated through multi-point correspondence;
[0133] Based on multiple feature points on the ground target object, a geometric reference is constructed, and the geometric features are converted into projections on the image plane under the current viewing angle through affine transformation;
[0134] The linear least squares method is used to estimate the parameters of the affine transformation to calculate the geometric relationship between the drone's view and the ground objects, and the target image coordinates in the current frame are obtained;
[0135] Through geometric information analysis and optimization algorithms, the precise position and posture parameters of the drone camera are iteratively solved to obtain the specific position and posture of the drone at the current time point;
[0136] The solution result of the current frame is used as the initial condition of the next frame, and the relative position of the drone is continuously tracked in combination with the constraint relationship between the previous and next frames.
[0137] Preferably, the linear least squares method is used to estimate the parameters of the affine transformation to calculate the geometric relationship between the drone's perspective and the ground object, and the target image coordinates in the current frame are obtained, including:
[0138] Get the landmark point dataset of the ground target object in the current frame;
[0139] Using all the landmark data, the linear least square method is used to estimate the parameters of the preliminary affine transformation to obtain the initial affine transformation parameters;
[0140] In the feature point data set, two groups of landmark points are removed pair by pair, and the linear least squares method is reapplied using the remaining landmark point data to calculate new affine transformation parameters;
[0141] After removing two sets of landmarks each time, calculate the difference between the newly obtained affine transformation parameters and the initial estimate, and record the difference value calculated each time;
[0142] The process of removing landmarks and calculating differences is repeated until the preset maximum number of iterations is reached;
[0143] Analyze all recorded difference values and select a set of affine transformation parameters with the smallest difference as the final estimate to ensure the best affine transformation model. Use the affine transformation model to calculate the geometric relationship between the drone's perspective and ground objects to obtain the target image coordinates in the current frame.
[0144] Preferably, the precise position and posture parameters of the drone camera are iteratively solved by an optimization algorithm to obtain the specific position and posture of the drone at the current time point, including:
[0145] Obtain past location information and camera data from the drone flight records, divide them into training sets and test sets, and calculate the initial posture fitness function value;
[0146] Set the upper and lower limits of the camera pose parameters, determine the dimensions of the pose parameters, and preset the initial population size and maximum number of iterations;
[0147] Initialize the pose parameter population of the drone camera according to the preset population size and pose parameter dimension;
[0148] Generate a reverse solution through geometric information analysis (i.e., elite reverse learning strategy), and calculate the fitness function value of each pose parameter in the initial population and the reverse population;
[0149] Specifically, the elite reverse learning strategy is a technique commonly used in optimization algorithms, especially in genetic algorithms and other evolutionary algorithms. The purpose of this strategy is to enhance the algorithm's exploration ability and use the information of the current solution to improve the efficiency of future searches. The elite reverse learning strategy relies on selecting the best performing solutions (i.e., elite solutions) from the current population and using these solutions as the basis for generating new solutions. The "reverse" part of the strategy involves reversing certain parameters of the elite solution, that is, reversing or negating the values of these parameters to explore new areas in the solution space.
[0150] Select the posture solution with the best fitness from the initial population and the reverse population as the new initial population, and continuously update the position and value of the best fitness;
[0151] According to the posture update formula, the posture parameter configuration of the drone camera is continuously adjusted to gradually approach the optimal solution;
[0152] Specifically, the posture update formula is the core process of the golden sine algorithm and the update process of its solution. The golden sine algorithm introduces the golden section number based on the relationship between the sine function and the unit circle to achieve a good balance between search and development.
[0153] Repeat the geometric analysis and optimization until the maximum number of iterations is reached, ending the optimization process;
[0154] The optimal posture parameters obtained in the iterative process are applied to the drone camera model to obtain the specific position and posture of the drone at the current moment.
[0155] Specifically, the optimization algorithm is an improved whale optimization algorithm. The improved whale optimization algorithm proposed in the present invention retains the encirclement hunting method and the random prey search method of the traditional whale algorithm, and uses the golden sine algorithm to improve the spiral ascending hunting method of the whale. The improved position update method, such as the position update formula, introduces the golden section number. In each iteration, the position information is exchanged with the prey (the current optimal solution), and the direction and distance of the individual movement are controlled by parameters, so that the individual whale approaches the direction of the prey (the current optimal solution) faster, which accelerates the convergence speed of the algorithm and the accuracy of optimization.
[0156] Preferably, the posture update formula is:
[0157] H(n+1)=H(n)|sin(l1)|+l2 sin(l1)U;
[0158] Where H(n+1) represents the position vector of the pose parameter configuration in the next iteration;
[0159] H(n) is the position vector of the pose parameter configuration in the current iteration;
[0160] n is represented by the golden section number;
[0161] l1 is represented by a random number;
[0162] l2 is represented as a random number;
[0163] U is denoted as the adjustment factor;
[0164] H represents the posture parameter configuration.
[0165] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A target recognition and tracking system based on drone, characterized in that: The target recognition and tracking system based on UAV includes: data processing and expansion module, target detection and tracking algorithm module, visual positioning and control module, information sampling and image processing module, intelligent analysis and data management module; The data processing and expansion module is used to standardize the images taken by the drone and to expand and normalize the original data; The target detection and tracking algorithm module is used to detect and track targets based on a lightweight neural network model, introduce a large convolution kernel attention mechanism, and use a bounding box regression loss function; The visual positioning and control module is used to use the monocular visual positioning algorithm and PID closed-loop control technology to achieve real-time positioning and stable flight of the UAV; The information sampling and image processing module is used to sample the video taken by the drone at intervals and perform image enhancement preprocessing; The intelligent analysis and data management module is used to perform semantic analysis on detected targets and store structured data in the database.
2. The target recognition and tracking system based on drone according to claim 1 is characterized in that: The standardization of the images taken by the drone and the data expansion and normalization of the original data include: The collection and annotation of drone images are completed by using existing dataset subsets, network collection and manual annotation in turn; The image data is cut, noise is removed, color is adjusted, and rotation and translation are performed to expand data diversity; The images are processed into a standardized standard data set through mean processing, normalization and size adjustment.
3. The target recognition and tracking system based on drone according to claim 1 is characterized in that: The lightweight neural network model introduces a large convolution kernel attention mechanism and uses a bounding box regression loss function for target detection and tracking, including: Design a target detection model based on a lightweight neural network model; Introducing a large convolution kernel attention mechanism to improve the target detection model’s ability to capture contextual information and enhance the recognition accuracy of target features; The bounding box regression loss function is used to optimize the bounding box regression by taking into account the scale, angle and position errors of the object.
4. The target recognition and tracking system based on drone according to claim 1 is characterized in that: The use of monocular vision positioning algorithm and PID closed-loop control technology to achieve real-time positioning and stable flight of the drone includes: The TLD algorithm is used to track the target for a long time to ensure the continuous tracking of the target in a dynamic environment; The visible light image of the UAV is used to obtain the image coordinates of the target based on the monocular vision positioning algorithm, and the position of the target in the geographic coordinate system is calculated by combining the UAV flight altitude and camera viewing angle parameters; Combined with the gimbal and fuselage status parameters of the drone, a PID closed-loop controller is used to adjust the flight status and gimbal angle to ensure that the target is always in the ideal position.
5. The target recognition and tracking system based on UAV according to claim 4 is characterized in that: The long-term tracking of the target using the TLD algorithm to ensure the continuous tracking of the target in a dynamic environment includes: Determine whether the video sequence shot by the drone is the first frame image. If it is the first frame image, manually select the initial target frame. Otherwise, start filtering according to the tracking target motion model and state update equation in the update formula; Through artificial fish swarm particle filtering technology, the area where the target is located is screened out and input into the detection block to further confirm the existence of the target; If the detection block passes through the variance classifier, random forest classifier and nearest neighbor classifier in sequence, it is determined that the area contains a target. At the same time, the fusion features of the color histogram and directional gradient histogram of the filter area are extracted, the similarity with the initial target is calculated as the particle weight, and the maximum a posteriori probability criterion is used to update the tracking area.
6. The target recognition and tracking system based on drone according to claim 5, characterized in that: The update formula is: In the formula, M represents the target state; M i Represented as the target state at time i; (x, y) represents the center coordinates of the target tracking area; k represents the width of the target tracking area; h represents the height of the target tracking area; α is the scale transformation factor; A and B are both expressed as particle update coefficients; C represents the external interference term; Z represents Gaussian noise.
7. The target recognition and tracking system based on UAV according to claim 4 is characterized in that: The method of obtaining the image coordinates of the target through the visible light image of the drone based on the monocular vision positioning algorithm, and calculating the position of the target in the geographic coordinate system in combination with the flight altitude of the drone and the camera viewing angle parameters includes the following steps: The forward-view image acquired by the UAV is converted from the image coordinate system to a parallel projection plane; If the drone camera has been calibrated, the internal parameters can be used directly to calculate the position; if it has not been calibrated, the internal parameters of the camera can be calculated through multi-point correspondence; Based on multiple feature points on the ground target object, a geometric reference is constructed, and the geometric features are converted into projections on the image plane under the current viewing angle through affine transformation; The linear least squares method is used to estimate the parameters of the affine transformation to calculate the geometric relationship between the drone's view and the ground objects, and the target image coordinates in the current frame are obtained; Through geometric information analysis and optimization algorithms, the precise position and posture parameters of the drone camera are iteratively solved to obtain the specific position and posture of the drone at the current time point; The solution result of the current frame is used as the initial condition of the next frame, and the relative position of the drone is continuously tracked in combination with the constraint relationship between the previous and next frames.
8. The target recognition and tracking system based on unmanned aerial vehicle according to claim 7, characterized in that: The linear least squares method is used to estimate the parameters of the affine transformation to calculate the geometric relationship between the drone's perspective and the ground object, and the target image coordinates in the current frame are obtained: Get the landmark point dataset of the ground target object in the current frame; Using all the landmark data, the linear least square method is used to estimate the parameters of the preliminary affine transformation to obtain the initial affine transformation parameters; In the feature point data set, two groups of landmark points are removed pair by pair, and the linear least squares method is reapplied using the remaining landmark point data to calculate new affine transformation parameters; After removing two sets of landmarks each time, calculate the difference between the newly obtained affine transformation parameters and the initial estimate, and record the difference value calculated each time; The process of removing landmarks and calculating differences is repeated until the preset maximum number of iterations is reached; Analyze all recorded difference values and select a set of affine transformation parameters with the smallest difference as the final estimate to ensure the best affine transformation model. Use the affine transformation model to calculate the geometric relationship between the drone's perspective and ground objects to obtain the target image coordinates in the current frame.
9. The target recognition and tracking system based on UAV according to claim 8, characterized in that: The method of iteratively solving the precise position and posture parameters of the drone camera through the optimization algorithm to obtain the specific position and posture of the drone at the current time point includes: Obtain past location information and camera data from the drone flight records, divide them into training sets and test sets, and calculate the initial posture fitness function value; Set the upper and lower limits of the camera pose parameters, determine the dimensions of the pose parameters, and preset the initial population size and maximum number of iterations; Initialize the pose parameter population of the drone camera according to the preset population size and pose parameter dimension; Generate the reverse solution through geometric information analysis, and calculate the fitness function value of each pose parameter in the initial population and the reverse population; Select the posture solution with the best fitness from the initial population and the reverse population as the new initial population, and continuously update the position and value of the best fitness; According to the posture update formula, the posture parameter configuration of the drone camera is continuously adjusted to gradually approach the optimal solution; Repeat the geometric analysis and optimization until the maximum number of iterations is reached, ending the optimization process; The optimal posture parameters obtained in the iterative process are applied to the drone camera model to obtain the specific position and posture of the drone at the current moment.
10. The target recognition and tracking system based on UAV according to claim 9, characterized in that: The posture update formula is: H(n+1)=H(n)|sin(l1)|+l2 sin(l1)U; Where H(n+1) represents the position vector of the pose parameter configuration in the next iteration; H(n) is the position vector of the pose parameter configuration in the current iteration; n is represented by the golden section number; l1 is represented by a random number; l2 is represented as a random number; U is denoted as the adjustment factor; H represents the posture parameter configuration.
Citation Information
Patent Citations
Low-spatial-resolution multisource remote sensing image space consistency correction method
CN103413272A
Matching auxiliary visual target tracking method and device, electronic equipment and storage medium
CN115222776A
Target real-time tracking method and system in unmanned aerial vehicle scene
CN117994288A
Cited By
GIS-based plateau airport RNP AR departure and emergency route planning method and system, and storage medium
CN119296385A
Flight control method and system of unmanned aerial vehicle tracking technology
CN119311025A
A flight control method and system for unmanned aerial vehicle tracking technology
CN119311025B
Multi-feature fused variable-scale moving target positioning and tracking method and device
CN119380044A
Multifunctional automatic space point location tracking measurement method and device
CN120043445A
