Target detection system in unmanned aerial vehicle aerial photography scene

Abstract

The invention relates to the technical field of target detection, and discloses a target detection system in an aerial scene of an unmanned aerial vehicle. According to the system, in a target image acquisition process, an advanced sensor carried by an unmanned aerial vehicle is combined to ensure that a high-quality aerial image is acquired, in a target image preprocessing stage, median filtering and image enhancement technologies are adopted to effectively improve the definition and availability of the image, and a target detection unit loads a high-efficiency detection algorithm to improve the detection accuracy of the target. The target can be accurately identified and a bounding box can be generated, and low-confidence detection can be removed based on a confidence threshold screening result, so that the accuracy and reliability of overall detection are improved, and the target tracking unit records the motion trail of the target in real time, so that a user can comprehensively know the dynamic state of the target, and the user experience is improved. And the display unit displays detection and tracking results in a visual manner, so that the operation experience and the data understanding ability of the user are improved, and the practical value and the application effect of the aerial photography data of the unmanned aerial vehicle are further improved.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a target detection system in an unmanned aerial vehicle (UAV) aerial photography scene. Background Art

[0002] In recent years, with the rapid development of drone technology, aerial photography technology has been widely used. Drones equipped with high-resolution cameras and a variety of sensors (such as GPS and IMU) can perform real-time high-definition image acquisition and are widely used in agriculture, environmental monitoring, post-disaster assessment and other fields. With the help of advanced technologies such as computer vision and deep learning, target detection systems can automatically identify and locate specific targets from images collected by drones. This is of great significance for improving data acquisition efficiency and accuracy, especially in complex and dynamic environments. Through efficient target detection and tracking, real-time monitoring and decision support can be achieved, providing innovative solutions for various industries.

[0003] Although existing drone target detection technology has achieved certain results in many application scenarios, it does not effectively remove low-confidence image data when collecting aerial target images, which reduces the overall accuracy and reliability of target detection. Summary of the Invention

[0004] (1) Technical problems solved

[0005] In response to the shortcomings of the existing technology, the present invention provides a target detection system in a drone aerial photography scene. By integrating a target image acquisition unit, a preprocessing unit, a detection unit, a tracking unit and a display unit, efficient target recognition and tracking functions are realized. During the target image acquisition process, the advanced sensors carried by the drone are combined to ensure the acquisition of high-quality aerial images. In the target image preprocessing stage, median filtering and image enhancement technology are used to effectively improve the clarity and usability of the image. By loading an efficient detection algorithm, the target detection unit can not only accurately identify the target and generate a bounding box, but also filter the results based on the confidence threshold and remove low-confidence detections, thereby improving the accuracy and reliability of the overall detection. The target tracking unit records the target's motion trajectory in real time, allowing the user to fully understand the target dynamics, while the display unit displays the detection and tracking results in a visual manner, improving the user's operating experience and data comprehension ability, ensuring that the user obtains high-quality detection results, thereby improving the practical value and application effect of the drone aerial photography data, and solving the above-mentioned problems.

[0006] (2) Technical solution

[0007] To achieve the above-mentioned object, the present invention provides the following technical solutions: a target detection system in a drone aerial photography scene, comprising a target image acquisition unit, a target image preprocessing unit, a target detection unit, a target tracking unit, and a display unit;

[0008] The target image acquisition unit acquires the target image taken by the drone through the camera, GPS, and IMU sensor carried by the drone, and sends the target image taken by the drone to the target image preprocessing unit;

[0009] The target image preprocessing unit performs median filtering, image enhancement, image data format conversion and image calibration on the target image taken by the drone, and then inputs the image into the target detection unit;

[0010] The target detection unit performs target detection on the pre-processed target image taken by the drone by loading the target detection algorithm, identifies the target in the target image taken by the drone, generates a bounding box for the target, calculates the target's coordinates, target category label and target confidence score, and filters the detected targets based on the confidence threshold to remove low-confidence results;

[0011] The target tracking unit tracks the detected target, calculates and records the updated target position and target motion trajectory;

[0012] The display unit displays the results of target detection and target tracking in the form of a visual map.

[0013] Preferably, the formula for performing median filtering on the target image photographed by the drone is as follows:

[0014] Z(x,y)=median{I(x i ,y j )|(x i ,y j )∈N(x,y)}

[0015] In the formula, Z(x,y) represents the pixel value of the target image taken by the drone at the coordinate (x,y) after median filtering, and I(x i ,y i ) represents the pixel value of the target image taken by the input drone in the window N, N(x,y) represents the neighborhood area centered on (x,y), which is a 3*3 window, and median represents the median in the neighborhood.

[0016] Preferably, the image enhancement formula is as follows:

[0017]

[0018] In the formula, H(i) represents the enhanced histogram value, j represents the image pixel index, δ represents the Diktak function, M represents the sum of the pixels in the image, and I(j) represents the image pixel value. It means to accumulate and sum all image pixel indices j from 0 to M-1, and δ(I(j)-i) means that each summation only counts the number of pixels with pixel value i.

[0019] Preferably, the formula for converting the image data format is as follows:

[0020]

[0021] In the formula, I' represents the pixel value of the image after image conversion, I represents the pixel value of the original image, M represents the sum of the pixels in the image, and N represents the total number of pixels in the input image. It means adding up all pixel values ​​and then dividing by the total number of pixels to get the average value of the image. The overall meaning of the formula is to standardize each pixel value to get an image with a mean of 0 and a standard deviation of 1.

[0022] Preferably, the image calibration formula is as follows:

[0023]

[0024] In the formula, u and v represent the pixel coordinates of the drone aerial image on the image plane, K represents the camera intrinsic parameter matrix, R represents the rotation matrix, t represents the translation vector, X, Y, and Z represent the world coordinates of the object in the drone aerial image, and 1 represents the reference value.

[0025] Preferably, the target coordinate calculation formula is as follows:

[0026]

[0027] In the formula, Center(x,y) represents the target center coordinates, (x min ,y min ) represents the coordinate of the upper left corner of the target bounding box, (x max ,y max ) represents the coordinate of the lower right corner of the target bounding box.

[0028] Preferably, the calculation formula of the target category label is as follows:

[0029] Cm=argmax(Softmax(z))

[0030] In the formula, Cm represents the predicted target category label, z represents the unadjusted original aerial image pixel value, Softmax(z) represents the function that converts z into a probability distribution, and argmax(Softmax(z)) represents the selection of the category with the highest probability as the predicted target category label result.

[0031] Preferably, the calculation formula of the target confidence score is as follows:

[0032]

[0033] In the formula, p represents the confidence score of category i, z i represents the original image pixel value of category i, Indicates taking the index of the original image pixel value of category i, It means taking the exponents of the original image pixel values ​​from 1 to Cm for all categories j and summing them up to get the probability distribution of all categories.

[0034] Preferably, the updated target position calculation formula is as follows:

[0035] x k =A*x k-1 +B*u k

[0036] In the formula, x k represents the target position state prediction at the current time k, A represents the state transfer matrix, x k-1 represents the target position state at the previous moment, B represents the input control matrix, u k Indicates acceleration.

[0037] Preferably, the target motion trajectory calculation formula is as follows:

[0038] Trajectory={(x k ,y k )|k=1,2,......,N}

[0039] In the formula, Trajectroy means storing the trajectory information of the target at different time points, (x k ,y k ) represents the coordinates of the target at the kth moment, and k ranges from 1 to N.

[0040] Compared with the existing technology, the present invention provides a target detection system in drone aerial photography scenes, which has the following beneficial effects:

[0041] The present invention realizes efficient target recognition and tracking functions by integrating a target image acquisition unit, a preprocessing unit, a detection unit, a tracking unit and a display unit. During the target image acquisition process, the advanced sensors carried by the drone are combined to ensure the acquisition of high-quality aerial images. In the target image preprocessing stage, median filtering and image enhancement technology are used to effectively improve the clarity and usability of the image. The target detection unit can not only accurately identify the target and generate a bounding box by loading an efficient detection algorithm, but also filter the results based on the confidence threshold and remove low-confidence detections, thereby improving the accuracy and reliability of the overall detection. The target tracking unit records the target's motion trajectory in real time, allowing the user to fully understand the target dynamics, while the display unit displays the detection and tracking results in a visual manner, improving the user's operating experience and data comprehension ability, ensuring that the user obtains high-quality detection results, and thereby improving the practical value and application effect of drone aerial photography data. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of the system flow of the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0044] In order to solve the problem that low-confidence image data is not effectively removed when collecting aerial target images, which reduces the overall accuracy and reliability of target detection, a target detection system in UAV aerial photography scenes is proposed. Figure 1 ,The system includes a target image acquisition unit, a target image preprocessing unit, a target detection unit, a target tracking unit and a display unit;

[0045] The target image acquisition unit relies on the drone's high-resolution camera, global positioning system (GPS), and inertial measurement unit (IMU) to achieve full-scale target image acquisition of the aerial scene. The system uses a high-performance camera to capture high-definition images under different lighting and weather conditions. Combined with the GPS module, it accurately records the location information of each image for subsequent analysis and processing. At the same time, the IMU sensor monitors the drone's attitude, speed, and direction of movement in real time to ensure the stability and consistency of image acquisition during flight. Through these technical means, the system can effectively reduce image blur caused by flight turbulence or changes in the external environment, ensuring that the acquired target images maintain high quality and clarity.

[0046] The target image preprocessing unit performs median filtering, image enhancement, image data format conversion, and image calibration on the target image taken by the UAV, including:

[0047] The formula for median filtering of the target image taken by the UAV is as follows:

[0048] Z(x,y)=median{I(x i ,y j )|(x i ,y j )∈N(x,y)}

[0049] Median filtering is an effective image denoising technology that aims to reduce noise in images. In drone aerial photography scenes, due to rapid movement, environmental interference and sensor limitations, images may be affected by noise. Median filtering can retain the edge information of the image while effectively removing random noise, thereby improving the clarity and recognizability of the image, which is crucial for subsequent target detection. In the formula, Z(x,y) represents the pixel value of the target image of the drone aerial photography after median filtering at the coordinate (x,y), and I(x i ,y j ) represents the pixel value of the target image taken by the input drone in the window N, N(x,y) represents the neighborhood area centered on (x,y), which is a 3*3 window, and median represents the median in the neighborhood;

[0050] The formula for image enhancement is as follows:

[0051]

[0052] The purpose of introducing image enhancement technology is to improve the visual effect of the image and make the target more obvious in the complex background. By applying the histogram equalization method, image enhancement can significantly improve the visualization of the target and improve the accuracy of the target detection algorithm in identifying and locating the target. In the formula, H(i) represents the enhanced histogram value, j represents the image pixel index, δ represents the Dirkat function, M represents the sum of the pixels in the image, and I(j) represents the image pixel value. Indicates the cumulative summation of all image pixel indices j from 0 to M-1, and δ(I(j)-i) means that each summation only counts the number of pixels with pixel value i;

[0053] The formula for image data format conversion is as follows:

[0054]

[0055] Image data format conversion is to ensure data compatibility and processing efficiency between components. Different image processing algorithms and models may have specific requirements for data formats. Therefore, converting the image to a format suitable for the target detection unit (such as converting from RGB to grayscale or downsampling) can reduce computing consumption, improve processing speed, and ensure the smooth operation of the entire system in real-time detection. In the formula, I' represents the image pixel value after image conversion, I represents the original image pixel value, M represents the sum of pixels in the image, and N represents the total number of pixels in the input image. It means adding up all pixel values ​​and dividing by the total number of pixels to get the average value of the image. The overall meaning of the formula is to standardize each pixel value to get an image with a mean of 0 and a standard deviation of 1.

[0056] The formula for image calibration is as follows:

[0057]

[0058] Image calibration helps improve the geometric accuracy of the system. The calibration process corrects the internal and external parameters of the camera to eliminate errors caused by lens distortion and sensor inaccuracies, thereby making the spatial coordinates of the target more accurate. This accuracy is particularly important for applications that require high reliability and accurate positioning (such as monitoring the motion trajectory of a specific target). In the formula, u and v represent the pixel coordinates of the drone aerial image on the image plane, K represents the camera intrinsic parameter matrix, R represents the rotation matrix, t represents the translation vector, X, Y, and Z represent the world coordinates of the object in the drone aerial image, and 1 represents the reference value;

[0059] The target detection unit loads the YOLO detection model, performs feature extraction, classification, and positioning on the input image, accurately identifies the target in the image, and generates a corresponding bounding box for each target, marking its position in the image. At the same time, the unit also calculates the coordinates, category label, and confidence score of each detected target. The confidence score reflects the model's confidence in the target classification. To ensure the reliability of the detection results, the detection results are screened based on the set confidence threshold. In this process, the confidence threshold is set to 0.5, which means that only when the confidence score of a target is higher than 0.5, the target is considered a valid detection result. This screening process not only improves the accuracy of the final detection result, but also effectively reduces false positives, thereby maintaining efficient target detection performance in complex environments.

[0060] The target coordinate calculation formula is as follows:

[0061]

[0062] By obtaining the accurate coordinates of the target, drones can perform real-time monitoring more efficiently, such as monitoring criminals in the security field or tracking packages in logistics transportation. In the formula, Center(x,y) represents the coordinates of the target center, (x min ,y min ) represents the coordinate of the upper left corner of the target bounding box, (x max ,y max ) represents the coordinates of the lower right corner of the target bounding box. By calculating the target coordinates, synthetic data is provided so that geographic information systems (GIS) can be combined for geographic analysis, such as assessing the impact of a specific area in environmental monitoring;

[0063] The target category label is calculated as follows:

[0064] Cm=argmax(Softmax(z))

[0065] By calculating the target category label, it is possible to intelligently classify different targets according to their categories (such as animals, vehicles, buildings, etc.), thereby providing valuable information for subsequent processing and decision-making. In the formula, Cm represents the predicted target category label, z represents the unadjusted original aerial image pixel value, Softmax(z) represents the function that converts z into a probability distribution, and argmax(Softmax(z)) represents the selection of the category with the highest probability as the predicted target category label result. When performing specific tasks (such as disaster relief or plant monitoring), adopting different response strategies for different categories of targets can improve the efficiency and effectiveness of the action;

[0066] The target confidence score is calculated as follows:

[0067]

[0068] The confidence score reflects the system's credibility in the recognition results. Setting a threshold based on the confidence score to filter activity results can remove low-confidence detections, reduce false positives and false negatives, and enhance the overall accuracy of the system. In the formula, p represents the confidence score of category i, z i represents the original image pixel value of category i, Indicates taking the index of the original image pixel value of category i, It means taking the exponents of the original image pixel values ​​from 1 to Cm for all categories j and summing them up to obtain the probability distribution of all categories. By calculating the confidence score, it provides a reference for subsequent data analysis and decision-making process, helping operators to assess the urgency and priority of the situation.

[0069] The target tracking unit uses the detected target position information and confidence score to identify and lock the target in each frame of the image. It processes the image data in real time through algorithms and maintains stable tracking of the target in complex backgrounds. It ensures that even when the target moves quickly or is partially obscured, its motion state can still be accurately recorded. The updated target position not only includes 2D plane coordinates (such as x and y coordinates in the image), but can also be converted into an actual geographic coordinate system in combination with the GPS data of the drone. The calculation and recording function of the target motion trajectory enables the system to analyze the target's behavior pattern and generate a complete motion path by tracking the target's position changes at different time points. This trajectory information is of great significance for subsequent behavior analysis, trend prediction and intelligent decision-making, among which:

[0070] The updated target position calculation formula is as follows:

[0071] x k =A*x k-1 +B*u k

[0072] By updating the target position, the system can analyze the target's dynamic behavior, help detect abnormal activities or pattern changes, and improve early warning capabilities. In the formula, x k represents the target position state prediction at the current time k, A represents the state transfer matrix, x k-1 represents the target position state at the previous moment, B represents the input control matrix, u k Expressing acceleration, the calculation of target position provides real-time data support for autonomous navigation of UAVs in changing environments, improving tracking accuracy and efficiency, especially in complex terrain or interference conditions;

[0073] The target motion trajectory calculation formula is as follows:

[0074] Trajectoroy={(x k ,y k )|k=1,2,......,N}

[0075] By analyzing the target's motion trajectory, we can predict the target's future position and behavior, and achieve more efficient dynamic obstacle avoidance and task planning. In the formula, Trajectory represents the storage of the target's trajectory information at different time points, (x k ,y k ) represents the coordinates of the target at the kth moment, where k ranges from 1 to N. Continuously tracking the target's trajectory helps identify its behavior pattern and implement differentiated management for different targets. This has important application value in security monitoring, ecological research, and other fields.

[0076] The display unit presents the results of target detection and tracking to the user in the form of an intuitive visual map. Using graphical user interface (GUI) technology, the unit overlays target information identified by the target detection unit, such as the target's bounding box, category label, and confidence score, on real-time aerial images. This combination of image and data not only enhances the user's understanding of the detection content but also provides immediate feedback, allowing operators to formulate appropriate response strategies based on the visual information.

[0077] In the visual map display, the target's location and movement trajectory will be marked and updated in real time based on a geographic information system (GIS). Different categories of targets can be identified using different colors or graphic symbols, allowing users to quickly identify the distribution of various types of targets, using red circles to represent vehicles, green circles to represent pedestrians, etc. This differentiated visual presentation allows monitors to more effectively capture and analyze target behavior, especially in complex operating environments.

[0078] Through the application of the above system, the accuracy and reliability of the overall detection are effectively improved, ensuring that users obtain high-quality detection results, thereby enhancing the practical value and application effect of drone aerial photography data.

[0079] Example 1:

[0080] In this experiment, in order to improve the accuracy and reliability of drone aerial photography target detection, the target confidence score was verified. When photographing the target, three target categories were collected, and the corresponding logits values ​​are as follows:

[0081] Class 1 logits z1 = 1.0, class 2 logits z2 = 2.0, class 3 logits z3 = 2.0, then calculate the index value of each class:

[0082] Then calculate the sum of the index values ​​of all categories:

[0083]

[0084] Now we can calculate the confidence score p for each class i :

[0085] For category 1, For category 2, For category 3,

[0086] According to the confidence scores calculated above, the confidence score of category 1 is 0.155, which is less than the confidence threshold of 0.5. The confidence score of category 2 is 0.422, which is less than the confidence threshold of 0.5. The confidence score of category 3 is 0.422, which is less than the confidence threshold of 0.5. This means that the detection results of all three categories are unacceptable. At this time, it is necessary to check whether there is an imbalance or labeling problem in the aerial photography target dataset. More representative target image samples may be needed.

[0087] Example 2:

[0088] In this experiment, in order to improve the accuracy and reliability of drone aerial photography target detection, the target confidence score was verified. When photographing the target, three target categories were collected, and the corresponding logits values ​​are as follows:

[0089] Class 1 logits z1 = 2.0, class 2 logits z2 = 1.0, class 3 logits z3 = 0.5. Next, calculate the index value of each class:

[0090] Then calculate the sum of the index values ​​of all categories:

[0091]

[0092] Now we can calculate the confidence score p for each class i :

[0093] For category 1, For category 2, For category 3,

[0094] According to the confidence scores calculated above, the confidence score of category 1 is 0.629, which is greater than the confidence threshold of 0.5; the confidence score of category 2 is 0.231, which is less than the confidence threshold of 0.5; and the confidence score of category 3 is 0.140, which is less than the confidence threshold of 0.5. This means that only the detection results of category 1 are accepted, while categories 2 and 3 are excluded because their confidence scores are lower than the set threshold. This process ensures that only high-confidence detection results will affect the subsequent target tracking and decision-making process, thereby improving the accuracy and reliability of the UAV target detection system.

[0095] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A target detection system in a drone aerial photography scene, characterized by: It includes a target image acquisition unit, a target image preprocessing unit, a target detection unit, a target tracking unit and a display unit; The target image acquisition unit acquires the target image taken by the drone through the camera, GPS, and IMU sensor carried by the drone, and sends the target image taken by the drone to the target image preprocessing unit; The target image preprocessing unit performs median filtering, image enhancement, image data format conversion and image calibration on the target image taken by the drone, and then inputs the image into the target detection unit; The target detection unit performs target detection on the pre-processed target image taken by the drone by loading the target detection algorithm, identifies the target in the target image taken by the drone, generates a bounding box for the target, calculates the target's coordinates, target category label and target confidence score, and filters the detected targets based on the confidence threshold to remove low-confidence results; The target tracking unit tracks the detected target, calculates and records the updated target position and target motion trajectory; The display unit displays the results of target detection and target tracking in the form of a visual map.

2. The target detection system in a drone aerial photography scene according to claim 1, characterized in that: The formula for performing median filtering on the target image taken by the drone is as follows: Z(x,y)=median{I(x i ,and j )|(x i ,and j )∈N(x,y)} In the formula, Z(x,y) represents the pixel value of the target image taken by the drone at the coordinate (x,y) after median filtering, and I(x i ,y j ) represents the pixel value of the target image taken by the input drone in the window N, N(x,y) represents the neighborhood area centered on (x,y), which is a 3*3 window, and median represents the median in the neighborhood.

3. The target detection system in a drone aerial photography scene according to claim 2, characterized in that: The formula for image enhancement is as follows: In the formula, H(i) represents the enhanced histogram value, j represents the image pixel index, δ represents the Diktak function, M represents the sum of the pixels in the image, and I(j) represents the image pixel value. It means to accumulate and sum all image pixel indices j from 0 to M-1, and δ(I(j)-i) means that each summation only counts the number of pixels with pixel value i.

4. The target detection system in a drone aerial photography scene according to claim 3, characterized in that: The formula for image data format conversion is as follows: In the formula, I' represents the pixel value of the image after image conversion, I represents the pixel value of the original image, M represents the sum of the pixels in the image, and N represents the total number of pixels in the input image. It means adding up all pixel values ​​and then dividing by the total number of pixels to get the average value of the image. The overall meaning of the formula is to standardize each pixel value to get an image with a mean of 0 and a standard deviation of 1.

5. The target detection system in a drone aerial photography scene according to claim 4, characterized in that: The image calibration formula is as follows: In the formula, u and v represent the pixel coordinates of the drone aerial image on the image plane, K represents the camera intrinsic parameter matrix, R represents the rotation matrix, t represents the translation vector, X, Y, and Z represent the world coordinates of the object in the drone aerial image, and 1 represents the reference value.

6. The target detection system in a drone aerial photography scene according to claim 5, characterized in that: The target coordinate calculation formula is as follows: In the formula, Center(x,y) represents the target center coordinates, (x min ,y min ) represents the coordinate of the upper left corner of the target bounding box, (x max ,y max ) represents the coordinate of the lower right corner of the target bounding box.

7. The target detection system in a drone aerial photography scene according to claim 6, characterized in that: The calculation formula of the target category label is as follows: Cm=argmax(Softmax(z)) In the formula, Cm represents the predicted target category label, z represents the unadjusted original aerial image pixel value, Softmax(z) represents the function that converts z into a probability distribution, and argmax(Softmax(z)) represents the selection of the category with the highest probability as the predicted target category label result.

8. The target detection system in a drone aerial photography scene according to claim 7, characterized in that: The calculation formula of the target confidence score is as follows: In the formula, p represents the confidence score of category i, z i represents the original image pixel value of category i, Indicates taking the index of the original image pixel value of category i, It means taking the exponents of the original image pixel values ​​from 1 to Cm for all categories j and summing them up to get the probability distribution of all categories.

9. The target detection system in a drone aerial photography scene according to claim 8, characterized in that: The updated target position calculation formula is as follows: x k =A*x k-1 +B*u k In the formula, x k represents the target position state prediction at the current time k, A represents the state transfer matrix, x k-1 represents the target position state at the previous moment, B represents the input control matrix, u k Indicates acceleration.

10. The target detection system in a drone aerial photography scene according to claim 9, characterized in that: The target motion trajectory calculation formula is as follows: Trajectory={(x k ,y k )|k=1,2,......,N} In the formula, Trajectory means storing the trajectory information of the target at different time points, (x k ,y k ) represents the coordinates of the target at the kth moment, and k ranges from 1 to N.

Images