Multi-unmanned aerial vehicle multi-view collaborative target detection and discrimination method based on target information

Abstract

The invention relates to a multi-unmanned aerial vehicle multi-view collaborative target detection and discrimination method based on target information, belongs to the technical field of unmanned aerial vehicle detection, and solves the problems that the target detection accuracy is reduced and missing detection and false detection are easy to occur in a complex environment in the prior art. The method comprises the following steps: controlling a plurality of unmanned aerial vehicle nacelles to simultaneously obtain images of the same target to form an image set; obtaining an original bounding box corresponding to a target in each image and attribute information of the bounding box based on a target detection model; based on the internal reference of the pod camera of the unmanned aerial vehicle and the position data of the unmanned aerial vehicle, converting the original bounding box to obtain a real bounding box; converting all the real bounding boxes into the same map to obtain bounding boxes of all the images in the same map; based on the attribute information of the bounding boxes, performing cyclic redundancy removal processing on all the converted bounding boxes to obtain the bounding boxes after cyclic redundancy removal processing; and detecting and judging the target based on the bounding box after the cyclic redundancy removal processing.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) detection technology, and in particular to a multi-UAV, multi-view collaborative target detection and discrimination method based on target information. Background Technology

[0002] With the development of UAV technology, UAV aerial image target detection technology has been widely used in many fields. However, single UAVs have significant limitations in target recognition. Because UAVs typically take pictures from a top-down perspective, the feature extraction angle is singular, resulting in limited detection performance for small targets against complex backgrounds. In addition, the large variation in target scale and the complexity of image backgrounds, along with the ease with which target objects are occluded, make it difficult for single UAVs to adapt to the detection needs of targets of different scales.

[0003] Furthermore, in drone aerial images, due to the limited field of view, small targets often have few pixels and simple features, making them prone to missed or false detections. At the same time, the target scale varies significantly, and the complex image background, including target occlusion, insufficient lighting, and interference from similar-shaped objects, further increases the difficulty of target detection. These factors lead to a significant decrease in target detection accuracy when there is occlusion or large changes in angle. Summary of the Invention

[0004] Based on the above analysis, the embodiments of the present invention aim to provide a multi-UAV, multi-view collaborative target detection and discrimination method based on target information, in order to solve the problems of decreased target detection accuracy and easy occurrence of missed detection and false detection in complex environments.

[0005] On one hand, embodiments of the present invention provide a multi-UAV, multi-view collaborative target detection and discrimination method based on target information, including the following steps:

[0006] Control multiple drone pods to simultaneously acquire images of the same target, forming an image set;

[0007] Based on the pre-trained target detection model, target detection and recognition are performed on each image in the image set to obtain the original bounding box corresponding to the target in each image and the attribute information of the bounding box.

[0008] The position data of each UAV is acquired separately. Based on the UAV pod camera intrinsics and the UAV position data, the original bounding box is transformed to obtain the true bounding box.

[0009] Transform all real bounding boxes to the same map to obtain bounding boxes of all images in the same map;

[0010] Based on the attribute information of the bounding boxes, perform cyclic deduplication on all the transformed bounding boxes to obtain the bounding boxes after cyclic deduplication.

[0011] Target detection and discrimination are performed based on the bounding boxes after cyclic deduplication.

[0012] As a further improvement of this application, the attribute information of the bounding box includes the center coordinates, width, height of the bounding box, and the target type, confidence level, and ID corresponding to the bounding box.

[0013] As a further improvement of this application, all transformed bounding boxes are subjected to cyclic deduplication based on their attribute information, resulting in bounding boxes after cyclic deduplication, including:

[0014] S11. Select the bounding box with the highest confidence level as the reference box, and the remaining bounding boxes as detection boxes;

[0015] S12. Traverse all detection boxes, and perform redundancy removal on the detection boxes based on the attribute information of the detection boxes and reference boxes using a preset target fusion algorithm to identify redundant detection boxes and abnormal detection boxes.

[0016] S13. Merge each identified redundant detection box with the reference box to obtain a new bounding box, and obtain the attribute information of the new bounding box based on the attribute information of the redundant detection box and the reference box. At the same time, delete the redundant detection boxes and the abnormal detection boxes to obtain the bounding box after redundancy processing. Return to step S11 until no redundant detection boxes can be identified, and obtain the bounding box after cyclic redundancy processing.

[0017] As a further improvement of this application, the detection box is deredundant by using a preset target fusion algorithm based on the attribute information of the detection box and the reference box, and redundant detection boxes and abnormal detection boxes are identified by: calculating the overlap area of ​​the current detection box and the reference box.

[0018] If the overlap area between the current detection box and the reference box is greater than or equal to the overlap area threshold, the current detection box is marked as a redundant detection box.

[0019] If the overlap area between the current detection box and the reference box is less than the overlap area threshold, then compare the target types of the current detection box and the reference box; if the target types of the current detection box and the reference box are inconsistent, then retain the detection box.

[0020] If the target type of the current detection box is the same as that of the reference box, then calculate the distance between the center point of the current detection box and the center point of the reference box; if the distance between the center point of the current detection box and the center point of the reference box is less than or equal to the distance threshold, then mark the current detection box as a redundant detection box.

[0021] If the distance between the center point of the current detection box and the center point of the reference box is greater than the distance threshold, then check whether the ID information of the current detection box is consistent with the ID information of the reference box; if the ID information of the current detection box is consistent with the ID information of the reference box, then the detection box is an abnormal detection box, and the current detection box is removed.

[0022] If the ID information of the current detection box is inconsistent with the ID information of the reference box, the detection box is retained.

[0023] As a further improvement to this application, calculating the overlap area of ​​all detection boxes and reference boxes includes:

[0024] Obtain the coordinates of all bounding boxes and reference boxes in a geocentric rectangular coordinate system;

[0025] Calculate the coordinates of the intersection region between each detection box and the reference box;

[0026] The area of ​​the intersection region is calculated based on the coordinates of the intersection region, and then the overlapping area is obtained.

[0027] As a further improvement to this application, the internal parameters of the UAV pod camera include the camera focal length; the UAV position data includes the distance between the UAV and the target;

[0028] Based on the UAV pod camera intrinsics and UAV position data, the transformation relationship for the original bounding box size transformation is shown in the calculation formula (1):

[0029]

[0030] Where μ is the pixel size of the original bounding box, f is the focal length of the drone pod camera, D is the distance between the drone and the target, and d is the actual bounding box size.

[0031] As a further improvement to this application, converting all real bounding boxes to the same map includes:

[0032] Acquire attitude data for each UAV pod when acquiring images; the attitude data includes pitch angle, roll angle, and yaw angle.

[0033] Transform all generated ground truth bounding boxes into the image space auxiliary coordinate system, and generate the image space auxiliary coordinate system coordinates of all ground truth bounding boxes;

[0034] Convert all generated image space auxiliary coordinates to body coordinates.

[0035] Convert all generated body coordinate system coordinates to station-centered horizontal coordinate system coordinates;

[0036] All generated station-centered horizontal coordinate system coordinates are transformed into the same map, which is a geocentric rectangular coordinate system.

[0037] As a further improvement to this application, the image space auxiliary coordinate system coordinates of the true bounding box are:

[0038]

[0039] The formula for converting image space auxiliary coordinates to body coordinates is:

[0040]

[0041] Where X1, Y1, Z1 are coordinates in the image space auxiliary coordinate system; Dis is the distance from the pod to the target; X2, Y2, Z2 are coordinates in the target's body auxiliary coordinate system; X1, Y1, Z1 are coordinates in the target's image space auxiliary coordinate system; X0, Y0, Z0 are the coordinates of the electro-optical pod in the body coordinate system; K is the positive rotation matrix for the pod placement angle; and A is the rotation matrix.

[0042] As a further improvement to this application, the formula for converting the coordinates of the body coordinate system to the station-centered horizontal coordinate system is as follows:

[0043]

[0044] Where X3, Y3, and Z3 are the coordinates of the target station in the horizontal coordinate system; X2, Y2, and Z2 are the coordinates of the target in the body coordinate system; θ is the heading angle of the UAV, ψ is the pitch angle of the UAV, and φ is the roll angle of the UAV.

[0045] As a further improvement to this application, the formula for converting the coordinates in the station-centered horizontal coordinate system to the geocentric rectangular coordinate system is as follows:

[0046]

[0047] Where X4, Y4, and Z4 are the target's geocentric rectangular coordinates; X3, Y3, and Z3 are the target's stationary horizontal coordinates; B is the UAV's latitude, and L is the UAV's longitude; XA, YA, and ZA are the UAV's geocentric rectangular coordinates.

[0048] On the other hand, embodiments of the present invention provide a multi-UAV, multi-view collaborative target detection and discrimination system based on target information, comprising:

[0049] The image acquisition module is used to control multiple UAV pods to simultaneously acquire images of the same target, forming an image set.

[0050] The detection module is used to perform target detection and recognition on each image in the image set based on a pre-trained target detection model, and obtain the original bounding box corresponding to the target in each image and the attribute information of the bounding box.

[0051] The bounding box transformation module is used to acquire the position data of each UAV separately, and transform the original bounding box to obtain the real bounding box based on the UAV pod camera intrinsic parameters and the UAV position data.

[0052] The map conversion module is used to convert all real bounding boxes to the same map, resulting in bounding boxes of all images in the same map;

[0053] The iterative deduplication module is used to perform iterative deduplication on all the transformed bounding boxes based on the attribute information of the bounding boxes, so as to obtain the bounding boxes after iterative deduplication.

[0054] The detection and discrimination module is used to detect and discriminate targets based on the bounding boxes after cyclic deduplication.

[0055] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0056] This application effectively solves the problem of decreased accuracy when there is occlusion or large angle changes by controlling multiple UAV pods to acquire images of the same target simultaneously and performing cyclic deduplication based on the attribute information of the bounding box. It can capture targets from multiple perspectives, reduce the impact of occlusion under a single perspective, and improve the accuracy of detection.

[0057] The technical solution of this application reduces the occurrence of missed detections and false detections by performing cyclic deduplication on all converted bounding boxes and identifying redundant and abnormal detection boxes. By merging target information from multiple perspectives, it enhances the target detection capability in complex environments and reduces the occurrence of missed detections and false detections.

[0058] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0059] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0060] Figure 1 This is a flowchart illustrating a multi-UAV, multi-view collaborative target detection and discrimination method based on target information according to an embodiment of the present invention.

[0061] Figure 2 This is a schematic diagram illustrating the size transformation of the original bounding box in an embodiment of the present invention;

[0062] Figure 3 This is a schematic diagram of the body coordinate system according to an embodiment of the present invention. Detailed Implementation

[0063] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0064] A specific embodiment of the present invention discloses a multi-UAV, multi-view collaborative target detection and discrimination method based on target information, such as... Figure 1 As shown. The method includes the following steps:

[0065] S1. Control multiple UAV pods to simultaneously acquire images of the same target and form an image set;

[0066] Multiple drones are deployed to surround or cover the same target area from different angles, collecting images of the target from different positions and angles to enhance the comprehensive capture capability of target features. The drone pod is the part of the drone equipped with a camera, and images taken by multiple drone pods from different positions and angles are combined to form an image set.

[0067] S2. Based on the pre-trained target detection model, target detection and recognition are performed on each image in the image set to obtain the original bounding box corresponding to the target in each image and the attribute information of the bounding box.

[0068] Object detection models can output the original bounding box and its attribute information for the target. YOLO series models, such as YOLOv5 and YOLOv8, can be selected for object detection. The bounding box is a rectangular region that encloses the target object, defining its position and size in the image. The bounding box can be composed of the center point coordinates, width, and height. In object detection, the bounding box is typically represented by the center point coordinates (cx, cy) and the width (w) and height (h), i.e., (cx, cy, w, h).

[0069] Pre-training involves setting up a training set, labeling samples in the training set, annotating the bounding boxes containing targets and their attribute information, and then training the target detection model. The original bounding boxes represent the predicted target locations directly on the image. The attribute information of each bounding box includes detailed information such as its center coordinates, width, height, target type, confidence score, and ID. The target type is the category to which the detected target belongs; the confidence score is the probability that the bounding box contains the target, typically a value between 0 and 1; and the ID is an identifier for each bounding box, used to distinguish bounding boxes in images taken by different drones. All bounding boxes in images taken by the same drone have the same ID.

[0070] S3. Acquire the position data of each UAV. Based on the UAV pod camera intrinsics and the UAV position data, transform the original bounding box to obtain the true bounding box.

[0071] S4. Convert all real bounding boxes to the same map to obtain bounding boxes of all images in the same map;

[0072] like Figure 2 As shown. The UAV pod camera intrinsics include the camera focal length; the UAV position data includes the distance between the UAV and the target; based on the UAV pod camera intrinsics and the UAV position data, the transformation relationship for the original bounding box size transformation is shown in calculation formula (1):

[0073]

[0074] Where μ is the pixel size of the original bounding box, f is the focal length of the drone pod camera, D is the distance between the drone and the target, and d is the actual bounding box size.

[0075] S5. Based on the attribute information of the bounding boxes, perform cyclic deduplication on all the transformed bounding boxes to obtain the bounding boxes after cyclic deduplication.

[0076] Specifically, based on the attribute information of the bounding boxes, a cyclic deduplication process is performed on all the transformed bounding boxes to obtain the cyclically deduplicated bounding boxes, including:

[0077] S11. Select the bounding box with the highest confidence level as the reference box, and the remaining bounding boxes as detection boxes;

[0078] S12. Traverse all detection boxes and perform redundancy removal on the detection boxes based on the attribute information of the detection boxes and reference boxes using a preset target fusion algorithm to identify redundant detection boxes and abnormal detection boxes.

[0079] S13. Merge each identified redundant detection box with the reference box to obtain a new bounding box, and obtain the attribute information of the new bounding box based on the attribute information of the redundant detection box and the reference box. At the same time, delete the redundant detection boxes and the abnormal detection boxes to obtain the bounding box after redundancy processing. Return to step S11 until no redundant detection boxes can be identified, and obtain the bounding box after cyclic redundancy processing.

[0080] When merging redundant detection boxes and reference boxes, the center point coordinates of the new bounding box are weighted according to the confidence levels of the redundant detection boxes and reference boxes. The center point coordinates are the weighted average of the center point coordinates of the redundant detection boxes and reference boxes. The width and height are the width and height of the smallest bounding rectangle that simultaneously contains the width and height of the redundant detection boxes and reference boxes, respectively, ensuring that the new bounding box can cover the target areas corresponding to the redundant detection boxes and reference boxes.

[0081] The confidence score of the new bounding box can be obtained using a weighted average method or the Soft NMS algorithm. The Soft NMS algorithm assigns weights to the redundant detection box and the reference box by calculating the intersection-union ratio (IU / U) and generates the confidence score of the new bounding box. The ID and attribute information of the new bounding box are the same as those of the original redundant detection box.

[0082] Further, step S12 includes:

[0083] Calculate the overlap area between the current detection box and the reference box;

[0084] If the overlap area between the current detection box and the reference box is greater than or equal to the overlap area threshold, the current detection box is marked as a redundant detection box.

[0085] If the overlap area between the current detection box and the reference box is less than the overlap area threshold, then compare the target types of the current detection box and the reference box; if the target types of the current detection box and the reference box are inconsistent, then retain the detection box.

[0086] If the target type of the current detection box is the same as that of the reference box, then calculate the distance between the center point of the current detection box and the center point of the reference box; if the distance between the center point of the current detection box and the center point of the reference box is less than or equal to the distance threshold, then mark the current detection box as a redundant detection box.

[0087] If the distance between the center point of the current detection box and the center point of the reference box is greater than the distance threshold, then check whether the ID information of the current detection box is consistent with the ID information of the reference box; if the ID information of the current detection box is consistent with the ID information of the reference box, then the detection box is an abnormal detection box, and the current detection box is removed.

[0088] If the ID information of the current detection box is inconsistent with the ID information of the reference box, the detection box is retained.

[0089] The overlap area between the current detection box and the reference box is the area of ​​the overlapping portion of the detection box and the reference box. Calculating the overlap area of ​​the detection box and the reference box includes:

[0090] Obtain the coordinates of all detection boxes and reference boxes in a geocentric rectangular coordinate system; calculate the coordinates of the intersection region of each detection box and reference box; calculate the area of ​​the intersection region based on the coordinates of the intersection region, and thus obtain the overlap area.

[0091] The overlap area threshold can be set empirically or determined by analyzing data from specific application scenarios. For example, an overlap area threshold of 0.5 means that if the overlap area between the detection box and the reference box reaches or exceeds 50%, the current detection box is marked as redundant. The overlap area threshold can be adaptively adjusted according to the complexity of the scene and the size of the target. In scenes with dense targets, the overlap area threshold needs to be lowered to reduce false positives; while in scenes with sparse targets, the overlap area threshold can be increased to improve the accuracy of redundancy removal.

[0092] The distance between the center point of the current detection box and the center point of the reference box can be calculated using the Euclidean distance formula. The center point is the geometric center of the detection box and the reference box. The distance threshold can be set according to the size of the target and the specific situation of the scene.

[0093] S6. Detect and identify targets based on the bounding boxes after cyclic deduplication.

[0094] In some embodiments, step S4, converting all real bounding boxes to the same map, includes:

[0095] Acquire attitude data for each UAV pod when acquiring images; the attitude data includes pitch angle, roll angle, and yaw angle.

[0096] Transform all generated ground truth bounding boxes into the image space auxiliary coordinate system, and generate the image space auxiliary coordinate system coordinates of all ground truth bounding boxes;

[0097] Convert all generated image space auxiliary coordinates to body coordinates.

[0098] Convert all generated body coordinate system coordinates to station-centered horizontal coordinate system coordinates;

[0099] All generated station-centered horizontal coordinate system coordinates are transformed into the same map, which is a geocentric rectangular coordinate system.

[0100] It should be noted that the attitude data acquired by the UAV pod when acquiring images includes pitch, roll, and yaw angles. Pitch angle is the degree of tilt of the UAV relative to the horizontal plane, roll angle is the rotation of the UAV fuselage around its direction of travel, and yaw angle is the rotation of the UAV relative to geographic north. Pitch, roll, and yaw angles can be obtained through the UAV's inertial measurement unit (IMU), which provides real-time attitude information for the UAV.

[0101] The image space auxiliary coordinate system is a local coordinate system relative to the camera lens, used to describe the position of the target in the image. The origin of the image space auxiliary coordinate system is located at the optical center of the camera lens, i.e., the projection center S. The image space auxiliary coordinate system and the body coordinate system can be defined as a left-handed system, with the X-axis pointing to the top of the image, the Y-axis pointing to the right side of the image, and the Z-axis pointing upwards.

[0102] The body coordinate system is a coordinate system relative to the drone's body, used to describe the position and orientation of the camera on the drone. The origin of the body coordinate system is located at the drone's center of mass, the X-axis points towards the nose, the Y-axis points to the right side of the drone's plane, and the Z-axis is perpendicular to the drone and points upwards. For example... Figure 3 As shown.

[0103] The station-centered horizontal coordinate system is a local coordinate system relative to the current position of the UAV. The origin of the station-centered horizontal coordinate system is the current position of the UAV, used to describe the geographical location of the UAV's surrounding environment. The station-centered horizontal coordinate system is defined as a left-handed system. The X-axis points towards the meridian, with north being positive; the Z-axis points towards the ellipsoidal normal passing through the origin, with zenith being positive; and the Y-axis points perpendicular to the plane of the X and Z axes, with east being positive.

[0104] The geocentric rectangular coordinate system is a globally unified coordinate system with the Earth's center of mass as its origin, used to describe the precise location of any point on Earth. The geocentric rectangular coordinate system is defined as a right-handed system, with the center of the reference ellipsoid as its origin. The intersection of the prime meridian and the equatorial plane is the X-axis, the direction orthogonal to the X-axis on the equatorial plane is the Y-axis, and the rotation axis of the ellipsoid is the Z-axis.

[0105] Specifically, the image space auxiliary coordinates of the true bounding box are:

[0106]

[0107] The formula for converting image space auxiliary coordinates to body coordinates is:

[0108]

[0109] Where X1, Y1, Z1 are coordinates in the image space auxiliary coordinate system; Dis is the distance from the pod to the target; X2, Y2, Z2 are coordinates in the target's body auxiliary coordinate system; X1, Y1, Z1 are coordinates in the target's image space auxiliary coordinate system; X0, Y0, Z0 are the coordinates of the electro-optical pod in the body coordinate system; K is the positive rotation matrix for the pod placement angle; and A is the rotation matrix.

[0110] The pod placement angle positive rotation matrix K is a rotation matrix describing the fixed position and orientation of the UAV pod relative to the UAV body. Rotation matrix A is a general rotation matrix used to describe rotational transformations from one coordinate system to another in three-dimensional space.

[0111] Specifically, the formula for calculating the positive rotation matrix K of the pod placement angle is:

[0112]

[0113] In the formula, y is the pod heading angle, with right yaw being positive; p is the pod pitch angle, with pitch up being positive and pitch down being negative; and r is the pod roll angle, with right pitch being positive.

[0114] The formula for calculating the rotation matrix A is:

[0115]

[0116] In the formula, Yaw is the azimuth angle of the pod relative to the nose of the UAV, with rightward deviation being positive; P is the pitch angle of the pod relative to the aircraft, with pitch up being positive and pitch down being negative; R is the roll angle of the pod relative to the UAV; and K is the pod placement angle correction rotation matrix.

[0117] The formula for converting the coordinates from the body coordinate system to the station-centered horizontal coordinate system is as follows:

[0118]

[0119] Where X3, Y3, and Z3 are the coordinates of the target station in the horizontal coordinate system; X2, Y2, and Z2 are the coordinates of the target in the body coordinate system; θ is the heading angle of the UAV, ψ is the pitch angle of the UAV, and φ is the roll angle of the UAV.

[0120] The formula for converting station-centered horizontal coordinates to geocentric rectangular coordinates is as follows:

[0121]

[0122] Where X4, Y4, and Z4 are the target's geocentric rectangular coordinates; X3, Y3, and Z3 are the target's stationary horizontal coordinates; B is the UAV's latitude, and L is the UAV's longitude; XA, YA, and ZA are the UAV's geocentric rectangular coordinates.

[0123] Compared with existing technologies, the multi-UAV multi-view collaborative target detection and discrimination method based on target information provided in this embodiment effectively reduces detection errors caused by single viewpoint, occlusion, or changes in lighting by integrating image data captured by multiple UAVs from different viewpoints, and enhances the ability to identify targets in complex environments; the bounding box is optimized through cyclic deduplication, reducing redundant detection and false detection, and improving detection efficiency.

[0124] Another specific embodiment of the present invention discloses a multi-UAV multi-view cooperative target detection and discrimination system based on target information, comprising:

[0125] The image acquisition module is used to control multiple UAV pods to simultaneously acquire images of the same target, forming an image set.

[0126] The detection module is used to perform target detection and recognition on each image in the image set based on a pre-trained target detection model, and obtain the original bounding box corresponding to the target in each image and the attribute information of the bounding box.

[0127] The bounding box transformation module is used to acquire the position data of each UAV separately, and transform the original bounding box to obtain the real bounding box based on the UAV pod camera intrinsic parameters and the UAV position data.

[0128] The map conversion module is used to convert all real bounding boxes to the same map, resulting in bounding boxes of all images in the same map;

[0129] The iterative deduplication module is used to perform iterative deduplication on all the transformed bounding boxes based on the attribute information of the bounding boxes, so as to obtain the bounding boxes after iterative deduplication.

[0130] The detection and discrimination module is used to detect and discriminate targets based on the bounding boxes after cyclic deduplication.

[0131] It is understandable that the steps in this multi-UAV multi-view collaborative target detection and discrimination system based on target information correspond to those in the multi-UAV multi-view collaborative target detection and discrimination method based on target information, and will not be elaborated here.

[0132] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0133] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-UAV, multi-view collaborative target detection and discrimination method based on target information, characterized in that, Includes the following steps: Control multiple drone pods to simultaneously acquire images of the same target, forming an image set; Based on the pre-trained target detection model, target detection and recognition are performed on each image in the image set to obtain the original bounding box corresponding to the target in each image and the attribute information of the bounding box. The position data of each UAV is acquired separately. Based on the UAV pod camera intrinsics and the UAV position data, the original bounding box is transformed to obtain the true bounding box. Transform all real bounding boxes to the same map to obtain bounding boxes of all images in the same map; Based on the attribute information of the bounding boxes, perform cyclic deduplication on all the transformed bounding boxes to obtain the bounding boxes after cyclic deduplication. Target detection and discrimination are performed based on the bounding boxes after cyclic deduplication.

2. The method according to claim 1, characterized in that, The attribute information of the bounding box includes the center coordinates, width, height of the bounding box, as well as the target type, confidence level, and ID corresponding to the bounding box.

3. The method according to claim 1, characterized in that, Based on the attribute information of the bounding boxes, a cyclic deduplication process is performed on all the transformed bounding boxes to obtain the following deduplicated bounding boxes: S11. Select the bounding box with the highest confidence level as the reference box, and the remaining bounding boxes as detection boxes; S12. Traverse all detection boxes, and perform redundancy removal on the detection boxes based on the attribute information of the detection boxes and reference boxes using a preset target fusion algorithm to identify redundant detection boxes and abnormal detection boxes. S13. Merge each identified redundant detection box with the reference box to obtain a new bounding box, and obtain the attribute information of the new bounding box based on the attribute information of the redundant detection box and the reference box. At the same time, delete the redundant detection boxes and the abnormal detection boxes to obtain the bounding box after redundancy processing. Return to step S11 until no redundant detection boxes can be identified, and obtain the bounding box after cyclic redundancy processing.

4. The method according to claim 3, characterized in that, Based on the attribute information of the detection box and the reference box, the detection box is deredundant through a preset target fusion algorithm to identify redundant detection boxes and abnormal detection boxes, including: calculating the overlap area between the current detection box and the reference box; If the overlap area between the current detection box and the reference box is greater than or equal to the overlap area threshold, the current detection box is marked as a redundant detection box. If the overlap area between the current detection box and the reference box is less than the overlap area threshold, then compare the target types of the current detection box and the reference box; if the target types of the current detection box and the reference box are inconsistent, then retain the detection box. If the target type of the current detection box is the same as that of the reference box, then calculate the distance between the center point of the current detection box and the center point of the reference box; if the distance between the center point of the current detection box and the center point of the reference box is less than or equal to the distance threshold, then mark the current detection box as a redundant detection box. If the distance between the center point of the current detection box and the center point of the reference box is greater than the distance threshold, then check whether the ID information of the current detection box is consistent with the ID information of the reference box; if the ID information of the current detection box is consistent with the ID information of the reference box, then the detection box is an abnormal detection box, and the current detection box is removed. If the ID information of the current detection box is inconsistent with the ID information of the reference box, the detection box is retained.

5. The method according to claim 4, characterized in that, Calculating the overlap area of ​​all detection boxes and reference boxes includes: Obtain the coordinates of all bounding boxes and reference boxes in a geocentric rectangular coordinate system; Calculate the coordinates of the intersection region between each detection box and the reference box; The area of ​​the intersection region is calculated based on the coordinates of the intersection region, and then the overlapping area is obtained.

6. The method according to claim 1, characterized in that, The internal parameters of the drone pod camera include the camera focal length; the drone position data includes the distance between the drone and the target; Based on the UAV pod camera intrinsics and UAV position data, the transformation relationship for the original bounding box size transformation is shown in the calculation formula (1): Where μ is the pixel size of the original bounding box, f is the focal length of the drone pod camera, D is the distance between the drone and the target, and d is the actual bounding box size.

7. The method according to claim 2, characterized in that, Converting all real-world bounding boxes to the same map includes: Acquire attitude data for each UAV pod when acquiring images; the attitude data includes pitch angle, roll angle, and yaw angle. Transform all generated ground truth bounding boxes into the image space auxiliary coordinate system, and generate the image space auxiliary coordinate system coordinates of all ground truth bounding boxes; Convert all generated image space auxiliary coordinates to body coordinates. Convert all generated body coordinate system coordinates to station-centered horizontal coordinate system coordinates; All generated station-centered horizontal coordinate system coordinates are transformed into the same map, which is a geocentric rectangular coordinate system.

8. The method according to claim 7, characterized in that, The image space auxiliary coordinates of the true bounding box are: The formula for converting image space auxiliary coordinates to body coordinates is: Where X1, Y1, Z1 are coordinates in the image space auxiliary coordinate system; Dis is the distance from the pod to the target; X2, Y2, Z2 are coordinates in the target's body auxiliary coordinate system; X1, Y1, Z1 are coordinates in the target's image space auxiliary coordinate system; X0, Y0, Z0 are the coordinates of the electro-optical pod in the body coordinate system; K is the positive rotation matrix for the pod placement angle; and A is the rotation matrix.

9. The method according to claim 8, characterized in that, The formula for converting the coordinates from the body coordinate system to the station-centered horizontal coordinate system is as follows: Where X3, Y3, and Z3 are the target station center-horizontal coordinates; X2, Y2, and Z2 are the target body coordinates; θ is the UAV heading angle, ψ is the UAV pitch angle, and φ is the UAV roll angle. The formula for converting station-centered horizontal coordinates to geocentric rectangular coordinates is as follows: Where X4, Y4, and Z4 are the target's geocentric rectangular coordinates; X3, Y3, and Z3 are the target's stationary horizontal coordinates; B is the UAV's latitude, and L is the UAV's longitude; XA, YA, and ZA are the UAV's geocentric rectangular coordinates.

10. A multi-UAV, multi-view collaborative target detection and discrimination system based on target information, characterized in that, include: The image acquisition module is used to control multiple UAV pods to simultaneously acquire images of the same target, forming an image set. The detection module is used to perform target detection and recognition on each image in the image set based on a pre-trained target detection model, and obtain the original bounding box corresponding to the target in each image and the attribute information of the bounding box. The bounding box transformation module is used to acquire the position data of each UAV separately, and transform the original bounding box to obtain the real bounding box based on the UAV pod camera intrinsic parameters and the UAV position data. The map conversion module is used to convert all real bounding boxes to the same map, resulting in bounding boxes of all images in the same map; The iterative deduplication module is used to perform iterative deduplication on all the transformed bounding boxes based on the attribute information of the bounding boxes, so as to obtain the bounding boxes after iterative deduplication. The detection and discrimination module is used to detect and discriminate targets based on the bounding boxes after cyclic deduplication.

Images