Unmanned aerial vehicle detection and tracking method and apparatus, electronic device, and storage medium
By using feature vector dimensionality reduction and loss function weighting, combined with Kalman filtering, the accuracy and efficiency of UAV detection and tracking algorithms in complex environments are improved, solving the problems of long detection time and decreased accuracy, and achieving efficient target object detection and tracking.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- WUHU AUTOMOBILE ADVANCED TECHNOLOGY INSTITUTE
- Filing Date
- 2026-04-29
- Publication Date
- 2026-07-14
AI Technical Summary
Existing drone detection and tracking algorithms suffer from decreased accuracy in complex environments (such as changes in lighting, occlusion, and cluttered backgrounds), leading to missed detections, false detections, and excessively long detection times, which affects tracking performance.
A pre-trained model is used to detect and recognize environmental images. By using feature vector dimensionality reduction and loss function weighting, combined with Kalman filtering, high-precision target object detection and tracking are achieved, reducing computational consumption.
It improves the accuracy and efficiency of UAV detection and tracking, reduces the amount of model training and the computational load of detection and recognition, and is applicable to fields such as intelligent transportation, security monitoring, and autonomous driving. It solves the accuracy problem of UAV detection and tracking algorithms in complex environments and achieves efficient target object detection and tracking.
Smart Images
Figure CN122391174A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a UAV detection and tracking method, apparatus, electronic device, and storage medium. Background Technology
[0002] Drone-based detection and tracking technology has shown broad application prospects in fields such as intelligent transportation, security monitoring, and autonomous driving.
[0003] Existing technologies mainly focus on research in areas such as UAV aerial image processing, multi-sensor fusion, target detection algorithm optimization, and real-time tracking control, with an emphasis on solving problems related to detection accuracy, tracking accuracy, and resource utilization in engineering applications.
[0004] Currently, in complex environments (such as changes in lighting, occlusion, cluttered backgrounds, etc.), the accuracy of the tracking algorithm may decrease, leading to missed detections, false detections, and excessively long detection times, which in turn affects the tracking performance. Summary of the Invention
[0005] In view of this, this application provides a method for detecting and tracking unmanned aerial vehicles (UAVs), which can reduce computing power consumption while achieving high-precision UAV detection and tracking.
[0006] On the one hand, this application provides a method for detecting and tracking unmanned aerial vehicles (UAVs), the method including: Capture environmental images.
[0007] The first model, which was trained in advance, was used to detect and recognize environmental images, and multiple candidate objects were identified.
[0008] Select the target object to be tracked from multiple candidate objects.
[0009] Control the drone based on the real-time position of the target object.
[0010] Optionally, before capturing environmental images, the method may also include: Acquire multiple sample images.
[0011] Extract the first feature vector from multiple sample images.
[0012] The first eigenvector is reduced in dimensionality to obtain the second eigenvector.
[0013] The first model is obtained by training using the second feature vector.
[0014] Optionally, the first model can be obtained by training using the second feature vector, including: Obtain the regression loss and classification loss during the detection process, and the contrast loss during the recognition process.
[0015] The regression loss, classification loss, and contrast loss are weighted to obtain the final loss.
[0016] The first model is then corrected using the final loss.
[0017] Optionally, a pre-trained first model is used to detect and identify environmental images, identifying multiple candidate objects including: The first model, which was trained in advance, was used to detect and recognize environmental images, and multiple initial screening boxes were determined.
[0018] Confidence filtering is performed on multiple initial screening boxes to obtain multiple pre-selected boxes.
[0019] Identify multiple candidate objects that correspond one-to-one with multiple preselection boxes.
[0020] On the other hand, this application provides a drone detection and tracking device, the device comprising: The camera module is configured to capture images of the environment.
[0021] The determination module is configured to detect and identify environmental images using a pre-trained first model, thereby identifying multiple candidate objects.
[0022] The selection module is configured to select the target object to be tracked from multiple candidate objects.
[0023] The control module is configured to control the drone based on the real-time position of the target object.
[0024] Optionally, the device also includes a training module, which is configured to: Acquire multiple sample images.
[0025] Extract the first feature vector from multiple sample images.
[0026] The first eigenvector is reduced in dimensionality to obtain the second eigenvector.
[0027] The first model is obtained by training using the second feature vector.
[0028] Alternatively, the training module can also be configured as follows: Obtain the regression loss and classification loss during the detection process, and the contrast loss during the recognition process.
[0029] The regression loss, classification loss, and contrast loss are weighted to obtain the final loss.
[0030] The first model is then corrected using the final loss.
[0031] Optionally, the determination module is also configured as follows: The first model, which was trained in advance, was used to detect and recognize environmental images, and multiple initial screening boxes were determined.
[0032] Confidence filtering is performed on multiple initial screening boxes to obtain multiple pre-selected boxes.
[0033] Identify multiple candidate objects that correspond one-to-one with multiple preselection boxes.
[0034] In another aspect, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the unmanned aerial vehicle detection and tracking method provided in the foregoing.
[0035] In another aspect, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the unmanned aerial vehicle detection and tracking method provided in the foregoing aspects.
[0036] The UAV detection and tracking method provided in this application uses a single model for the detection and recognition of environmental images after they are captured. That is, the detection and recognition of environmental images are completed using a pre-trained first model, thereby reducing the amount of model training and the amount of computation for detection and recognition. After identifying multiple candidate objects from the environmental images, the target object to be tracked can be selected from the multiple candidate objects, and the UAV can be controlled according to the real-time position of the target object, thereby realizing the detection and tracking of the target object. Attached Figure Description
[0037] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0038] Figure 1 A flowchart of the drone detection and tracking method provided in the embodiments of this application; Figure 2 Another flowchart of the drone detection and tracking method provided in the embodiments of this application; Figure 3 This is an architectural diagram of the drone detection and tracking device provided in an embodiment of this application. Detailed Implementation
[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0040] This application provides a method for detecting and tracking unmanned aerial vehicles (UAVs). This method can be executed by a vehicle controller, or by a control terminal or processor of an electronic device outside the vehicle. Figure 1 As shown, the method includes steps S101, S102, S103, and S104, wherein: In step S101, an environmental image is captured.
[0041] In step S102, the pre-trained first model is used to detect and identify the environmental image, and multiple candidate objects are identified.
[0042] In step S103, the target object to be tracked is selected from multiple candidate objects.
[0043] In step S104, the drone is controlled according to the real-time position of the target object.
[0044] In some optional embodiments, the method further includes, before capturing environmental images: Acquire multiple sample images.
[0045] Extract the first feature vector from multiple sample images.
[0046] The first eigenvector is reduced in dimensionality to obtain the second eigenvector.
[0047] The first model is obtained by training using the second feature vector.
[0048] In some optional embodiments, training the first model using the second feature vector includes: Obtain the regression loss and classification loss during the detection process, and the contrast loss during the recognition process.
[0049] The regression loss, classification loss, and contrast loss are weighted to obtain the final loss.
[0050] The first model is then corrected using the final loss.
[0051] In some optional embodiments, a pre-trained first model is used to detect and identify environmental images, identifying multiple candidate objects including: The first model, which was trained in advance, was used to detect and recognize environmental images, and multiple initial screening boxes were determined.
[0052] Confidence filtering is performed on multiple initial screening boxes to obtain multiple pre-selected boxes.
[0053] Identify multiple candidate objects that correspond one-to-one with multiple preselection boxes.
[0054] The UAV detection and tracking method provided in this application uses a single model for the detection and recognition of environmental images after they are captured. That is, the detection and recognition of environmental images are completed using a pre-trained first model, thereby reducing the amount of model training and the amount of computation for detection and recognition. After identifying multiple candidate objects from the environmental images, the target object to be tracked can be selected from the multiple candidate objects, and the UAV can be controlled according to the real-time position of the target object, thereby realizing the detection and tracking of the target object.
[0055] This application also provides a method for detecting and tracking unmanned aerial vehicles (UAVs). This method can be executed by a vehicle controller, or by a control terminal or processor of an electronic device outside the vehicle. Figure 2 As shown, the method includes steps S201, S202, S203, S204, S205, S206, S207, S208, S209, and S210, wherein: In step S201, multiple sample images are acquired.
[0056] In some optional embodiments, the sample images can be obtained by taking pictures using a drone. After the drone takes pictures of the sample images, it can directly save them in the drone's storage medium, or it can send them to another storage terminal via a wired or wireless connection. The storage terminal can also send the sample images to a server. Therefore, the sample images in step S201 can be obtained from a drone, a storage terminal, or a server.
[0057] In step S202, a first feature vector is extracted from multiple sample images.
[0058] In some alternative embodiments, the process of extracting a first feature vector from multiple sample images may include: In step one, a lightweight deep learning convolutional network is used to obtain low-level features from multiple sample images to form a low-level feature map.
[0059] In step two, high-level features are extracted from multiple sample images to form a high-level feature map, and the corresponding semantic information is extracted.
[0060] In step three, the detection task is performed, which uses a Feature Pyramid Network (FPN). Through feature fusion, the semantic information of the high-level feature map is fused with the spatial information of the low-level feature map to generate feature representations with rich multi-scale information. These feature representations can form an intermediate map, thereby improving the detection performance of multi-scale targets, especially the detection capability of small targets, without significantly increasing the computational load.
[0061] In step four, the recognition task is performed, which involves extracting local branch features from the intermediate image obtained in the previous step using a horizontal segmentation method. By dividing the image region, fine-grained local semantic information is captured, and the local features are fused with the global features to obtain the final recognition feature vector, i.e., the first feature vector, thereby improving the model's robustness to viewpoint changes and occlusion.
[0062] In step S203, the first feature vector is subjected to dimensionality reduction processing to obtain the second feature vector.
[0063] In some alternative embodiments, Principal Component Analysis (PCA) can be used to reduce the dimensionality of the first eigenvector to obtain the second eigenvector. PCA is an unsupervised linear dimensionality reduction algorithm whose core objective is to maximize the variance of the projected data and retain the most informative dimension in the data.
[0064] In some alternative embodiments, the first feature vector can also be reduced in dimensionality using the Linear Discriminant Analysis (LDA) method to obtain the second feature vector. LDA is a supervised linear dimensionality reduction algorithm whose core objective is to maximize the inter-class distance and minimize the intra-class distance, making the dimensionality-reduced data more suitable for classification tasks.
[0065] In step S204, the first model is obtained by training using the second feature vector.
[0066] In some optional embodiments, the trained first model can serve as both the backbone network of a detection model for image detection and a recognition model for identifying target objects in the image, thereby improving tracking accuracy. It is understood that "detecting an image" refers to obtaining the location information of the target in the image, i.e., coordinate values, while "recognizing an image" refers to obtaining the feature values of the target object, which is another numerical representation of the target object with higher dimensions. The complete network of the trained first model can be used for image detection, and the backbone network structure of this network is essentially equivalent to a recognition model network, with outputs being the feature values of the image. Therefore, during image detection, the output of the backbone network of the first model can be used as the result of image recognition. In other words, the method provided in this embodiment integrates the recognition network into the detection network, reducing the number of models used, lowering the computational load, improving detection and recognition efficiency, and thus improving tracking performance. In some optional embodiments, after obtaining the first model, structured pruning can be performed on the first model, deleting channels in the convolutional neural network of the first model with activation values less than a preset activation threshold, i.e., deleting unimportant convolutional channels, reducing the number of parameters and the model size without reducing accuracy.
[0067] In some optional embodiments, the INT8 quantization model can be further utilized to quantize the first model, which is usually calculated using 32-bit floating-point numbers (FP32), into a model that is calculated using 8-bit integers (INT8), thereby reducing the size of the first model and increasing the inference speed.
[0068] In step S205, an environmental image is captured.
[0069] In some optional embodiments, the environmental image is obtained by taking pictures using a drone. After the drone takes pictures of the environmental image, it can be directly saved in the drone's storage medium, or it can be sent to another storage terminal via a wired or wireless connection. The storage terminal can also send the environmental image to a server. Therefore, the environmental image in step S205 can be obtained from the drone, the storage terminal, or the server.
[0070] In step S206, the pre-trained first model is used to detect and recognize the environmental image, and multiple initial screening boxes are determined.
[0071] In some optional embodiments, a pre-trained first model is used to extract multi-scale features from the environmental image, thereby realizing the detection and recognition of the environmental image. The final output is a set of multiple preliminary screening boxes, each of which has a corresponding preliminary screening box coordinate point (x,y,w,h) and a preliminary screening box confidence score.
[0072] Each initial screening box contains an object detected and identified from the environmental image.
[0073] In step S207, confidence filtering is performed on multiple initial screening boxes to obtain multiple pre-selection boxes.
[0074] In some optional embodiments, a confidence threshold T{score} is preset, and the set of multiple initial screening boxes is filtered using the confidence threshold T{score}. The initial screening boxes with low confidence are filtered out, thereby filtering out objects with low confidence in the environmental image and reducing the amount of subsequent calculations.
[0075] In some optional embodiments, the confidence threshold T{score} = 0.25. In some optional embodiments, after performing confidence filtering on multiple initial screening boxes to obtain multiple pre-selected boxes, the remaining pre-selected boxes can be refined using the Distribution Focal Loss (DFL) algorithm to adjust the coordinates (indicating the size and position of the pre-selected boxes), thereby obtaining more accurate pre-selected boxes.
[0076] The DFL algorithm outputs the location of the bramble by integrating the discrete probability distribution. Traditional regression algorithms directly predict coordinate values, while the DFL algorithm discretizes the coordinate range (e.g., 0~255) into multiple intervals, outputs the probability of each interval, uses cross-entropy to concentrate the distribution near the actual location, and then weights and sums them to obtain the final coordinates of the preselected box. The DFL algorithm is more accurate in locating small targets, dense areas, and occluded scenes.
[0077] For the same object, multiple overlapping bounding boxes often appear after detection and recognition using a model. Therefore, in some optional embodiments, non-maximum suppression (NMS) can be further applied to these bounding boxes to remove duplicates. For multiple bounding boxes corresponding to the same object, NMS sorts them by confidence level, keeping only the box with the highest confidence level, thereby further reducing the computational load.
[0078] In step S208, multiple candidate objects are identified that correspond one-to-one with multiple preselection boxes.
[0079] In step S209, the target object to be tracked is selected from multiple candidate objects.
[0080] In some optional embodiments, the terminal's interactive screen can be used to select a pre-selection box from multiple pre-selection boxes corresponding to multiple candidate objects using touch operation. The selected pre-selection box is used as the target box, and the candidate object corresponding to the target box is used as the target object, thereby realizing the selection of the target object to be tracked from multiple candidate objects.
[0081] In step S210, the drone is controlled according to the real-time position of the target object.
[0082] In some optional embodiments, the trained first model can further utilize the Kalman filter method to predict the real-time position of the target object. The Kalman filter method predicts the possible position of the target object in the current frame based on the target object's position and velocity in the previous frame, thereby reducing the search range.
[0083] In some optional embodiments, the location information of the interaction and the detected target object can be matched to lock the target object. On the other hand, the algorithm result (pixel coordinates of the target object) is converted to the UAV body coordinate system. Then, by fusing historical trajectories, the running direction of the target object is calculated, and the running direction of the UAV is controlled by the running direction of the target object, so that the target object is as close as possible to the center of the image captured by the UAV, so as to achieve real-time positioning and tracking of the target object.
[0084] In some optional embodiments, training the first model using the second feature vector includes: Obtain the regression loss and classification loss during the detection process, and the contrast loss during the recognition process.
[0085] The regression loss, classification loss, and contrast loss are weighted to obtain the final loss.
[0086] The first model is then corrected using the final loss.
[0087] In some optional embodiments, the regression loss uses two loss functions: CIOU and DFL, where: The DFL loss function is a loss function based on distribution learning. Its purpose is to make the predicted distribution as close as possible to the true distribution. It uses cross-entropy loss to measure the difference between the two, models the coordinates of the initial screening box corresponding to the object as a probability distribution, and predicts the final coordinate value by learning this distribution, thereby significantly improving the prediction accuracy and robustness of the first model.
[0088] For the target coordinate y (such as the y-coordinate of the center point), assume its true value lies between discrete values. and The mathematical expression for DFL loss is as follows:
[0089] in , For model prediction in and The discrete probabilities of the two locations force the model to learn the optimal distribution near the true coordinates through the loss function.
[0090] The CIOU (Complete Intersection over Union Loss) loss function is an advanced loss function suitable for bounding box regression in object detection. It uses an aspect ratio penalty term, jointly optimizes the overlap area, center point distance, and shape difference, and can significantly improve localization accuracy. The function formula is as follows:
[0091] in: IOU (Intersection over Union) is calculated by dividing the intersection of the predicted bounding box (A) and the ground truth bounding box (B) by their union. The formula is as follows:
[0092] It is the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box.
[0093] It is the diagonal length of the smallest enclosing rectangle.
[0094] It represents the aspect ratio difference between the predicted bounding box and the ground truth bounding box.
[0095] It is a weighting term used to balance the differences between IoU and aspect ratio.
[0096] For classification loss, BCE Loss (Binary Cross-Entropy Loss) can be used. It is a loss function for binary classification problems. When there are relatively few target categories and the number of samples is relatively balanced, using this loss function can greatly reduce the amount of computation.
[0097]
[0098] in: N is the batch size. It is the true label of the i-th sample. It is the predicted probability of the i-th sample.
[0099] Contrastive loss uses a ternary loss function, the goal of which is to make features with the same label as close as possible in spatial location, while features with different labels as far apart as possible in spatial location. In order to prevent the features of the samples from aggregating into a very small space, for two positive examples and one negative example of the same class, the negative example should be at least m (margin) farther away than the positive example.
[0100] in, and These represent the distances between the anchor sample and the positive sample, and between the anchor sample and the negative sample in the embedding space. It is a preset threshold used to control the difference between positive and negative samples. We want the anchor sample to be at least as large as the distance to the negative sample as the distance to the positive sample.
[0101] Loss function fusion: The three loss functions mentioned above are weighted and fused to calculate the final loss function. The weighting method uses linear weighting.
[0102]
[0103] Among them , , , The weights corresponding to each loss function.
[0104] Understandably, the role of the loss function is to guide the training direction of the first model.
[0105] The UAV detection and tracking method provided in this application accurately locates the target position through a detection and tracking algorithm and tracks the target in real time. Due to the limited computing power of UAV chips and the unsatisfactory tracking effect in complex environments, target tracking may be lost or the tracked object may switch. This application addresses this by adding a recognition network and integrating it into the detection network, using the backbone network of the detection model as the recognition network. This significantly reduces the computational complexity of the model while improving tracking accuracy. Furthermore, during inference, a portion of pre-selected targets are filtered through a threshold before applying loss to the detection model, greatly reducing the post-processing computation time.
[0106] This application also provides a drone detection and tracking device. The device can be a vehicle controller, or a control terminal or processor of an electronic device outside the vehicle, such as... Figure 3 As shown, the device includes: The shooting module 301 is configured to capture images of the environment.
[0107] The determination module 302 is configured to detect and recognize environmental images using a pre-trained first model, and determine multiple candidate objects.
[0108] Selection module 303 is configured to select the target object to be tracked from multiple candidate objects.
[0109] The control module 304 is configured to control the drone based on the real-time position of the target object.
[0110] In some optional embodiments, the apparatus further includes a training module 305, which is configured to: Acquire multiple sample images.
[0111] Extract the first feature vector from multiple sample images.
[0112] The first eigenvector is reduced in dimensionality to obtain the second eigenvector.
[0113] The first model is obtained by training using the second feature vector.
[0114] In some alternative embodiments, the process of extracting a first feature vector from multiple sample images may include: In step one, a lightweight deep learning convolutional network is used to obtain low-level features from multiple sample images to form a low-level feature map.
[0115] In step two, high-level features are extracted from multiple sample images to form a high-level feature map, and the corresponding semantic information is extracted.
[0116] In step three, the detection task is performed, which uses a Feature Pyramid Network (FPN). Through feature fusion, the semantic information of the high-level feature map is fused with the spatial information of the low-level feature map to generate feature representations with rich multi-scale information. These feature representations can form an intermediate map, thereby improving the detection performance of multi-scale targets, especially the detection capability of small targets, without significantly increasing the computational load.
[0117] In step four, the recognition task is performed, which involves extracting local branch features from the intermediate image obtained in the previous step using a horizontal segmentation method. By dividing the image region, fine-grained local semantic information is captured, and the local features are fused with the global features to obtain the final recognition feature vector, i.e., the first feature vector, thereby improving the model's robustness to viewpoint changes and occlusion.
[0118] In some optional embodiments, the trained first model can serve as both the backbone network of a detection model for image detection and a recognition model for identifying target objects in the image, thereby improving tracking accuracy. It is understood that "detecting an image" refers to obtaining the location information of the target in the image, i.e., coordinate values, while "recognizing an image" refers to obtaining the feature values of the target object, which is another numerical representation of the target object with higher dimensions. The complete network of the trained first model can be used for image detection, and the backbone network structure of this network is essentially equivalent to a recognition model network, with outputs being the feature values of the image. Therefore, during image detection, the output of the backbone network of the first model can be used as the result of image recognition. In other words, the method provided in this embodiment integrates the recognition network into the detection network, reducing the number of models used, lowering the computational load, improving detection and recognition efficiency, and thus improving tracking performance. In some optional embodiments, after obtaining the first model, structured pruning can be performed on the first model, deleting channels in the convolutional neural network of the first model with activation values less than a preset activation threshold, i.e., deleting unimportant convolutional channels, reducing the number of parameters and the model size without reducing accuracy.
[0119] In some optional embodiments, the training module 305 is also configured to: Obtain the regression loss and classification loss during the detection process, and the contrast loss during the recognition process.
[0120] The regression loss, classification loss, and contrast loss are weighted to obtain the final loss.
[0121] The first model is then corrected using the final loss.
[0122] In some optional embodiments, the regression loss uses two loss functions: CIOU and DFL, where: The DFL loss function is a loss function based on distribution learning. Its purpose is to make the predicted distribution as close as possible to the true distribution. It uses cross-entropy loss to measure the difference between the two, models the coordinates of the initial screening box corresponding to the object as a probability distribution, and predicts the final coordinate value by learning this distribution, thereby significantly improving the prediction accuracy and robustness of the first model.
[0123] For the target coordinate y (such as the y-coordinate of the center point), assume its true value lies between discrete values. and The mathematical expression for DFL loss is as follows:
[0124] in , For model prediction in and The discrete probabilities of the two locations force the model to learn the optimal distribution near the true coordinates through the loss function.
[0125] The CIOU (Complete Intersection over Union Loss) loss function is an advanced loss function suitable for bounding box regression in object detection. It uses an aspect ratio penalty term, jointly optimizes the overlap area, center point distance, and shape difference, and can significantly improve localization accuracy. The function formula is as follows:
[0126] in: IOU (Intersection over Union) is calculated by dividing the intersection of the predicted bounding box (A) and the ground truth bounding box (B) by their union. The formula is as follows:
[0127] It is the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box.
[0128] It is the diagonal length of the smallest enclosing rectangle.
[0129] It represents the aspect ratio difference between the predicted bounding box and the ground truth bounding box.
[0130] It is a weighting term used to balance the differences between IoU and aspect ratio.
[0131] For classification loss, BCE Loss (Binary Cross-Entropy Loss) can be used. It is a loss function for binary classification problems. When there are relatively few target categories and the number of samples is relatively balanced, using this loss function can greatly reduce the amount of computation.
[0132]
[0133] in: N is the batch size. It is the true label of the i-th sample. It is the predicted probability of the i-th sample.
[0134] Contrastive loss uses a ternary loss function, the goal of which is to make features with the same label as close as possible in spatial location, while features with different labels as far apart as possible in spatial location. In order to prevent the features of the samples from aggregating into a very small space, for two positive examples and one negative example of the same class, the negative example should be at least m (margin) farther away than the positive example.
[0135] in, and These represent the distances between the anchor sample and the positive sample, and between the anchor sample and the negative sample in the embedding space. It is a preset threshold used to control the difference between positive and negative samples. We want the anchor sample to be at least as large as the distance to the negative sample as the distance to the positive sample.
[0136] Loss function fusion: The three loss functions mentioned above are weighted and fused to calculate the final loss function. The weighting method uses linear weighting.
[0137]
[0138] Among them , , , The weights corresponding to each loss function.
[0139] Understandably, the role of the loss function is to guide the training direction of the first model.
[0140] In some alternative embodiments, the determining module 302 is further configured to: The first model, which was trained in advance, was used to detect and recognize environmental images, and multiple initial screening boxes were determined.
[0141] Confidence filtering is performed on multiple initial screening boxes to obtain multiple pre-selected boxes.
[0142] Identify multiple candidate objects that correspond one-to-one with multiple preselection boxes.
[0143] In some optional embodiments, a pre-trained first model is used to extract multi-scale features from the environmental image, thereby realizing the detection and recognition of the environmental image. The final output is a set of multiple preliminary screening boxes, each of which has a corresponding preliminary screening box coordinate point (x,y,w,h) and a preliminary screening box confidence score.
[0144] Each initial screening box contains an object detected and identified from the environmental image.
[0145] In some optional embodiments, a confidence threshold T{score} is preset, and the set of multiple initial screening boxes is filtered using the confidence threshold T{score}. The initial screening boxes with low confidence are filtered out, thereby filtering out objects with low confidence in the environmental image and reducing the amount of subsequent calculations.
[0146] In some optional embodiments, the confidence threshold T{score} = 0.25. In some optional embodiments, after performing confidence filtering on multiple initial screening boxes to obtain multiple pre-selected boxes, the remaining pre-selected boxes can be refined using the Distribution Focal Loss (DFL) algorithm to adjust the coordinates (indicating the size and position of the pre-selected boxes), thereby obtaining more accurate pre-selected boxes.
[0147] The DFL algorithm outputs the location of the bramble by integrating the discrete probability distribution. Traditional regression algorithms directly predict coordinate values, while the DFL algorithm discretizes the coordinate range (e.g., 0~255) into multiple intervals, outputs the probability of each interval, uses cross-entropy to concentrate the distribution near the actual location, and then weights and sums them to obtain the final coordinates of the preselected box. The DFL algorithm is more accurate in locating small targets, dense areas, and occluded scenes.
[0148] For the same object, multiple overlapping bounding boxes often appear after detection and recognition using a model. Therefore, in some optional embodiments, non-maximum suppression (NMS) can be further applied to these bounding boxes to remove duplicates. For multiple bounding boxes corresponding to the same object, NMS sorts them by confidence level, keeping only the box with the highest confidence level, thereby further reducing the computational load.
[0149] The UAV detection and tracking device provided in this application accurately locates the target position through a detection and tracking algorithm and tracks the target in real time. Due to the limited computing power of UAV chips and the unsatisfactory tracking effect in complex environments, target tracking may be lost or the tracked object may switch. This application significantly reduces the computational complexity of the model by adding a recognition network and integrating it into the detection network. Furthermore, during the inference process, a portion of the pre-selected targets are filtered by thresholding before applying loss to the detection model, which greatly reduces the post-processing computation time.
[0150] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the aforementioned drone detection and tracking method.
[0151] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned drone detection and tracking method.
[0152] This application also provides a vehicle that includes the aforementioned electronic equipment.
[0153] In this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.
[0154] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only.
[0155] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
[0156] The above is merely for the purpose of enabling those skilled in the art to understand the technical solution of this application, and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application shall be included within the scope of protection of this application.
Claims
1. A method for detecting and tracking unmanned aerial vehicles (UAVs), characterized in that, The method includes: Capture environmental images; The environmental image is detected and identified using a pre-trained first model, and multiple candidate objects are determined. Select the target object to be tracked from the plurality of candidate objects; The drone is controlled based on the real-time position of the target object.
2. The UAV detection and tracking method according to claim 1, characterized in that, Before capturing the environmental image, the method further includes: Acquire multiple sample images; Extract a first feature vector from the plurality of sample images; The first feature vector is reduced in dimensionality to obtain the second feature vector. The first model is obtained by training using the second feature vector.
3. The UAV detection and tracking method according to claim 2, characterized in that, The step of training the first model using the second feature vector includes: Obtain the regression loss and classification loss during the detection process, and the contrastive loss during the recognition process; The regression loss, the classification loss, and the contrast loss are weighted to obtain the final loss. The first model is then corrected using the final loss.
4. The UAV detection and tracking method according to claim 1, characterized in that, The process of using a pre-trained first model to detect and identify the environmental image, and determining multiple candidate objects, includes: The environmental image is detected and identified using a pre-trained first model to determine multiple initial screening boxes; Confidence filtering is performed on the multiple initial screening boxes to obtain multiple pre-selection boxes; Identify the multiple candidate objects that correspond one-to-one with the multiple preselection boxes.
5. A drone detection and tracking device, characterized in that, The device includes: The camera module is configured to capture images of the environment. The determination module is configured to detect and identify the environmental image using a pre-trained first model, and determine multiple candidate objects; The selection module is configured to select the target object to be tracked from the plurality of candidate objects; The control module is configured to control the drone based on the real-time position of the target object.
6. The UAV detection and tracking device according to claim 5, characterized in that, The device further includes a training module, which is configured to: Acquire multiple sample images; Extract a first feature vector from the plurality of sample images; The first feature vector is reduced in dimensionality to obtain the second feature vector. The first model is obtained by training using the second feature vector.
7. The UAV detection and tracking device according to claim 6, characterized in that, The training module is also configured to: Obtain the regression loss and classification loss during the detection process, and the contrastive loss during the recognition process; The regression loss, the classification loss, and the contrast loss are weighted to obtain the final loss. The first model is then corrected using the final loss.
8. The UAV detection and tracking device according to claim 5, characterized in that, The determining module is further configured to: The environmental image is detected and identified using a pre-trained first model to determine multiple initial screening boxes; Confidence filtering is performed on the multiple initial screening boxes to obtain multiple pre-selection boxes; Identify the multiple candidate objects that correspond one-to-one with the multiple preselection boxes.
9. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the UAV detection and tracking method according to any one of claims 1-4.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the UAV detection and tracking method according to any one of claims 1-4.