Method, device and equipment for detecting and tracking air target and medium

Abstract

The invention provides an aerial target detection and tracking method, device and equipment and a medium, and the method can comprise the steps: carrying out the detection of a current image frame through a trained YOLOv7 model which integrates an attention mechanism in a head network, and obtaining the position detection result of each target in the current image frame; correspondingly constructing an extended Kalman filter for each target; predicting a predicted position result of each target in the next image frame through a corresponding extended Kalman filter according to the detection result of each target, and calculating a intersection-union ratio according to the predicted position result of each target and the position detection result of each target in the current image frame; obtaining an optimal matching pair between the predicted position result of each target and the position detection result of each target by using a Hungary algorithm according to the intersection-to-union ratio; and tracking the position state of each target according to the optimal matching pair.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of target detection and tracking, and in particular to a method, device, equipment and medium for detecting and tracking an aerial target. Background Art

[0002] In the process of detecting and tracking aerial targets, related solutions usually adopt template matching algorithms to track aerial targets, use this algorithm to obtain the target's position information, and then obtain the deviation information between the target's centroid and the center of the field of view to complete the tracking control of aerial moving targets.

[0003] However, the template matching algorithm is very sensitive to changes in lighting and viewing angle. In the specific implementation and application of aerial target detection and tracking technology, its field of view angle is constantly changing and irregular, and at high altitudes, it faces the influence of clouds on lighting, which will greatly reduce the accuracy of the template matching algorithm. In addition, the relevant scheme needs to prepare templates in advance, which limits the targets and application scenarios it targets. Different templates need to be regenerated for new targets and application scenarios, which increases the workload and complexity. When the image contains noise or blur, it may cause large errors in the results of the template matching algorithm, further affecting the accuracy of target detection. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure are intended to provide a method, device and medium for detecting and tracking aerial targets, which can improve the accuracy of aerial target detection and tracking.

[0005] The technical solution of the embodiment of the present disclosure is implemented as follows: In a first aspect, the present disclosure provides a method for detecting and tracking an aerial target, the method comprising: Detect the current image frame using the trained YOLOv7 model that integrates the attention mechanism in the head network to obtain a position detection result of each target in the current image frame; Construct an extended Kalman filter for each target correspondence; Predicting the predicted position result of each target in the next image frame through the corresponding extended Kalman filter according to the detection result of each target, and calculating the intersection-over-union ratio according to the predicted position result of each target and the position detection result of each target in the current image frame; Obtaining an optimal matching pair between the predicted position result of each target and the position detection result of each target by using the Hungarian algorithm according to the intersection-union ratio; The position state of each target is tracked according to the optimal matching pair.

[0006] In a second aspect, the present disclosure provides a detection and tracking device for an aerial target, the device comprising: a detection part, a construction part, a prediction part and a matching part; wherein: The detection part is configured to detect the current image frame through the trained YOLOv7 model integrating the attention mechanism in the head network, and obtain the position detection result of each target in the current image frame; The construction part is configured to construct an extended Kalman filter for each target; The prediction part is configured to predict the predicted position result of each target in the next image frame through the corresponding extended Kalman filter according to the detection result of each target, and calculate the intersection-over-union ratio according to the predicted position result of each target and the position detection result of each target in the current image frame; The matching part is configured to obtain an optimal matching pair between the predicted position result of each target and the position detection result of each target by using the Hungarian algorithm according to the intersection-union ratio; And, tracking the position state of each target according to the optimal matching pair.

[0007] In a third aspect, the present disclosure provides a computing device, the computing device comprising: a communication interface, a memory and a processor; wherein: The memory is used to store a computer program that can be run on the processor; The processor is used to execute the aerial target detection and tracking method described in the first aspect when running the computer program.

[0008] In a fourth aspect, the present disclosure provides a computer storage medium storing at least one instruction, wherein the at least one instruction is used to be executed by a processor to implement the method for detecting and tracking aerial targets described in the first aspect.

[0009] The present invention provides a method, device, equipment and medium for detecting and tracking aerial targets. By introducing the attention mechanism into the YOLOv7 model, after the YOLOv7 model is used to detect the target, the Kalman filter is used to predict the position of the target in the next frame, and then cascade matching is performed, and finally the Hungarian algorithm is used to associate the optimal match, thereby improving the tracking effect, reducing the identity switching problem, and improving the accuracy of aerial target detection and tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of the architecture of a yolov7 model provided in the present invention.

[0011] Figure 2 A schematic diagram of a flow chart of a method for detecting and tracking an aerial target provided by the present invention.

[0012] Figure 3 A schematic diagram of a training process provided by the present invention.

[0013] Figure 4 A schematic diagram of the composition of an aerial target detection and tracking device provided by the present invention.

[0014] Figure 5 A schematic diagram of another aerial target detection and tracking device provided in the present invention.

[0015] Figure 6 A schematic diagram of the hardware structure of a computing device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure.

[0017] For ease of understanding, the following is an introduction to relevant terms and concepts that may be involved in the present disclosure.

[0018] See also Figure 1 , which shows the architecture of the YOLOv7 model used in Figure 1 As shown in , the YOLOv7 model includes: a backbone network, a neck network, and a head network. In some examples, the neck network can be combined with the head network to be called the head network.

[0019] Combination Figure 1 The modules shown in the solid box in the figure, the backbone network includes four CBS modules, one ELAN module, and three consecutive network output modules composed of MP modules and ELAN modules in sequence. In detail, the input image (input), for example, a 640×640×3 image, will form a 160×160×128 feature map after being processed by four CBS modules. After being processed by the ELAN module, the feature map will be processed by three consecutive network output modules to output feature maps of different scales.

[0020] In the neck network, the 32-fold downsampled feature map C5 output by the last network output module of the backbone network is firstly reduced in number of channels by the SPPCSP module. Then, based on the Feature Pyramid Network (FPN) structure, the feature channels and spatial positions are fused with the feature maps C3 and C4 output by the other two network output modules of the backbone network in a top-down order using two consecutive fusion modules consisting of an upsampling module, a Cat module, and an ELANW module, respectively, to obtain P3, P4, and P5. Then, based on the FPN structure, P3 is fused with P4 and P5 in a bottom-to-top order using two consecutive fusion modules consisting of an MP module, a Cat module, and an ELANW module, thereby obtaining three fused feature maps of different sizes.

[0021] In the head network, there are three fused feature maps of different sizes, and the output module composed of the REP module and the Conv module in sequence outputs three layers of feature maps of different sizes, and the detection results are obtained in each feature map.

[0022] based on Figure 1 The model architecture of YOLOv7 shown in the present disclosure expects to introduce an attention mechanism in the Neck network to highlight important features and suppress minor features, enhance the extraction of feature information of aerial targets, improve accurate positioning, and improve the tracking accuracy of controlled targets.

[0023] Based on this, see Figure 2 , which shows a method for detecting and tracking an aerial target provided by the present disclosure, the method comprising steps S201 to S205.

[0024] In step S201, the current image frame is detected by using the trained YOLOv7 model that integrates the attention mechanism in the head network to obtain the position detection result of each target in the current image frame.

[0025] In the present disclosure, Figure 1 In the architecture shown in the figure, in the neck network of the YOLOv7 model, an attention mechanism (CBAM) module is set between the SPPCSPC module and the first upsampling (UpSample) module, and between the ELANW module and the second upsampling (UpSample) module, as shown in the dotted box. Specifically, for each attention mechanism module, not only the attention of important feature channels is enhanced, but also the attention of important positions is enhanced. Based on the above improvements, combined with Figure 1In some possible implementations of the architecture shown in FIG. 1 , the current image frame is detected by the trained YOLOv7 model integrating the attention mechanism in the head network in step S201 to obtain the position detection result of each target in the current image frame, including: Inputting the current image frame into the trained YOLOv7 model integrating the attention mechanism in the head network; Performing step-by-step feature extraction and downsampling on the current image frame through the backbone network of the YOLOv7 model to obtain a multi-scale feature map corresponding to the current image frame; The multi-scale feature map is reconstructed by the neck network of the YOLOv7 model that conforms to the feature pyramid structure according to channel attention and spatial attention, and then feature fusion is performed to obtain comprehensive feature maps of different sizes; The comprehensive feature maps of different sizes are passed through the head network of the YOLOv7 model to perform target detection on the comprehensive feature maps of different sizes to obtain the position detection results of each target in the feature maps of different sizes.

[0026] For the above implementation, specifically, the detection result may include: the type of target, the confidence level, and the detection frame position of the target. Specifically, the target detection frame position is composed of (x, y, w, h), where (x, y) is the coordinate of the point in the upper left corner of the detection frame, w is the width of the detection frame, and h is the height of the detection frame.

[0027] For the above implementation, based on the setting of the attention mechanism module and the attention mechanism including the attention of the feature channel and the attention of the position. In some examples, the multi-scale feature map is reconstructed by the neck network of the YOLOv7 model that conforms to the feature pyramid structure according to the channel attention and the spatial attention, and then the feature fusion is performed to obtain comprehensive feature maps of different sizes, including: For the feature maps output by the SPPCSPC module and the ELANW module, important feature channels and positions are enhanced by the attention mechanism module to reconstruct the feature maps output by the SPPCSPC module and the ELANW module; The reconstructed feature maps are fused to obtain comprehensive feature maps of different sizes.

[0028] For the above example, specifically, the feature pyramid (FPN) structure of the neck network can refer to the aforementioned description of the Top-Down and Down-Top order of the neck network, which will not be repeated here. By adopting the FPN structure, the model can maintain high-precision detection when dealing with far and near targets or targets of different sizes, and the accuracy will not be reduced due to changes in target size. In addition, the multi-scale feature fusion based on the FPN structure increases the robustness of the model in complex scenarios and adapts to the detection needs of targets at different heights and distances.

[0029] For the attention mechanism (CBAM) module, in the exemplary implementation process, important feature channels generate channel attention weights through global average pooling and global maximum pooling to adjust the importance of each channel in the feature map, so that the YOLOv7 model pays more attention to the feature channels related to target detection. In addition, in the exemplary implementation process, for important spatial positions, a spatial attention map can be generated through convolution operations to focus on the important positions of the target in the image and suppress background noise. Based on the above explanation of the implementation process of the attention mechanism, the model can adaptively adjust the importance distribution of feature channels and spatial positions in the feature map, especially in complex backgrounds, to more accurately detect targets. It can also highlight key feature channels and spatial positions, so that the performance of the model in small target detection is significantly improved, reducing missed detections and false detections.

[0030] In step S202, an extended Kalman filter is constructed for each target.

[0031] It should be noted that after obtaining the detection result of the target in step S201, the present disclosure uses the deepsort algorithm for tracking. Based on the deepsort algorithm, a Kalman filter is generated for each target to predict its motion trajectory.

[0032] In step S203, the predicted position result of each target in the next image frame is predicted by the corresponding extended Kalman filter according to the detection result of each target, and the intersection-over-union ratio is calculated according to the predicted position result of each target and the position detection result of each target in the current image frame.

[0033] In the present disclosure, for each target, a Kalman filter is used to predict the position of the target in the next image frame (which can be represented by a tracking box), and the matching degree with the target detection box in the detection result is calculated. In some examples, the matching degree can be characterized by the intersection over union (IoU). Specifically, the intersection over union is the ratio of the intersection area of ​​the detection box and the tracking box to the union area of ​​the detection box and the tracking box.

[0034] In step S204, the optimal matching pair between the predicted position result of each target and the position detection result of each target is obtained by using the Hungarian algorithm according to the intersection-over-union ratio.

[0035] In the present disclosure, after obtaining the intersection-and-union ratio, the optimal matching can be performed using the Hungarian algorithm. In some possible implementations, the method of obtaining the optimal matching pair between the predicted position result of each target and the position detection result of each target using the Hungarian algorithm according to the intersection-and-union ratio includes: Obtaining a predicted position result that preliminarily matches the position detection result of each target in each predicted position result of each target based on the confidence according to the intersection-over-union ratio; The predicted position result of the preliminary matching is matched with the position detection result of each target to obtain the final predicted position result according to feature matching; The finally determined predicted position result of each target is data-associated with the position detection result of each target through the Hungarian algorithm to obtain the predicted position result that best matches the position detection result of each target.

[0036] For the above implementation, specifically, when there are multiple predicted unknown results for each target, a matching index can be obtained based on the intersection-over-union ratio, and the matching index can be based on the confidence level SIoU. Perform preliminary screening based on the matching index to obtain a preliminary matching predicted position result. Subsequently, feature matching is performed based on the position detection result of each target based on step S201 to obtain the final predicted position result from the preliminary matching predicted position result. The above matching process can also be referred to as a cascade matching process. It should be noted that the above cascade matching process reduces the number of detection boxes that need to be deep feature matched by staged screening, thereby improving the overall matching efficiency; in addition, the position and feature information are combined to ensure the uniqueness and accuracy of the matching, and reduce the occurrence of false matching and missed matching.

[0037] After the above cascade process, a matching matrix can be constructed, and the Hungarian algorithm is used to optimally match the final predicted position result and the position detection result of step S201 to update the position information and feature description of each target. It should be noted that the Hungarian algorithm can complete the matching of the target and the detection frame from a global optimal perspective, reducing false matches and missed matches.

[0038] Based on the above implementation, after obtaining the best matching pair, the predicted position result of the best matching can be output. If the best matching pair is not obtained, the tracking process needs to continue for a period of time. In some examples, the tracking process that needs to continue for a period of time may include: For any target, when the best matching predicted position result cannot be obtained, a target tracking cache is formed, and the best matching predicted position results in the subsequent set number of image frames are recorded; When a first number of subsequent image frames, for example, three, all obtain the best matching predicted position results, the matched target is confirmed as a new target; When a first number of subsequent image frames, for example, three, do not obtain the best matching predicted position result, it is determined that the target is lost, and the position information of the target is retained in the target tracking cache; When a second number of subsequent image frames, such as 60, do not obtain the most matching predicted position result of the target, the target tracking cache is cleared.

[0039] It should be noted that after clearing the target tracking cache, the target can be re-tracked through Kalman filtering and the running trajectory can be updated.

[0040] In step S205, the position state of each target is tracked according to the best matching pair. In the present disclosure, after obtaining the best matching pair, the position state of each target can be updated according to the predicted position result of the best match, and the target can be intuitively visualized in the form of a tracking box.

[0041] Based on the aforementioned technical solution, the present invention introduces the attention mechanism into the YOLOv7 model. After the YOLOv7 model is used to detect the target, the Kalman filter is used to predict the position of the target in the next frame, and then cascade matching is performed. Finally, the Hungarian algorithm is used to associate the optimal match, thereby improving the tracking effect, reducing the identity switching problem, and improving the accuracy of aerial target detection and tracking.

[0042] Based on the above technical solution, in the specific implementation process, the process of training the above improved YOLOv7 model can also be included, see Figure 3 , the training process may include: S301: Initialize a YOLOv7 model, wherein the neck network in the YOLOv7 model conforms to the feature pyramid structure and an attention mechanism is integrated into the neck network.

[0043] In this disclosure, Figure 1The architecture of the YOLOv7 model with the fusion attention mechanism shown in the figure initializes a YOLOv7 model that conforms to the architecture. Specifically, the neck network of the model is a feature pyramid (FPN) structure. In addition, in the neck network, an attention mechanism (CBAM) module is set between the SPPCSPC module and the first upsampling (UpSample) module, and between the ELANW module and the second upsampling (UpSample) module. Specifically, for each attention mechanism module, not only the attention of important feature channels is enhanced, but also the attention of important positions is enhanced. For the detailed description of the model architecture, please refer to the relevant content in the aforementioned technical solution, which will not be repeated here.

[0044] S302: Construct an image library of different types of aerial targets and a corresponding label library.

[0045] In the present disclosure, images of aerial targets in various background environments can be collected, and corresponding labels can be formed for the aerial targets in the images. The collected images are formed into an image library, and the labels corresponding to the targets in the images are formed into a label library. In some examples, these images can be complete high-definition images with an image size of 720×540, and the labels in the label library can be established in the PASCAL VOC format.

[0046] S303: Based on the aerial target images in the image library, processing is performed by data enhancement to increase the number of aerial target images in the image library; In the present disclosure, in order to increase the diversity of training data, the images of aerial targets collected in the image library can be processed by data enhancement to increase the number of images in the image library. For example, data enhancement operations such as rotation, filling cropping, and Mosaic can be randomly performed on any collected original image to form a new image. Specifically, rotation means rotating the collected original image according to a custom angle and direction to simulate targets at different perspectives. Filling cropping means partially cropping the collected original image and filling the cropped image to the same size as the original image to increase the situation of partially occluding the target; the Mosaic data enhancement operation is to randomly crop the selected original image and three random images and splice them into a picture as a new image to increase the complexity of the data set. It should be noted that the above-mentioned data enhancement operation is applied to the collected original image with a certain probability, and the same original image will be subjected to multiple data enhancement operations with a certain probability, which will not be described in detail in the present disclosure.

[0047] S304: Train the initialized YOLOv7 model according to the added image library and the corresponding label library, and construct a loss function based on SIoU during the training process, and update the parameters in the YOLOv7 model by calculating the loss function based on SIoU to obtain the trained YOLOv7 model that integrates the attention mechanism in the head network.

[0048] In the present disclosure, after the images in the image library are increased by the data augmentation operation, the increased image library and the corresponding label library can be divided into a training set and a test set at a ratio of 9: 1. The YOLOv7 model initialized in step S301 is trained using the back propagation algorithm through the training set, and during the training process, the currently trained model is saved every 50 steps (epochs), and the performance test is performed on the test set.

[0049] In the above training process, the parameter update method of the model exemplarily adopts the stochastic gradient descent (SGD), the initial learning rate is 0.01, the momentum coefficient is 0.937, the batch size is 32, and the IoU threshold of non-maximum suppression is 0.2. In addition, for the loss function used in the training process, the present disclosure exemplarily constructs the loss function with SIoU to characterize the difference in confidence between the training results and the true results. In some examples, the loss function can be calculated by the following formula:

[0050] in, It represents the intersection-over-union ratio between the predicted location box of the target and the actual location box of the target during training.

[0051] It should be noted that the conventional loss function based on CIoU, although taking into account the overlapping area, center point distance and aspect ratio of the predicted location box and the actual location box, compares the difference in aspect ratio between the two, rather than the difference in length, width and confidence between the two, which will hinder the optimization of the model during training. The present disclosure adopts a loss function based on SIoU, which not only reflects the difference in confidence, but also can effectively improve the regression speed and prediction accuracy.

[0052] Through the above training process, when the loss function converges, it can be said that the initialized YOLOv7 model has been trained and can be applied to the present disclosure. Figure 2 In the technical solution shown.

[0053] In order to verify the superiority of the improved YOLOv7 model, the corresponding detection algorithms used in each model were experimentally compared. Five mainstream detection algorithms were selected, namely Faster Region-based Convolutional Neural Network (Faster RCNN), Structured Data Distribution (SDD), YOLOv4, YOLOv5s, and YOLOv7, and compared with the improved YOLOv7 used in this solution. The results are shown in Table 1 below. It can be seen from the table that when the input image size is set to 640×640, the improved YOLOv7 detection algorithm has better detection accuracy than other algorithms while ensuring real-time detection. Compared with YOLOV4, the mean average precision (mAP) is increased by 12.73 percentage points, and the number of Params and floating point operations (FLOPs) are reduced by 2.48 respectively. , 2.19 Compared with YOLOv5s, the mAP of the improved YOLOv7 algorithm increased by 9.95 percentage points. In general, the improved YOLOv7 detection algorithm adds an attention module to the backbone network, which improves the feature extraction capability of the target, making the improved YOLOv7 model have greater advantages in the detection task. The comparative experimental results of different detection algorithms are shown in Table 1 below:

[0054] Table 1 In order to compare the tracking effect, four classic and high-performance target tracking algorithms were selected, namely the Recurrent Attention Network (RAN) tracking algorithm, the Point-o-Interest (POI) tracking algorithm, the Simple Online and Realtime Tracking (SORT) algorithm, and the Fair Multiple Object Tracking (FairMOT) algorithm, and were compared with the deepsort tracking algorithm used in this solution. The comparison results are shown in Table 2 below. It can be seen from the results that the deepsort tracking algorithm used in this solution is significantly better than other types of tracking algorithms due to its excellent detection performance. Among them, the multiple object tracking accuracy (MOTA) and multiple object tracking precision (MOTP) of the deepsort tracking algorithm reached 78.6% and 81.5% respectively, and the tracking speed reached 52 FPS, meeting the real-time requirements. The comparative experimental results of different tracking algorithms are shown in Table 2 below:

[0055] Table 2 Based on the same inventive concept as the above technical solution, see Figure 4 , which shows an aerial target detection and tracking device 40 provided by the present disclosure, the device 40 comprises: a detection part 401, a construction part 402, a prediction part 403 and a matching part 404; wherein, The detection part 401 is configured to detect the current image frame through the trained YOLOv7 model integrating the attention mechanism in the head network, and obtain the position detection result of each target in the current image frame; The construction part 402 is configured to construct an extended Kalman filter for each target; The prediction part 403 is configured to predict the predicted position result of each target in the next image frame through the corresponding extended Kalman filter according to the detection result of each target, and calculate the intersection-over-union ratio according to the predicted position result of each target and the position detection result of each target in the current image frame; The matching part 404 is configured to obtain an optimal matching pair between the predicted position result of each target and the position detection result of each target by using the Hungarian algorithm according to the intersection-union ratio; And, tracking the position state of each target according to the optimal matching pair.

[0056] In some examples, the detection portion 401 is configured to: Inputting the current image frame into the trained YOLOv7 model integrating the attention mechanism in the head network; Performing step-by-step feature extraction and downsampling on the current image frame through the backbone network of the YOLOv7 model to obtain a multi-scale feature map corresponding to the current image frame; The multi-scale feature map is reconstructed by the neck network of the YOLOv7 model that conforms to the feature pyramid structure according to channel attention and spatial attention, and then feature fusion is performed to obtain comprehensive feature maps of different sizes; The comprehensive feature maps of different sizes are passed through the head network of the YOLOv7 model to perform target detection on the comprehensive feature maps of different sizes to obtain the position detection results of each target in the feature maps of different sizes.

[0057] In some examples, in the neck network of the YOLOv7 model, an attention mechanism module is provided between the SPPCSPC module and the first upsampling module, and between the ELANW module and the second upsampling module; accordingly, the detection part 401 is configured as follows: For the feature maps output by the SPPCSPC module and the ELANW module, important feature channels and positions are enhanced by the attention mechanism module to reconstruct the feature maps output by the SPPCSPC module and the ELANW module; The reconstructed feature maps are fused to obtain comprehensive feature maps of different sizes.

[0058] In some examples, the matching portion 404 is configured to: Obtaining a predicted position result that preliminarily matches the position detection result of each target in each predicted position result of each target based on the confidence according to the intersection-over-union ratio; The predicted position result of the preliminary matching is matched with the position detection result of each target to obtain the final predicted position result according to feature matching; The finally determined predicted position result of each target is data-associated with the position detection result of each target through the Hungarian algorithm to obtain the predicted position result that best matches the position detection result of each target.

[0059] In some examples, the matching portion 404 is further configured to: For any target, when the best matching predicted position result cannot be obtained, a target tracking cache is formed, and the best matching predicted position results in the subsequent set number of image frames are recorded; When the first number of subsequent image frames all obtain the best matching predicted position results, the matched target is confirmed as a new target; When the first number of subsequent image frames do not obtain the best matching predicted position result, it is determined that the target is lost, and the position information of the target is retained in the target tracking cache; When a subsequent second number of image frames fails to obtain the most matching predicted position result of the target, the target tracking cache is cleared; wherein the second number is greater than the first number.

[0060] For some examples, see Figure 5 , the apparatus 40 further comprises a training part 405, configured to: Initialize a YOLOv7 model, wherein the neck network in the YOLOv7 model conforms to the feature pyramid structure and an attention mechanism is integrated into the neck network; Build image libraries of different types of aerial targets and corresponding label libraries; Based on the aerial target images in the image library, processing is performed by data enhancement to increase the number of aerial target images in the image library; The initialized YOLOv7 model is trained according to the increased image library and the corresponding label library, and a loss function is constructed based on SIoU during the training process. The parameters in the YOLOv7 model are updated by calculating the loss function based on SIoU to obtain the trained YOLOv7 model that integrates the attention mechanism in the head network; wherein the loss function based on SIoU is used to express the difference in confidence between the training result and the true result.

[0061] Please refer to Figure 6 , which shows a hardware structure diagram of a computing device 60 provided by an exemplary embodiment of the present disclosure. In some examples, the computing device 60 may be at least one of a smart phone, a smart watch, a desktop computer, a laptop, a virtual reality terminal, an augmented reality terminal, a wireless terminal, and a laptop portable computer. The computing device 60 has a communication function and can access a wired network or a wireless network. The computing device 60 may generally refer to one of a plurality of terminals, and those skilled in the art may know that the number of the above terminals may be more or less. In some examples, the computing device 60 may receive data based on the wired network or wireless network to which it is connected. It can be understood that the computing device 60 undertakes the calculation and processing work of the technical solution of the present disclosure, and the embodiments of the present disclosure are not limited to this.

[0062] like Figure 6 As shown, the computing device 60 in the present disclosure may include one or more of the following components: a processor 610 and a memory 620 .

[0063] Optionally, the processor 610 uses various interfaces and lines to connect various parts within the entire computing device 60, and executes various functions and processes data of the computing device 60 by running or executing instructions, programs, code sets or instruction sets stored in the memory 620, and calling data stored in the memory 620. Optionally, the processor 610 can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 610 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), and a baseband chip. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the touch display; the NPU is used to implement artificial intelligence (AI) functions; and the baseband chip is used to process wireless communications. It is understandable that the above baseband chip may not be integrated into the processor 610, but may be implemented by a separate chip.

[0064] The memory 620 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 620 includes a non-transitory computer-readable storage medium. The memory 620 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 620 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above various method embodiments, etc.; the data storage area may store data created according to the use of the computing device 60, etc.

[0065] In addition, those skilled in the art can understand that the structure of the computing device shown in the above drawings does not constitute a limitation on the computing device, and the computing device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, the computing device also includes a display screen, a camera assembly, a microphone, a speaker, a radio frequency circuit, an input unit, a sensor (such as an acceleration sensor, an angular velocity sensor, an optical fiber sensor, etc.), an audio circuit, a WiFi module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

[0066] The embodiments of the present disclosure also provide a computer-readable storage medium storing at least one instruction, wherein the at least one instruction is used to be executed by a processor to implement the aerial target detection and tracking method as described in the above embodiments.

[0067] The embodiments of the present disclosure also provide a computer program product, which includes computer instructions stored in a computer-readable storage medium; a processor of a computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computing device executes to implement the detection and tracking methods of aerial targets described in the above-mentioned embodiments.

[0068] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present disclosure can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0069] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A method for detecting and tracking an aerial target, characterized in that: The method comprises: Detect the current image frame using the trained YOLOv7 model that integrates the attention mechanism in the head network to obtain a position detection result of each target in the current image frame; Construct an extended Kalman filter for each target correspondence; Predicting the predicted position result of each target in the next image frame through the corresponding extended Kalman filter according to the detection result of each target, and calculating the intersection-over-union ratio according to the predicted position result of each target and the position detection result of each target in the current image frame; Obtaining an optimal matching pair between the predicted position result of each target and the position detection result of each target by using the Hungarian algorithm according to the intersection-union ratio; The position state of each target is tracked according to the optimal matching pair.

2. The method according to claim 1, characterized in that The detecting the current image frame by using the trained YOLOv7 model integrating the attention mechanism in the head network to obtain the position detection result of each target in the current image frame includes: Input the current image frame into the trained YOLOv7 model integrating the attention mechanism in the head network; Performing step-by-step feature extraction and downsampling on the current image frame through the backbone network of the YOLOv7 model to obtain a multi-scale feature map corresponding to the current image frame; The multi-scale feature map is reconstructed by the neck network of the YOLOv7 model that conforms to the feature pyramid structure according to channel attention and spatial attention, and then feature fusion is performed to obtain comprehensive feature maps of different sizes; The comprehensive feature maps of different sizes are passed through the head network of the YOLOv7 model to perform target detection on the comprehensive feature maps of different sizes to obtain the position detection results of each target in the feature maps of different sizes.

3. The method according to claim 2, characterized in that In the neck network of the YOLOv7 model, an attention mechanism module is set between the SPPCSPC module and the first upsampling module, and between the ELANW module and the second upsampling module; accordingly, the multi-scale feature map is reconstructed according to the channel attention and spatial attention through the neck network of the YOLOv7 model that conforms to the feature pyramid structure, and then feature fusion is performed to obtain comprehensive feature maps of different sizes, including: For the feature maps output by the SPPCSPC module and the ELANW module, important feature channels and positions are enhanced by the attention mechanism module to reconstruct the feature maps output by the SPPCSPC module and the ELANW module; The reconstructed feature maps are fused to obtain comprehensive feature maps of different sizes.

4. The method according to claim 1, characterized in that: The method of obtaining the optimal matching pair between the predicted position result of each target and the position detection result of each target by using the Hungarian algorithm according to the intersection-over-union ratio includes: Obtaining a predicted position result that preliminarily matches the position detection result of each target in each predicted position result of each target based on the confidence according to the intersection-over-union ratio; The predicted position result of the preliminary matching is matched with the position detection result of each target to obtain the final predicted position result according to feature matching; The finally determined predicted position result of each target is data-associated with the position detection result of each target through the Hungarian algorithm to obtain the predicted position result that best matches the position detection result of each target.

5. The method according to claim 4, characterized in that The method further comprises: For any target, when the best matching predicted position result cannot be obtained, a target tracking cache is formed, and the best matching predicted position results in the subsequent set number of image frames are recorded; When the first number of subsequent image frames all obtain the best matching predicted position results, the matched target is confirmed as a new target; When the first number of subsequent image frames do not obtain the best matching predicted position result, it is determined that the target is lost, and the position information of the target is retained in the target tracking cache; When a subsequent second number of image frames fails to obtain the most matching predicted position result of the target, the target tracking cache is cleared; wherein the second number is greater than the first number.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Initialize a YOLOv7 model, wherein the neck network in the YOLOv7 model conforms to the feature pyramid structure and an attention mechanism is integrated into the neck network; Build image libraries of different types of aerial targets and corresponding label libraries; Based on the aerial target images in the image library, processing is performed by data enhancement to increase the number of aerial target images in the image library; The initialized YOLOv7 model is trained according to the increased image library and the corresponding label library, and a loss function is constructed based on SIoU during the training process. The parameters in the YOLOv7 model are updated by calculating the loss function based on SIoU to obtain the trained YOLOv7 model that integrates the attention mechanism in the head network; wherein the loss function based on SIoU is used to express the difference in confidence between the training result and the true result.

7. A detection and tracking device for an aerial target, characterized in that: The device comprises: a detection part, a construction part, a prediction part and a matching part; wherein, The detection part is configured to detect the current image frame through the trained YOLOv7 model integrating the attention mechanism in the head network, and obtain the position detection result of each target in the current image frame; The construction part is configured to construct an extended Kalman filter for each target; The prediction part is configured to predict the predicted position result of each target in the next image frame through the corresponding extended Kalman filter according to the detection result of each target, and calculate the intersection-over-union ratio according to the predicted position result of each target and the position detection result of each target in the current image frame; The matching part is configured to obtain an optimal matching pair between the predicted position result of each target and the position detection result of each target by using the Hungarian algorithm according to the intersection-union ratio; And, tracking the position state of each target according to the optimal matching pair.

8. The device according to claim 7, characterized in that The apparatus further comprises a training portion configured to: Initialize a YOLOv7 model, wherein the neck network in the YOLOv7 model conforms to the feature pyramid structure and an attention mechanism is integrated into the neck network; Build image libraries of different types of aerial targets and corresponding label libraries; Based on the aerial target images in the image library, processing is performed by data enhancement to increase the number of aerial target images in the image library; The initialized YOLOv7 model is trained according to the increased image library and the corresponding label library, and a loss function is constructed based on SIoU during the training process. The parameters in the YOLOv7 model are updated by calculating the loss function based on SIoU to obtain the trained YOLOv7 model that integrates the attention mechanism in the head network; wherein the loss function based on SIoU is used to express the difference in confidence between the training result and the true result.

9. A computing device, characterized in that The computing device comprises: a processor and a memory; wherein, The memory is used to store a computer program that can be run on the processor; The processor is used to execute the aerial target detection and tracking method according to any one of claims 1 to 6 when running the computer program.

10. A computer storage medium, characterized in that: The storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the detection and tracking method of an aerial target as described in any one of claims 1 to 6.

Images