Lightweight multi-terminal deployment remote sensing warship detection algorithm

Abstract

The invention provides a lightweight multi-terminal deployment remote sensing ship and aircraft detection algorithm, which is a remote sensing image target detection lightweight algorithm based on an RT-DETR improved YOLOv8. The core improvement comprises the steps of designing an S2A2C2 lightweight feature extraction network, and fusing SAConv and AIFI self-attention to improve local and global feature extraction capability; a cross-scale fusion network is constructed, and multi-scale information is efficiently integrated with low complexity; an FIMPDIOU loss function is provided, and the detection effect of the small target in the complex background is adjusted and optimized through adaptive parameters; and an SELayerV2 attention mechanism is embedded to enhance the feature expression ability. Experimental results show that the precision of the model is improved by 1.7%, the parameter quantity and the calculated quantity are respectively reduced by 56.7% and 59.3%, the model is successfully deployed on Jetson Orin, Windows, Ubuntu and Android platforms, the characteristics of good practicability and easy deployment of the model are verified, and an effective thought is provided for lightweight target detection.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and proposes a lightweight, multi-terminal deployment algorithm for remote sensing ship and aircraft detection. It improves YOLOv8's remote sensing ship and aircraft image target detection based on RT-DETR, and deploys it on the Jetson Orin platform and Android. Background Technology

[0002] Target detection is a fundamental visual task involving the identification and localization of targets in images. Deep learning-based target detection algorithms dominate the field. The core issue is not hardware, but the quality of the algorithm itself. Remote sensing aircraft and ship target detection is a crucial application in remote sensing technology, but it faces several technological limitations. Firstly, there is a shortage of data quality and quantity. While a large amount of aerial, shipborne, and satellite data for remote sensing images exists, these data are often limited by factors such as resolution and remote sensing bands, making target detection challenging. Secondly, identification is difficult in complex environments. In complex conditions involving overlapping land and sea areas and drastic changes in lighting, traditional target detection algorithms often struggle to effectively distinguish target information. Furthermore, there are real-time requirements. Fields such as aerial and marine monitoring demand high levels of real-time acquisition and processing of target information, and current technologies still fall short in this aspect.

[0003] To address these issues, a series of improvements are needed. These include increasing the resolution of remote sensing data and adding band information to improve data quality and quantity, thereby enhancing the accuracy of target detection. Utilizing new technologies such as deep learning, combined with optimized target detection algorithms, will improve target recognition capabilities in complex environments. Furthermore, more efficient real-time processing technologies need to be developed to meet the demands for rapid acquisition and processing of target information. With the release of YOLOv8, the field of computer vision has made significant progress; this model defines a new level of technology for target detection and instance segmentation.

[0004] As accuracy improves, the demand for computing resources increases, leading to significant memory consumption and higher costs. The core of lightweight network design is to reduce the computational and spatial complexity of the model while maintaining accuracy as much as possible. This allows deep neural networks to be deployed on embedded edge devices with limited computing power and storage space, resulting in a significant improvement in detection speed and enabling a leap from academia to industry. Popular lightweight network designs include MobileNet, ShuffleNet, and SqueezeNet. The introduction of this invention will significantly improve the accuracy of target detection in complex remote sensing environments involving ships and aircraft, facilitating more precise acquisition of target information. It can meet the needs of real-time acquisition and processing of target information, improve the timeliness of monitoring feedback, and help expand the application scenarios of remote sensing technology in fields such as aviation and marine monitoring. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention proposes a lightweight, multi-terminal deployment algorithm for remote sensing ship and aircraft detection, specifically addressing the application requirements of target optical identification for remote sensing ship and aircraft classification. Based on RT-DETR, this lightweight algorithm for target detection in remote sensing ship and aircraft images using improved YOLOv8 achieves a 1.7% increase in detection accuracy, a 56.7% reduction in parameter count, and a 59.3% reduction in GFLOPs. Deployed on Windows 10, Ubuntu 20, and Android 12, the algorithm demonstrates low deployment cost and ease of use, providing a new approach for lightweight networks.

[0006] The technical solution adopted in this invention is a lightweight algorithm for target detection in remote sensing ship and aircraft images based on RT-DETR and improved YOLOv8, deployed on the Jetson Orin platform and Android. It includes the following steps:

[0007] Step 1: Using publicly available information, images, and publicly available remote sensing and visible datasets online, create a dataset containing different types of remote sensing ships and aircraft, manually label different categories, and divide it into a training set (train), a test set (val), and a validation set (test) in a ratio of 8:1:1.

[0008] Step 2: To address the issues of model compression and lightweighting, this invention proposes a lightweight feature extraction network S2A2C2 based on the structure of ShuffletNetv2, replacing the network's backbone.

[0009] Step 3: To address the issue of poor detection accuracy in lightweight networks, this invention proposes a lightweight cross-scale feature map fusion network as the neck part of the network. This network aims to effectively integrate feature layer information from different scales while maintaining low time complexity, thereby accelerating image prediction computation.

[0010] Step 4: This invention proposes the FIMPDIoU loss function, replacing the network's bounding box regression loss function. This focuses on targets that are easily overlooked in complex backgrounds, improving the model's performance in target detection and recognition. The loss prediction box is adjusted using a ratio to accommodate targets in datasets of different sizes.

[0011] Step 5: In addition, the present invention adds the SENetV2 attention mechanism to the three extracted output feature layers P3, P4 and P5 to improve the detection accuracy of the model for large, medium and small targets in visible and remote sensing ship and aircraft images.

[0012] Step 6: Train the improved lightweight algorithm on Windows 10 using the training set `train`, adjust the hyperparameters using the test set `val`, and promptly assess the learning status. Evaluate the generalization ability of the algorithm using the validation set `test`. Deploy the trained model to Windows 10, Ubuntu 20, and Android 12 for real-time image prediction.

[0013] Furthermore, in step 1, a dataset of visible and remote sensing images containing different types of ships and aircraft is constructed and divided into a training set, a test set, and a validation set in an 8:1:1 ratio, as follows:

[0014] Using publicly available online remote sensing datasets and images, after manual annotation and cleaning to remove useless images and data, 8427 images were obtained. These were divided into training, test, and validation sets in approximately an 8:1:1 ratio: 6825 images in the training set, 759 in the test set, and 843 in the validation set. The improved lightweight algorithm was trained using the training set (train), learning from the training data to continuously reduce training error. The validation set (val) serves as the test set during training, allowing users to monitor the learning progress and adjust hyperparameters to monitor for overfitting. The validation set is used after training to evaluate the model's performance and assess its generalization ability.

[0015] Furthermore, in step 2, the performance of the feature extraction network is improved based on the four network criteria of ShuffleNet V2. During feature extraction, the number of input channels is kept as equal as possible to minimize memory access costs. Simultaneously, group convolutions are reduced, branching and element-wise operations are decreased, and the complexity of the network structure is reduced, as detailed below:

[0016] Based on the structure of each layer in ShuffleNet V2, a "channel split" operation is introduced. Before each unit begins, the input channels are divided into two branches, one of which maintains an identity mapping. The other branch involves convolutions with the same input and output channels. Then, the two branches are fused at the channel level, followed by a channel shuffling operation to enhance the network's information exchange capabilities. For spatial downsampling operations, the channel split operation is removed, and the number of output channels is doubled.

[0017] In ShuffleNet V2, switchable atrous convolutions (SACs) are added to the features P3, P4, and P5 extracted from each stage to enhance feature extraction in object detection tasks. SACs apply different dilation rates to the same input features. Dilated convolutions expand the receptive field by introducing additional space into the convolution kernel without increasing the number of parameters or computational cost. Furthermore, using a spatially dependent switching function to combine the results of convolutions with different dilation rates means that each location in the feature map may have a different switch to control the SAC output, increasing the network's flexibility.

[0018] AIFI is used to process high-level features P5. The resulting feature map is then processed by a 1×1 Conv network in conjunction with a lightweight feature map fusion network to adjust the width and depth of the feature map. AIFI uses only one layer of a standard Transformer encoder, applying self-attention operations to higher-level features with richer semantic concepts. This allows it to capture the relationships between conceptual entities in the image, facilitating subsequent modules for object detection and recognition. Combining AIFI and Conv replaces YOLOv8's SPPF, providing a more comprehensive and in-depth understanding of features.

[0019] Furthermore, in step 3, this invention proposes a lightweight cross-scale feature map fusion network as the neck of the network, connecting the backbone and the head network, responsible for feature fusion and processing, in order to improve the accuracy and efficiency of detection, as detailed below:

[0020] Considering that deep features contain richer information, different processing methods are used to handle deep and shallow features. Globally normalized convolutions are used to process deep features to maintain their integrity and consistency. For shallow features, traditional 1x1 convolution operations are chosen, as shallow features are richer in detail and require more attention to subtle local feature changes.

[0021] A lightweight fusion module is also proposed in the lightweight cross-scale feature map fusion network. This module integrates features from different scales through fusion operations to enhance the model's adaptability to scale changes and its ability to detect small-scale objects. This module includes GSConv, Conv, and VoV-GSCSP modules. The shape of the convolution is mainly controlled by two parameters: the number of input channels c1 and the number of output channels c2. Let h and w be the spatial size of the feature map, then the computational cost (FLOPs) of a 1x1 convolution is: B = hwc1c2, and the storage access cost is: The computational cost (FLOPs) of a 1x1 GSconv is: So

[0022] After processing, deep and shallow features are concatenated to form a richer feature representation. This fully leverages the advantages of both deep and shallow features, thereby improving the model's ability to perceive features at different scales. Information fusion via VoV-GSCSP utilizes feature information from different scales to enhance image details and contrast, improving the model's performance in cross-scale tasks.

[0023] Furthermore, in step 4, the loss function FIMPDIoU proposed in this invention is an MPDIoU that integrates the Inner and Focal ideas, improving detection accuracy, focusing on target details, and comprehensively considering overlapping areas, center point distance, and width and height deviations, thereby improving the performance and accuracy of YOLOv8 for target detection in images. Specifically, as follows:

[0024] Because traditional IoU (Intersection over Union) calculates the overall overlap between the predicted and ground truth bounding boxes, it's difficult for traditional Bounding Box Regression (BBR) loss functions to distinguish between predicted and ground truth boxes that have the same aspect ratio but different dimensions. Existing Bounding Box Regression metrics include GIoU, DIoU, CIoU, EIoU, and MPDIoU. YOLOv8's default loss function for object detection is CIOU_LOSS, which uses Non-Maximum Suppression (NMS) to search for local maxima and suppress redundant boxes during detection. The formula for calculating CIOU_LOSS is as follows: The Intersection over Union (IOU) ratio measures the degree of overlap between two bounding boxes. A is the area of ​​the intersection of the two frames, and B is the area of ​​the union of the two frames. Distance C is the length of the diagonal of the smallest bounding rectangle of the two frames. The Euclidean distance Distance2 between the center points of the graph is used to measure the distance between the center points of two boxes. v is a parameter that measures the consistency of aspect ratio. Where w and h represent the width and height of the diagram, respectively.

[0025] MPDIoU simplifies the similarity comparison between two bounding boxes based on other metrics, and can adapt to bounding box regression with or without overlap. It aims to compute the loss by directly minimizing the keypoint distance between the predicted and ground truth boxes, utilizing the minimum point distance between two axis-aligned rectangles. Its minimization loss function is expressed as follows: in:

[0026] Inner-IoU improves the accuracy of bounding box regression (BBR) in object detection by introducing auxiliary bounding boxes, providing a more precise evaluation of overlapping regions. Furthermore, by controlling the size of the auxiliary bounding boxes, fine-tuning is possible for different datasets and detection tasks. For high IoU samples, using smaller auxiliary boxes accelerates model learning; while for low IoU samples, using larger auxiliary boxes improves regression performance. The variable "ratio" corresponds to the scaling factor, typically ranging from [0.5, 1.5]. Its minimization loss function is expressed as follows:

[0027] Focal Loss modifies the cross-entropy loss by increasing an adjustment factor, reducing the loss value for correctly classified samples, thus focusing the model's training on harder-to-classify samples. Its specific form is: make Therefore, it can be unified as follows: Where α i Used to solve the problem of imbalanced positive and negative samples, p i p reflects the degree of proximity to the ground truth, i.e., category y. i A larger value indicates a closer match to category y, meaning more accurate classification, which helps address the imbalance between easy and difficult samples. γ > 0 is an adjustable factor. Typically, γ = 0.5 is chosen.

[0028] Applying the concepts of Inner and Focal to MPDIoU improves the accuracy of image object detection. It focuses on the overlapping areas within the bounding box, emphasizing the core parts of the bounding box rather than the whole, which is beneficial for improving the detection performance of small targets or targets easily overlooked in complex backgrounds. Especially when the distance between the sensor and the target is large, the target's image area on the focal plane is small, and the signal-to-noise ratio is low, making the target easily obscured by noise. Therefore, FIMPDIoU is considered to allow the model to "focus" on learning samples that are more critical to improving overall performance.

[0029] In FIMPDIoU, the keypoint distance is the minimum point distance between the auxiliary predicted box and the ground truth box. The cross-entropy loss also focuses on the intersection-union ratio of the auxiliary predicted box and the ground truth box. All factors, such as the overlap area, width, and height deviation, can be determined by the coordinates of the top left and bottom right points of the auxiliary predicted box and the ground truth box.

[0030] During the training phase, each predicted box By minimizing the loss function Forcing it to approach its true bounding box The derivation of the FIMPDIoU minimization loss function is as follows: When the Inner concept is applied to MPDIoU, the minimization loss function is: When the Focal idea is applied to MPDIoU, the minimization loss function is: The loss function minimized by FIMPDIoU is as follows:

[0031] As the formula shows, the value of the ratio also affects the calculation of the loss function FIMPDIoU. Compared to IoU loss, when the ratio is less than 1, the auxiliary bounding box size is smaller than the actual bounding box, and its effective regression range is smaller than that of IoU loss. However, its absolute gradient value is greater than that obtained by IoU loss, which can accelerate the convergence of high IoU samples. Conversely, when the ratio is greater than 1, the larger-scale auxiliary bounding box expands the effective regression range, providing a benefit for low IoU regression. The value of the ratio also varies for different datasets.

[0032] Furthermore, in step 5, the SENetV2 attention mechanism is added to the three extracted output feature layers P3, P4, and P5 to improve the model's detection accuracy for large, medium, and small targets in visible and remotely sensed ship and aircraft images. Specifically:

[0033] Adding the SENet V2 attention mechanism to each layer of the fused features yielded the best results; the key operations are squeeze and excitation. The SaE module optimizes the transmission of key features by increasing the cardinality between layers. Through automatic learning, it uses another new neural network to obtain the importance of each channel of the feature map, and then uses this importance to assign a weight value to each feature, thus allowing the neural network to focus on certain feature channels. Experiments show that using SENet V2 improves the model's ability to detect objects in images, enhances the channels of feature maps useful for the current task, and suppresses feature channels that are not useful for the current task.

[0034] Further, in step 6, the improved lightweight algorithm is trained on Windows 10 using the training set `train`, and the hyperparameters are adjusted using the test set `val` to promptly assess the learning status. The generalization ability of the algorithm is evaluated using the validation set `test`. The trained model is then deployed to Windows 10, Ubuntu 20, and Android 12 for real-time image prediction to verify the lightweight nature of the algorithm. Details are as follows:

[0035] The improved lightweight algorithm is trained on Windows 10 using the training set `train`, and the hyperparameters are tuned using the test set `val` to assess the learning progress. The validation set `test` is used to evaluate the algorithm's performance and generalization ability. To test the model's final generalization ability—that is, to evaluate the machine learning algorithm's ability to handle samples beyond what it has seen before—the training and test sets must be distinct. The test set is used to evaluate the model's generalization ability, while the validation set is used to evaluate the model's performance during training; neither set can be used for hyperparameter tuning or training.

[0036] Configure the Android Studio environment and JDK on Windows 10 and download the corresponding project version. Deploy to Ubuntu 20; training and testing can be done on the Jeston Orin platform, mainly involving environment configuration. Deploying to Android 12 requires modifying the configuration files to convert the custom dataset and the improved model's pre-trained .pt files on Windows to ONNX files, and further convert them to ncnn format. Deploy the ncnn format to Android 12 by finding and downloading the Google USB Driver. The process is: File -> Settings -> Languages ​​& Frameworks -> Android SDK -> SDK Tools -> Google USB Driver. On your phone, enable Developer Mode and USB debugging permissions. Connect your phone and computer via USB, allow debugging, and select file transfer mode. The connected phone should then appear in the Android Studio device selector. Run the Android Studio project on your phone; the front-facing camera will be used by default for real-time detection.

[0037] The present invention will now be further described with reference to the accompanying drawings. Attached Figure Description

[0038] Figure 1 This is a schematic diagram illustrating the implementation process of the network structure of the present invention.

[0039] Figure 2 This is a schematic diagram of the improved YOLOv8 lightweight algorithm structure based on RT-DETR.

[0040] Figure 3 This is a schematic diagram of the lightweight feature extraction network S2A2C2.

[0041] Figure 4 This is a schematic diagram of the related convolutions in a lightweight cross-scale feature map fusion network.

[0042] Figure 5This is a schematic diagram of the FIMPDIOU loss function.

[0043] Figure 6 The final test results of this invention deployed on three platforms: Windows 10, Ubuntu 20, and Android 12 are shown in the image. Detailed Implementation

[0044] To better understand the technical content of the present invention, specific embodiments are described below in conjunction with the accompanying drawings.

[0045] This invention provides a lightweight, multi-device-deployable remote sensing ship and aircraft detection algorithm. Based on RT-DETR and an improved YOLOv8 algorithm, this lightweight algorithm for target detection in remote sensing ship and aircraft images is deployed on Windows 10, Ubuntu 20, and Android 12, and includes the following steps:

[0046] Step 1: Using publicly available remote sensing datasets and images, select images related to remote sensing of ships and aircraft, and manually label different types of ship or aircraft targets. After mixing and cleaning to remove invalid data, retain the original dataset of 8524 images. The total number of labeled images in the manually labeled dataset is 8427. These are divided into training, test, and validation sets in approximately an 8:1:1 ratio: 6825 images for the training set, 759 for the test set, and 843 for the validation set.

[0047] Figure 1 This is a schematic diagram of the overall network flow for implementing the method of this invention. This invention considers RT_DETR, which does not require NMS post-processing. Compared to the YOLO series detectors, the DETR series detectors are slower and have larger model sizes, which contradicts the goal of lightweight design. These issues prompted this invention to explore real-time end-to-end detectors, aiming to utilize the excellent network architecture of RT_DETR and other lightweight convolutional models, combined with YOLOv8, to design a lightweight real-time detection network algorithm. First, a lightweight feature extraction network S2A2C2 is proposed, improving the flexibility and adaptability of feature extraction, and using self-attention plus convolution to replace SPPF, maintaining high detection performance. Second, a lightweight cross-scale feature map fusion network is proposed, effectively integrating detailed features and feature layer information at different scales, accelerating image prediction computation. Furthermore, the SENetV2 attention mechanism is added to the obtained feature layer information to improve the model's object detection accuracy. Finally, the FIMPDIOU loss function is proposed, focusing on targets easily overlooked in complex backgrounds, improving the model's performance in object detection and recognition. This addresses the speed delay issue that NMS causes to real-time detectors at its root, and provides better target detection capabilities for images under conditions such as nighttime, low light, and occlusion.

[0048] Figure 2 This is a schematic diagram of the specific framework of the improved YOLOv8 lightweight algorithm based on RT-DETR.

[0049] On the input side, Mosaic data augmentation, adaptive anchor box calculation, and minimum black border fill scaling were used.

[0050] The backbone network is replaced by a lightweight feature extraction network, S2A2C2. S2A2C2 is based on the ShuffleNetv2 architecture and incorporates lightweight SAConv convolutions to extract features, adding image-level information to the features and improving accuracy and efficiency in object detection and segmentation tasks. Simultaneously, AIFI+Conv replaces the original SPPF, flattening 2D features into vectors for feature map-level fusion of local and global features. The output is then adjusted back to 2D to facilitate subsequent cross-scale feature fusion.

[0051] Secondly, the network's neck structure was improved by proposing a lightweight cross-scale feature map fusion network. A lightweight fusion module was also introduced, utilizing CCFM combined with GSConv and VoV-GSCSP modules to fully leverage multi-scale features and accelerate target prediction and segmentation computation. The SENetV2 attention mechanism was added to the three extracted output feature layers P3, P4, and P5 to improve the model's detection accuracy for large, medium, and small targets in images.

[0052] Finally, this invention proposes FIMPDIoU, replacing the network's bounding box regression loss function. In actual training, DFL and FIMPDIoU losses are used in combination. DFL calculates the loss on the "bounding box probability distribution" and the "label probability distribution" in the form of cross-entropy, thereby optimizing each edge. FIMPDIoU loss, on the other hand, restores the "bounding box probability distribution" to the predicted box, and optimizes the overall predicted box by calculating the loss on the "actual box" of the predicted box and the label, introducing the concepts of Focal and Inner loss.

[0053] Multi-scale training is performed. During the model pre-training phase, a small learning rate is first used to train for a few epochs or steps, and then the learning rate is adjusted to a pre-set learning rate for warmup training. A cosine learning rate descent strategy is also used. EMA is employed to update weights, essentially giving the parameters momentum during training, resulting in smoother updates. Mixed-precision training (AMP) is used to reduce GPU memory usage and speed up training, but GPU support is required.

[0054] Step 2 employs the S2A2C2 network to extract features. Feature extraction is performed at each layer based on ShuffleNetv2, and SAConv is used to collect lost semantic information. AIFI+Conv applies self-attention to semantically richer high-level features, handling complex image features both lightweightly and effectively. Dilated convolution expands the receptive field by introducing additional space into the convolution kernel without increasing the number of parameters or computational cost. Furthermore, using a spatially dependent switching function to combine the results of convolutions with different dilation rates means that each location in the feature map may have different switches to control the SAC output, increasing the network's flexibility. Combining AIFI and Conv replaces YOLOv8's SPPF, providing a more comprehensive and in-depth feature understanding. The S2A2C2 design minimizes memory access by removing redundant and unimportant channels, allowing for a greater focus on learning meaningful and discriminative features. This not only improves the flexibility and adaptability of feature extraction but also enhances the accuracy and efficiency in object detection tasks.

[0055] Figure 3 This diagram illustrates the lightweight feature extraction network S2A2C2. Switchable Atrous Convolution (SAC) is added to the features P3, P4, and P5 extracted from each stage of ShuffleNet V2 to enhance feature extraction in object detection tasks. SAC applies different dilation rates to the same input features, contributing to a more comprehensive understanding of image content. Each layer of the input image's features is extracted using two different convolution kernel configurations, capturing feature information at different scales for a more comprehensive understanding and analysis of the input data. High-level features are processed using AIFI, and the resulting feature map is then adjusted in width and depth using a 1×1 Conv and a lightweight feature map fusion network. AIFI uses only a single Transformer encoder layer, applying self-attention operations to higher-level features with richer semantic concepts. It adaptively selects and weights important feature information to generate the final output features, capturing the relationships between conceptual entities in the image, thus facilitating subsequent modules for object detection and recognition. This allows for the effective handling of complex feature patterns and the extraction of deeper details, even under conditions of significant scale variation.

[0056] Step 3 proposes a lightweight cross-scale feature map fusion network as the neck of the network, connecting the backbone and head networks. This network is responsible for feature fusion and processing to improve detection accuracy and efficiency. Considering the richer information contained in deep features, different processing methods are employed for deep and shallow features. Globally normalized convolutions (GSConv) are used to process deep features to maintain their integrity and consistency. For shallow features, traditional 1x1 convolution operations are chosen, as shallow features are richer in detail and require more attention to subtle local feature changes.

[0057] Figure 4 This is a schematic diagram of a lightweight cross-scale feature map fusion network. With a fixed total computational cost, the more groups used, the slower the actual running speed. Therefore, in the fusion network, only GSConv convolutions are considered for deep features. VoV-GSCSP is a cross-stage partial network module designed using a one-time aggregation method, reducing computational and network structure complexity while maintaining sufficient accuracy. The GCVFusion module is optimized based on different features, upsampling and fusing adjacent features into new features, which are then output to other modules for downsampling to obtain richer feature representations. When the feature map reaches the Neck, the channel dimension of the feature reaches its maximum, and the width and height dimensions reach their minimum. The GCVFusion module maximizes the preservation of implicit connections between each channel, better integrating and utilizing multi-scale features. The lightweight cross-scale feature map fusion network processes deep and shallow features in different ways and performs VoV-GSCSP information fusion, aiming to fully utilize multi-scale features and give the network better detection and cross-scale perception capabilities. It employs multi-scale feature fusion technology to enhance feature representation capabilities.

[0058] Step 4 introduces the proposed loss function FIMPDIoU, which integrates the Inner and Focal principles of MPDIoU. This improves detection accuracy by focusing on target details and comprehensively considers overlapping regions, center point distance, and width and height deviations, thereby enhancing the performance and accuracy of YOLOv8 for target detection in images. Applying the Inner and Focal principles to MPDIoU improves the accuracy of image target detection by focusing on the overlapping parts within the bounding box and the core parts of the bounding box rather than the whole. This is beneficial for improving the detection performance of small targets or targets that are easily overlooked in complex backgrounds. Especially when the distance between the sensor and the target is large, the target's imaging area on the focal plane is small, and the signal-to-noise ratio is low, making the target easily overwhelmed by noise. Therefore, FIMPDIoU is used to allow the model to "focus" on learning samples that are more critical to improving overall performance.

[0059] Figure 5This is a schematic diagram of the FIMPDIoU loss function. In FIMPDIoU, the keypoint distance is the minimum point distance between the auxiliary predicted box and the ground truth box. The cross-entropy loss also focuses on the intersection-union ratio (IU) of the auxiliary predicted box and the ground truth box. All factors, such as overlap area, width, and height deviation, can be determined by the coordinates of the top-left and bottom-right points of the auxiliary predicted box and the ground truth box. The bounding box regression loss function of the network has been replaced. In actual training, DFL is used in combination with FIMPDIoU loss. DFL calculates the loss on the "bounding box probability distribution" and the "probability distribution" of the label in the form of cross-entropy, thereby optimizing each edge. FIMPDIoU loss, on the other hand, restores the "bounding box probability distribution" to the predicted box. By calculating the loss on the "actual box" of the predicted box and the label, it introduces the concepts of Focal and Inner to optimize the overall predicted box.

[0060] Step 5: Since the detection accuracy decreased after adding the S2A2C2 network, this paper introduces an attention mechanism to make the importance of each feature channel inconsistent, allowing the neural network to focus on certain channels with large weights, thereby improving the model's detection accuracy. Different attention mechanisms are added to each feature channel dimension after fusion, and attention mechanisms are only added to high-level fused features of small targets. By introducing the SENetV2 attention mechanism, the network can focus on features crucial to the target detection task, thus improving the model's expressive power and generalization ability. These contributions contribute to the excellent performance of this invention, making the model more lightweight while maintaining high accuracy, facilitating deployment in embedded devices or other resource-constrained environments. The SaE module optimizes the transmission of key features by increasing the cardinality between layers. Through automatic learning, it uses another new neural network to obtain the importance of each channel of the feature map, and then uses this importance to assign a weight value to each feature, allowing the neural network to focus on certain feature channels.

[0061] In step 6, the improved lightweight algorithm is trained on Windows 10 using the training set `train`, and the hyperparameters are adjusted using the test set `val` to assess the learning status. The generalization ability of the algorithm is evaluated using the validation set `test`. The trained model is then deployed to Windows 10, Ubuntu 20, and Android 12 for real-time image prediction to verify the lightweight implementation of the algorithm.

[0062] Figure 6 This is a diagram showing the final detection results of the present invention.

[0063] While the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A lightweight, multi-terminal deployment remote sensing ship and aircraft detection algorithm is based on RT-DETR and improves YOLOv8 for target detection in ship and aircraft images. It is deployed on the Jetson Orin platform and Android. The steps are as follows: Step 1: Using publicly available information, images, and publicly available remote sensing and visible datasets online, create a dataset containing different types of ships and aircraft, manually label different categories, and divide it into a training set (train), a test set (val), and a validation set (test) in an 8:1:1 ratio. Step 2: To address the issues of model compression and lightweighting, this invention proposes a lightweight feature extraction network S2A2C2 based on the structure of ShuffletNetv2, replacing the network's backbone. Step 3: To address the issue of poor detection accuracy in lightweight networks, this invention proposes a lightweight cross-scale feature map fusion network as the neck part of the network. This network aims to effectively integrate feature layer information from different scales while maintaining low time complexity, thereby accelerating image prediction computation. Step 4: This invention proposes the FIMPDIoU loss function, replacing the network's bounding box regression loss function. This focuses on targets that are easily overlooked in complex backgrounds, improving the model's performance in target detection and recognition. The loss prediction box is adjusted using a ratio to adapt to targets in datasets of different sizes. Step 5: In addition, the present invention adds the SENetV2 attention mechanism to the three output feature layers P3, P4 and P5 after extraction to improve the detection accuracy of the model for large, medium and small targets in visible and remote sensing ship and aircraft images. Step 6: Train the improved lightweight algorithm on Windows 10 using the training set `train`, adjust the hyperparameters using the test set `val`, and promptly assess the learning status. Evaluate the generalization ability of the algorithm using the validation set `test`. Deploy the trained model to Windows 10, Ubuntu 20, and Android 12 for real-time image prediction.

2. The method according to claim 1, characterized in that: Step 1 involves the following steps: Using publicly available remote sensing datasets and images, images related to ships and aircraft are selected, and different types of ship or aircraft targets are manually labeled. After mixing and cleaning to remove invalid data, 8524 images from the original dataset are retained. The total number of labeled images in the manually labeled dataset is 8427. These images are divided into training, testing, and validation sets in approximately an 8:1:1 ratio, with 6825 images in the training set, 759 in the testing set, and 843 in the validation set.

3. The method according to claim 1, characterized in that: Step 2 involves the following process: The lightweight network S2A2C2, based on the ShuffleNetV2 backbone, introduces dilated convolution operations to fully utilize information from surrounding pixels to enhance feature representation. This facilitates a more comprehensive understanding of image content and enables effective feature extraction. Each layer of the input image's features is extracted using two different convolutional kernel configurations, capturing feature information at different scales for a more comprehensive understanding and analysis of the input data. The resulting deep features are then processed by AIFIConv using a selective attention mechanism to adaptively select and weight important feature information, generating the final output features. This approach effectively handles complex feature patterns and extracts deeper details even with significant scale variations.

4. The method according to claim 1, characterized in that: Step 3 involves the following steps: Deep features are processed using Global Normalized Convolution (GSConv) to maintain their integrity and consistency. For shallow features, a traditional 1x1 convolution operation (Conv) is used, as shallow features are richer in detail and require more attention to subtle local feature changes. Then, the deep and shallow features are concatenated to form a richer feature representation. This fully utilizes the advantages of both deep and shallow features, improving the model's ability to perceive features at different scales. Finally, VoV-GSCSP is used for information fusion, leveraging feature information from different scales to enhance image detail and contrast, thus improving the model's performance in cross-scale tasks.

5. The method according to claim 1, characterized in that: The specific process of step 4 is as follows: The proposed loss function FIMPDIoU is an MPDIoU that integrates the Inner idea and the Focal idea, which improves the detection accuracy, focuses on target details, and can comprehensively consider the overlapping area, the distance between the center point and the deviation of width and height, thereby improving the performance and accuracy of YOLOv8 in target detection of images. During the training phase, each predicted box By minimizing the loss function Forcing it to approach its true bounding box The derivation of the FIMPDIoU minimization loss function is as follows: When the Inner concept is applied to MPDIoU, the minimization loss function is: When the Focal idea is applied to MPDIoU, the minimization loss function is: The loss function minimized by FIMPDIoU is as follows: in: The FIMPDIoU loss function alleviates the mismatch between the ground truth bounding box and the predicted bounding box, improving the object bounding box fitting ability. It also allows for different ratio parameter settings, optimizing object detection for different sizes. It accelerates model convergence, reduces the loss value, and maintains a smooth curve, thereby improving object detection and segmentation performance and enabling the model to reach its ideal performance level more quickly.

6. The method according to claim 1, characterized in that: Step 5 is as follows: Since the detection accuracy decreases after adding the S2A2C2 network, this paper introduces an attention mechanism to make the importance of each feature channel inconsistent, allowing the neural network to focus on certain channels with large weights, thereby improving the model's detection accuracy. Different attention mechanisms are added to each feature channel dimension after fusion, and attention mechanisms are only added to high-level fused features of small targets. By introducing the SENetV2 attention mechanism, the network can focus on features crucial to the target detection task, thus improving the model's expressive power and generalization ability. These contributions contribute to the excellent performance of this invention, making the model more lightweight while maintaining high accuracy, facilitating deployment on embedded devices or other resource-constrained environments.

7. The method according to claim 1, characterized in that: Step 6 involves training the improved lightweight algorithm on Windows 10 using the training set `train`, adjusting hyperparameters using the test set `val`, and promptly assessing the learning status. The generalization ability of the algorithm is evaluated using the validation set `test`. The trained model is then deployed to Windows 10, Ubuntu 20, and Android 12 for real-time image prediction to verify the lightweight implementation of the algorithm.

Images