Complex sea area unmanned aerial vehicle small target detection method based on space conversion feature optimization

Abstract

The invention discloses a complex sea area unmanned aerial vehicle small target detection method based on space conversion feature optimization. The method comprises the following steps: A, obtaining a sea image data set and marking the sea image data set; b, preprocessing the images in the ocean image data set, and converting the preprocessed images from the spatial dimension to the depth dimension; c, integrating a cross self-attention and feature fusion strategy in the DETR neural network model, introducing a Soft-NMS mechanism, and constructing a neural network model based on space depth conversion feature fusion; d, training a neural network model based on space depth conversion feature fusion by using the ocean image data set after preprocessing and dimension conversion, and finally obtaining a trained model; and E, acquiring a to-be-detected image, and detecting the image by using the trained neural network model. The method can effectively solve the problems of large number of small targets, shielding, clustering and the like in the existing small target detection method, and improves the accuracy of target detection.

Description

Technical Field

[0001] The present invention relates to the field of UAV target detection and proposes a method for detecting small UAV targets in complex sea areas based on space transformation and feature optimization. Background Art

[0002] With the rapid development of drone technology and deep learning, drones have evolved into powerful computing devices, which are of great significance in fields such as ocean monitoring, fishery management, maritime rescue and maritime safety.

[0003] Currently, drone target detection algorithms are primarily categorized into two-stage and one-stage algorithms. Two-stage algorithms first generate candidate regions and then perform classification and regression on them, while one-stage algorithms solely use convolutional neural networks to extract features. Compared to two-stage algorithms, one-stage algorithms offer advantages such as simpler structure, lower computational effort, and improved real-time performance, making them more widely used in drone intelligent systems. While one-stage algorithms have achieved promising results in detecting large objects, detection of small objects at resolutions less than 32 pixels × 32 pixels becomes challenging due to their small size and resolution, insufficient contextual information, and object clustering, resulting in significant false positives and missed detections.

[0004] By introducing spatial depth conversion and feature fusion technology, the proposed UAV target detection method can not only improve the accuracy of target detection but also adapt to changing conditions.

[0005] Currently, small drone target detection methods commonly use object detection technologies based on the YOLO family. For example, the technical solution disclosed in patent application number 202411155147.8, titled "A Method and System for Intelligent Detection and Localization of Small Drone Targets in Complex Scenarios," uses visual perception for video stream preprocessing and calculates inter-frame differences using HOG features and grayscale entropy to select valid frames for processing. A dataset of small drone targets was constructed, and data augmentation techniques were used to optimize training samples to improve detection generalization. The target detection component is optimized based on a modified YOLOv5, employing a self-attention mechanism to enhance feature extraction capabilities and channel and spatial attention mechanisms to improve small target detection accuracy. Furthermore, this method decouples the target detection model, optimizing the classification and localization tasks separately. It also combines inter-frame differences for motion detection to improve the ability to discriminate drone targets. Finally, target detection yields the drone's regressed anchor frames, which are then combined with coordinate system transformation to calculate the drone's three-dimensional coordinates, improving localization accuracy.

[0006] While the aforementioned YOLO-based small-target detection technology for drones offers good real-time performance, it still lags behind state-of-the-art object detection models in terms of feature extraction, detection accuracy, and computational efficiency. Furthermore, while data augmentation and self-attention mechanisms improve its adaptability to complex scenarios, detection performance may be affected in extreme weather conditions (such as heavy rain, dense fog, and strong winds) or lighting conditions (such as backlighting and strong light reflections). Summary of the Invention

[0007] The purpose of the present invention is to provide a small target detection method for UAVs in complex sea areas based on spatial transformation feature optimization, which can effectively solve the problems of a large number of small targets, occlusion and clustering in existing small target detection methods, and improve the accuracy of target detection.

[0008] The present invention adopts the following technical solutions:

[0009] A method for detecting small targets of UAVs in complex sea areas based on spatial transformation feature optimization includes the following steps:

[0010] A: Obtain and annotate ocean image datasets;

[0011] B: Preprocess the images in the ocean image dataset and convert the preprocessed images from spatial dimension to depth dimension;

[0012] C: Integrate cross-self-attention and feature fusion strategies into the DETR neural network model, introduce the Soft-NMS mechanism, and build a neural network model based on spatial depth conversion feature fusion;

[0013] D: Using the pre-processed and dimensionally transformed ocean image dataset, a neural network model based on spatial depth conversion feature fusion is trained to obtain the trained model.

[0014] E: Obtain the image to be detected and use the trained neural network model to detect the image.

[0015] The neural network model based on spatial depth conversion feature fusion consists of an encoder-decoder module, a feature fusion and cross-self-attention mechanism module, a Soft-NMS module, and a tiny prediction head module;

[0016] In the encoder-decoder module, the encoder is used to extract global context information including text features and image features from the preprocessed image; the decoder is used to generate the target category and candidate box based on the global context information;

[0017] The feature fusion and cross-self-attention mechanism module is set between the encoder and decoder to fuse text features and image features, generate fused features through self-attention and cross-attention mechanisms, and output fused feature maps;

[0018] The Soft-NMS module optimizes the detection results of overlapping objects by adjusting the confidence scores of candidate boxes;

[0019] The tiny prediction head module is used to perform small target optimization processing on the fused feature map, receive the output feature map of the encoder, and output accurate small target prediction results.

[0020] The encoder-decoder module includes a text encoder, an image encoder, a text decoder, and an image decoder; the text encoder performs semantic extraction on the input text, generates a text semantic feature map through the Transformer network, and obtains a feature representation containing the key semantic information of the text; the image encoder performs feature extraction on the input image, generates an image feature map through the ResNet-50 network, and obtains a feature representation containing the visual features of each object in the image.

[0021] The feature fusion and cross-self-attention mechanism module first obtains the output of the self-attention mechanism for a given image feature map using the query matrix, key matrix, and value matrix of the image feature map, combined with the key vector dimension of the image feature map; then, for a given image feature map and text feature map, the query matrix of the text feature map, the key matrix and value matrix of the image feature map, combined with the key vector dimension of the image feature map, obtains the output of the cross-self-attention mechanism; finally, the output of the self-attention mechanism and the output of the cross-self-attention mechanism are fused to obtain the final output feature map.

[0022] The Soft-NMS module first sorts all candidate boxes in descending order according to the confidence score to obtain a candidate list and initializes the final detection result list; then it determines whether each candidate box in the candidate list is suppressed. If not, the candidate box is added to the final detection result list and continues to be judged; if it is suppressed, the confidence score of the candidate box is updated using the attenuation coefficient, and the candidate box is left in the candidate list and continues to be judged; then, after each candidate box in the candidate list is judged, the intersection-over-union ratio between each candidate box in the final detection result list and all selected candidate boxes after the candidate box is calculated in turn. If the intersection-over-union ratio result is greater than the intersection-over-union ratio threshold, the confidence score is updated using the attenuation coefficient; finally, after completing the confidence update of all candidate boxes, all candidate boxes with confidence scores lower than the set removal threshold are removed, and the candidate boxes after overlapping detection are finally obtained.

[0023] The tiny prediction head module first obtains the output feature map of the encoder, and extracts it after convolution processing to obtain a feature map suitable for small target detection. It then uses the convolved feature map to predict and output the category, bounding box coordinates, and existence confidence of each target.

[0024] Step D includes the following specific steps:

[0025] D1: Divide the preprocessed and converted ocean image dataset into a training set and a validation set;

[0026] D2: Use the training set to iteratively train the neural network model based on spatial depth conversion feature fusion and adjust the model parameters to optimize performance;

[0027] D3: Use the validation set to evaluate the performance of the neural network model based on spatial depth conversion feature fusion and save the trained model weights.

[0028] In step B, the images in the ocean image dataset are first preprocessed, including enhancement, noise reduction, edge detection, scaling, and rotation. Then, a space-to-depth conversion module is constructed, and the space-to-depth conversion technology is used to convert the preprocessed images from the spatial dimension to the depth dimension.

[0029] The performance of the neural network model based on spatial depth conversion feature fusion is evaluated using accuracy, precision, recall and F1 score to determine whether the model is overfitting or underfitting.

[0030] In step A, the ocean image dataset is annotated, including weather conditions, location and type of target objects.

[0031] The present invention applies the preprocessing technology of the space-to-depth conversion module to enhance the feature representation of small targets in the image to cooperate with subsequent multimodal data fusion; and by integrating self-attention and feature fusion strategies in the DETR neural network model, it strengthens the feature integration between different modalities, and introduces the Soft-NMS mechanism to optimize the model's detection ability for overlapping targets. By integrating space-depth conversion and feature fusion technology in the DETR model, and introducing a tiny prediction head and Soft-NMS mechanism, the present invention can effectively improve the accuracy and efficiency of small target detection in drone ocean images, and can also adapt to changing conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the process of the present invention;

[0033] Figure 2 This is the structure diagram of the HRSC2016 dataset in the present invention;

[0034] Figure 3Schematic diagram of the structure of the neural network model based on spatial depth conversion feature fusion in the present invention;

[0035] Figure 4 This is a structural diagram of the space-to-depth conversion module in the present invention;

[0036] Figure 5 This is a structural diagram of the feature fusion and cross self-attention modules in the present invention;

[0037] Figure 6 A flowchart of the steps of the neural network model in the present invention;

[0038] Figure 7 This is an example of an ocean image in the present invention. DETAILED DESCRIPTION

[0039] The present invention is described in detail below with reference to the accompanying drawings and embodiments:

[0040] like Figures 1 to 7 As shown, the method for detecting small targets of unmanned aerial vehicles in complex sea areas based on spatial transformation feature optimization of the present invention includes the following steps:

[0041] A: Get ocean image dataset;

[0042] Among them, the ocean image dataset is derived from the HRSC2016 dataset;

[0043] Before conducting small target detection by drones in complex sea areas, it is necessary to build a neural network model for detecting ocean images. When building the neural network model, it is necessary to obtain a large amount of ocean image data of different types for training the model. In order to improve the performance of the training model, the present invention performs preprocessing operations on the initially acquired training data to ensure the subsequent stability and generalization ability of the model.

[0044] In this embodiment, step A includes the following specific steps:

[0045] A1: Obtain ocean image dataset;

[0046] In this embodiment, several images are selected from the HRSC2016 dataset to form an ocean image dataset; the HRSC2016 dataset contains a wealth of aerial images of sea and coastal areas under various weather conditions and scenarios, with a total of 1070 images and 2976 instances, of which the training set, validation set, and test set contain 436 images (including 1207 samples), 181 images (including 541 samples), and 444 images (including 1228 samples), respectively, for training and testing aerial ship detection in rivers, oceans, harbors, and other waters. Figure 7 As shown, the file structure of the HRSC2016 dataset is as follows Figure 2 shown.

[0047] A2: Annotate the ocean image dataset.

[0048] Since the original structure of the HRSC2016 dataset is not suitable for the DETR model, the present invention annotates the HRSC2016 dataset to make it suitable for the DETR model, that is, converts the dataset into the COCO format.

[0049] First, we use tools like Labelme to annotate the images in the HRSC2016 dataset, specifically including information such as weather conditions, the location and type of target objects (e.g., ships), etc. This will facilitate subsequent data processing and model training, as well as facilitate quick retrieval of images under specific weather conditions.

[0050] Next, we convert the labeled data from Labelme to COCO format. We also convert each image's annotation information (including category, bounding box, etc.) into COCO's JSON format. Finally, we create a corresponding JSON file containing images, annotations, and categories.

[0051] Among them, images contains detailed information of the image, such as file name, size, etc.; annotations contains the annotation information of each target, such as category ID, bounding box coordinates, etc.; categories contains the name and ID of the category.

[0052] B: Preprocess the images in the ocean image dataset and convert the preprocessed images from spatial dimension to depth dimension;

[0053] In order to highlight the key information in the image and help the model better detect the image, the present invention performs preprocessing operations on the images in the ocean image dataset, thereby making it easier for the model to learn and identify targets, and improving the accuracy and robustness of the model.

[0054] In this embodiment, step B includes the following specific steps:

[0055] B1: Preprocess the images in the ocean image dataset, including enhancement, noise reduction, edge detection, scaling, and rotation.

[0056] First, the aerial images are enhanced to reduce the effects of sea surface reflections and cloud cover. This enhancement process includes adjusting contrast, brightness, and color balance to enhance the distinction between the vessel and the background.

[0057] Subsequently, noise reduction techniques such as Gaussian blur or median filtering are applied to reduce random noise in the image and improve image quality.

[0058] Second, use an edge detection algorithm, such as the Canny edge detector, to highlight the outline of the ship and separate it from the background.

[0059] Next, the image is scaled to keep the size of the vessel consistent across the image, reducing recognition errors caused by size differences.

[0060] Finally, rotation invariance processing is implemented, such as rotating the image by its center point, to ensure that the orientation of the ship does not affect the recognition results.

[0061] B2: Build a space-to-depth conversion module and use the space-to-depth conversion technology (SPD-Conv) to convert the pre-processed image from the spatial dimension to the depth dimension;

[0062] Although DETR does not use traditional convolutional layers, the core idea of the SPD-Conv module, which is to retain more spatial information during the downsampling process, can be transformed into the form of an attention mechanism. In DETR, the effect of SPD-Conv can be simulated by designing a special self-attention module. This self-attention module can focus more on local features while retaining global context information to help the model capture objects at different scales, especially the features of small objects.

[0063] In the present invention, the structure diagram of the space-to-depth conversion module is as follows: Figure 4 As shown. The spatial to depth conversion (SPD-Conv) module is a convolution operation that converts the spatial dimension to the depth dimension. Assume that the dimension of the input feature map X is H×W×C, where H is the height, W is the width, and C is the number of channels. The SPD-Conv module is used to convert the input feature map X into an output feature map of dimension H′×W′×C′, where H′=H / b, W′=W / b, and C′=C×b 2 , then the SPD-Conv operation can be expressed as:

[0064]

[0065] Among them, Y represents the output feature map, i represents the height index of the output feature map, j represents the width index of the output feature map, k represents the channel index of the output feature map, and each element Y of the output feature map i,j,k The element at the corresponding position in the input feature map Converted to, and kmodb represent the row and column positions of channel index k in the block, respectively. b represents the block size, and mod denotes a modulo operation. In this method, the row index in the input feature map is calculated based on the block size b, i.e., the quotient of i·b plus k / b, and the column index is the remainder of j·b plus kmodb. In this way, each b·b block in the input feature map is mapped to a channel in the output feature map, completing the information conversion from the spatial dimension to the depth dimension.

[0066] For example, if the input feature map has a dimension of 4×4×1 and a block size of 2, the output feature map will have a dimension of 2×2×4. The input feature map is divided into 2×2 blocks, and each block is converted into a channel in the output feature map. This operation converts the spatial dimension into the depth dimension, helping the model capture more contextual information.

[0067] C: Integrate cross-self-attention and feature fusion strategies into the DETR neural network model, introduce the Soft-NMS mechanism, and build a neural network model based on spatial depth conversion feature fusion;

[0068] In the present invention, the neural network model based on spatial depth conversion feature fusion consists of an encoder-decoder module, a feature fusion and cross self-attention mechanism module, a Soft-NMS module and a small prediction head module. The structural diagram is shown in FIG. Figure 3 shown.

[0069] In this paper, the encoder extracts global context information containing text semantic features and image features from the preprocessed image; the decoder generates the target category and bounding box based on this global context information. The feature fusion module fuses image features and text features to improve the detection ability of objects of different sizes. Self-attention and cross-attention mechanisms are used to strengthen the interaction between features and enhance the understanding of complex scenes. The Soft-NMS module is used to optimize the detection of overlapping objects, reducing errors and omissions. The tiny prediction head specifically optimizes the detection results of small objects, further improving the performance of the model.

[0070] The encoder-decoder module in the present invention consists of four parts: a text encoder, an image encoder, a text decoder, and an image decoder. The text encoder performs semantic extraction processing on the input text, generates a text semantic feature map through the Transformer network, and obtains a feature representation containing the key semantic information of the text. These feature maps accurately capture the key semantic information in the text, such as the category name and attribute description of the target, and provide semantic support for subsequent target detection. The image encoder performs feature extraction processing on the input image, generates an image feature map through the ResNet-50 network, and obtains a feature representation containing the visual features of each object in the image. These feature maps describe in detail the details such as the edge, texture, color, shape and relative position of each object in the image, providing basic information for subsequent target positioning and classification.

[0071] The feature fusion module fuses the text semantic feature map generated by the text encoder and the image feature map generated by the image encoder, and generates a fused feature map through the self-attention and cross-attention mechanisms to obtain a feature representation that integrates text and image information, strengthens the feature interaction between different modalities, and enhances the model's ability to recognize the relationship between objects in complex scenes.

[0072] The decoder module decodes the fused feature map generated by the feature fusion module, uses the global context information of the image captured by the encoder, combines the self-attention and cross-attention mechanisms, directly predicts the category and bounding box of each target, generates a detection result feature map, and defines the predicted target category and bounding box results as candidate boxes.

[0073] The Soft-NMS module performs confidence adjustment on the detection result feature map generated by the decoder module. By adjusting the confidence score of the candidate box, it optimizes the detection results of overlapping targets, reduces false detections and missed detections, and improves the overall detection accuracy of the model.

[0074] The tiny prediction head module performs small target optimization processing on the fused feature map generated by the feature fusion module, receives the output feature map of the encoder, outputs accurate small target prediction results, and generates a small target prediction feature map, further improving the model's performance in small target detection.

[0075] The encoder-decoder module uses self-attention and cross-attention mechanisms and feature fusion strategies through subsequent feature fusion and cross-self-attention modules to integrate features from different modalities (such as images and text) and feature maps of different scales, thereby improving the model's ability to understand complex scenes.

[0076] In the present invention, the structure diagram of the feature fusion and cross self-attention module is as follows Figure 5As shown in the figure, in the DETR model, feature fusion is usually achieved through self-attention and cross-attention mechanisms in the Transformer encoder and decoder. The image feature map F from the image encoder and the text feature map T from the text encoder are fused and used for object detection.

[0077] In this embodiment, for the image feature map F, the self-attention mechanism can be expressed as:

[0078]

[0079] Among them, F self-attention represents the output of the self-attention mechanism, softmax represents the normalized exponential function, QF, KF, and VF are the query matrix, key matrix, and value matrix of the image feature map F, respectively. k is the dimension of the key vector and T represents the transpose of the matrix.

[0080] That is, for a given image feature map F, the present invention generates a query matrix QF, a key matrix KF, and a value matrix VF through three different linear transformations; the dot product of the query matrix and the key matrix after transposition (i.e., the transposition operation of the key matrix) is calculated, and the result is divided by the square root of the key vector dimension dk to scale to prevent the gradient from disappearing due to excessive values. Then, we apply the softmax function to the obtained dot product result to convert it into a probability distribution that represents the correlation between each position in the input feature map. Finally, this probability distribution is multiplied by the value matrix VF to obtain the output of the self-attention mechanism, which is the weighted sum of the input feature map F, and the weight is determined by the output of the softmax function.

[0081] For the image feature map F and the text feature map T, the cross attention mechanism can be expressed as:

[0082]

[0083] Among them, F cross-attention represents the output of the cross-self-attention mechanism, and QT is the query matrix of the text feature map T.

[0084] That is, for a given image feature map F and text feature map T, the text feature map is used to generate the query matrix QT through linear transformation, and the image feature map F is also used to generate the key matrix KF and the value matrix VF. First, the dot product of the transpose of the query matrix and the key matrix (that is, the transpose operation of the key matrix) is calculated, and then the scale is divided by the square root of the dimension of the key vector. Next, the softmax function is applied to this scaled result to obtain the attention weight. Finally, the attention weight is multiplied by the value matrix VF to obtain the final output feature map that combines the information of the image input feature map and the text input feature map.

[0085] The results of self-attention and cross-attention are combined to obtain the final feature map F out :

[0086] F out =F self-attention +F cross-attention ;

[0087] In the present invention, the Soft-NMS (Soft Non-Maximum Suppression) module is used to adjust the confidence score of the candidate box to reduce repeated detections caused by overlap.

[0088] For a given set of candidate boxes B = {b1, b2, ..., b N}, each candidate box b i Each has a confidence score s i The Soft-NMS module adjusts the confidence score of the candidate box according to the following steps:

[0089] The first step is to use the confidence score s i Sort all candidate boxes in descending order to obtain a candidate list and initialize the final detection result list

[0090] The second step is to judge each candidate box b in the candidate list in turn i Suppressed:

[0091] If it is not suppressed, the candidate box b i Add it to the final detection result list D; then continue to the next candidate box b i Make a judgment;

[0092] If it is suppressed, the candidate box b i The confidence score s i Updated to s′ using the decay coefficient i , and the candidate box b i Stay in the candidate list; then continue to judge the next candidate box;

[0093] In this embodiment, if a candidate box is suppressed, its confidence score is multiplied by a set decay coefficient less than 1, and the determination of whether the next candidate box is suppressed continues. The decay coefficient is calculated based on the intersection over union (IoU) of this candidate box b i and the candidate box b max with the highest confidence. Subsequently, the suppressed candidate box will still remain in the candidate list. In subsequent processing, the above-mentioned suppressed candidate box will be filtered out because its confidence is lower than the set filtering threshold, or not considered when integrating the final detection results.

[0094]

[0095] Among them, IoU(b i , b j ) represents the intersection over union IoU of the candidate box b i and the candidate box b max with the highest confidence; exp() is the natural exponential function, and σ is a hyperparameter used to control the sensitivity of confidence reduction;

[0096] Third step, after each candidate box b i in the candidate list has been judged, for each candidate box b j in the final detection result list D (where j < i), the intersection over union IoU between b i and all selected candidate boxes b j is calculated in sequence:

[0097] If IoU(b i , b j ) > threshold, then update the confidence score s′ j of b j to:

[0098] Among them, threshold is the intersection over union threshold, and the candidate box b i is calculated in sequence with all candidate boxes b j that meet the conditions, that is, j < i; Area(b i ∩b j ) represents the area of the intersection region of the candidate boxes b i and b j ; Area(b i ∪b j ) represents the area of the union region of the candidate boxes b i and b j , and c represents the latest confidence score of b i , and σ is a hyperparameter used to control the sensitivity of confidence reduction.

[0099] In this paper, Soft-NMS is used to address the overlap problem between candidate boxes. The overlap ratio of each pair of candidate boxes is calculated—the ratio of their intersection area to their union area—to measure their degree of overlap. When the overlap ratio exceeds a preset threshold, the confidence score of one of the candidate boxes is multiplied by an exponential decay function value less than 1 based on the overlap ratio and related parameters, thereby reducing its confidence score. This process helps reduce duplicates in the detection results while suppressing low-confidence candidate boxes with excessive overlap, thereby optimizing the detection results.

[0100] The fourth step is to remove all candidate boxes whose confidence scores are lower than the set removal threshold s after completing the confidence update of all candidate boxes. min Finally, the candidate box after overlapping detection is obtained.

[0101] In the DETR (Detection Transformer) model, the Tiny Prediction Head module is designed to better detect small objects in images. The Tiny Prediction Head module is typically a neural network module that receives the encoder's output feature map and outputs a prediction result for the small object.

[0102] In this embodiment, by adding an additional tiny prediction head, the resolution of the detection feature map (which can be set to 160×160) can be effectively increased to detect small objects larger than the set resolution (which can be set to 4×4). Smaller detection pixels can extract more small target features, thereby improving the model's performance in small target detection.

[0103] In this embodiment, the tiny prediction head first transforms the encoder’s output feature map F encoder Processed through one or more convolutional layers to extract feature maps F suitable for small target detection processed ;

[0104] F processed =Conv(F encoder );

[0105] Among them, Conv represents the convolution operation.

[0106] Then use the convolutional feature map F processed To predict the category C of each target;

[0107] C=softmax(W c ·F processed +b c );

[0108] Among them, W c and bc are learnable weight and bias parameters, and the softmax function is used to convert the output into a probability distribution;

[0109] At the same time, predict the bounding box coordinates B of each target;

[0110] B=W b ·F processed +b b ;

[0111] Among them, W b and b b are learnable weight and bias parameters.

[0112] Then predict the existence confidence S of each target;

[0113] S=sigmoid(W s ·F processed +b s );

[0114] Among them, W s and b s are learnable weight and bias parameters, and the sigmoid function is used to convert the output into a probability value.

[0115] Finally, the tiny prediction head outputs three prediction results: category probability C, bounding box coordinates B, and existence confidence S; the above output prediction results can be used for subsequent object detection tasks, such as non-maximum suppression (NMS) or Soft-NMS.

[0116] D: Using the preprocessed and dimensionally transformed ocean image dataset, a neural network model based on spatial depth conversion feature fusion is trained to obtain the trained model.

[0117] After data processing is completed, the model needs to be trained using the preprocessed and converted ocean image dataset so that the neural network model can accurately detect small targets. The training steps of the neural network model are as follows: Figure 6 shown.

[0118] In the present invention, step D comprises the following specific steps:

[0119] D1: Divide the preprocessed and converted ocean image dataset into a training set and a validation set; the training set is used for model learning, and the validation set is used to evaluate the model's performance on unseen data to prevent model overfitting.

[0120] Specifically, the preprocessed and converted ocean image dataset is divided into training set, validation set and test set according to preset ratios to ensure that the model has sufficient data during the training process, while reserving some data for model verification and tuning.

[0121] D2: Use the training set to iteratively train the neural network model based on spatial depth conversion feature fusion and adjust the model parameters to optimize performance.

[0122] Before training begins, the neural network model's weights and bias parameters are initialized. Initialization can be random or use pretrained weights to achieve faster model convergence. The model is then iteratively trained using the training set. In each iteration (or epoch), the model calculates predictions using forward propagation and then updates the weights using backpropagation to minimize the loss function. These training steps are repeated until the model achieves satisfactory performance on the training set. During training, performance is optimized by adjusting model parameters, such as the learning rate, optimizer selection, batch size, and regularization parameters, to help the model better learn the data characteristics and improve generalization.

[0123] In this embodiment, the learning rate is 0.0001, the batch size is 32, the optimizer uses Adam, and the number of iterations is set to 100 epochs.

[0124] D3: Use the validation set to evaluate the performance of the neural network model based on spatial depth conversion feature fusion and save the trained model weights.

[0125] In this embodiment, the performance of the neural network model based on spatial depth conversion feature fusion is evaluated using evaluation indicators such as accuracy, precision, recall rate and F1 score to determine whether the model is overfitting or underfitting.

[0126] The accuracy calculation formula is:

[0127]

[0128] The calculation formula for accuracy is:

[0129]

[0130] The calculation formula for recall is:

[0131]

[0132] The calculation formula of F1 score is

[0133] F1=2*P*R / (P+R);

[0134] Among them, TP represents the number of positive samples predicted as positive samples, TP represents the number of negative samples predicted as positive samples, FN represents the number of positive samples predicted as negative samples, P, R and F1 represent precision, recall rate and F1 score respectively.

[0135] Fine-tune the model using the metrics feedback obtained on the validation set. Once the model achieves optimal performance on the validation set, save the model weights and configuration.

[0136] E: Obtain the image to be detected and use the trained neural network model to detect the image.

[0137] In this method, an image to be detected with similar features to the training data is obtained. This image is then preprocessed and transformed. The trained model is then used to detect small objects, outputting a detection result that includes category, location, and confidence. Finally, the accuracy and reliability of the detection results are analyzed and verified by comparison with the true labels, allowing for necessary model optimization and iteration.

[0138] The present invention provides a method for detecting small targets of drones in complex sea areas based on spatial conversion and feature optimization, aiming to solve the problems of a large number of small targets in drone aerial photography, as well as occlusion and clustering in drone target detection. The method comprises: collecting a large number of drone aerial images from the HRSC2016 dataset; performing necessary preprocessing on the images, applying the preprocessing technology of the space-to-depth conversion (SPD-Conv) module, enhancing the feature representation of small targets in the images, and preparing for subsequent multimodal data fusion; integrating self-attention and feature fusion strategies in the DETR neural network model to strengthen the feature integration between different modalities, introducing the Soft-NMS mechanism, and optimizing the model's detection capability for overlapping targets; using the preprocessed HRSC2016 image dataset to train the model to obtain a trained model; and using the trained model to detect small targets of drones. By integrating spatial depth conversion and feature fusion technology in the DETR model, and introducing a tiny prediction head and the Soft-NMS mechanism, the present invention can not only improve the accuracy of target detection, but also adapt to changing conditions.

[0139] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for detecting small targets of UAVs in complex sea areas based on spatial transformation feature optimization, characterized by: The following steps are involved: A: Obtain and annotate ocean image datasets; B: Preprocess the images in the ocean image dataset and convert the preprocessed images from spatial dimension to depth dimension; C: Integrate cross-self-attention and feature fusion strategies into the DETR neural network model, introduce the Soft-NMS mechanism, and build a neural network model based on spatial depth conversion feature fusion; D: Using the pre-processed and dimensionally transformed ocean image dataset, a neural network model based on spatial depth conversion feature fusion is trained to obtain the trained model. E: Obtain the image to be detected and use the trained neural network model to detect the image.

2. The method for detecting small targets of unmanned aerial vehicles in complex sea areas based on spatial transformation feature optimization according to claim 1 is characterized by: The neural network model based on spatial depth conversion feature fusion consists of an encoder-decoder module, a feature fusion and cross-self-attention mechanism module, a Soft-NMS module, and a tiny prediction head module; In the encoder-decoder module, the encoder is used to extract global context information including text features and image features from the preprocessed image; The decoder is used to generate the target category and candidate box based on the global context information; The feature fusion and cross-self-attention mechanism module is set between the encoder and decoder to fuse text features and image features, generate fused features through self-attention and cross-attention mechanisms, and output fused feature maps; The Soft-NMS module optimizes the detection results of overlapping objects by adjusting the confidence scores of candidate boxes; The tiny prediction head module is used to perform small target optimization processing on the fused feature map, receive the output feature map of the encoder, and output accurate small target prediction results.

3. The method for detecting small targets of unmanned aerial vehicles in complex sea areas based on spatial transformation feature optimization according to claim 1 is characterized by: The encoder-decoder module includes a text encoder, an image encoder, a text decoder, and an image decoder; the text encoder performs semantic extraction on the input text, generates a text semantic feature map through the Transformer network, and obtains a feature representation containing the key semantic information of the text; the image encoder performs feature extraction on the input image, generates an image feature map through the ResNet-50 network, and obtains a feature representation containing the visual features of each object in the image.

4. The method for detecting small targets of unmanned aerial vehicles in complex sea areas based on spatial transformation feature optimization according to claim 1 is characterized by: The feature fusion and cross-self-attention mechanism module first uses the query matrix, key matrix, and value matrix of the image feature map, combined with the key vector dimension of the image feature map, to obtain the output of the self-attention mechanism for a given image feature map. Then, for a given image feature map and text feature map, the query matrix of the text feature map, the key matrix and value matrix of the image feature map are used, combined with the key vector dimension of the image feature map to obtain the output of the cross-self-attention mechanism; finally, the output of the self-attention mechanism and the output of the cross-self-attention mechanism are fused to obtain the final output feature map.

5. The method for detecting small targets of unmanned aerial vehicles in complex sea areas based on spatial transformation feature optimization according to claim 1 is characterized by: The Soft-NMS module first sorts all candidate boxes in descending order according to the confidence score to obtain a candidate list and initializes the final detection result list; then it determines whether each candidate box in the candidate list is suppressed. If not, it adds the candidate box to the final detection result list and continues to judge; If it is suppressed, the confidence score of the candidate box is updated using the attenuation coefficient, and the candidate box is left in the candidate list and continues to be judged; Then, after each candidate box in the candidate list has been judged, the intersection-over-union (IoU) between each candidate box in the final detection result list and all selected candidate boxes after the candidate box is calculated in turn. If the IoU result is greater than the IoU threshold, the confidence score is updated using the attenuation coefficient. Finally, after completing the confidence update of all candidate boxes, all candidate boxes with confidence scores lower than the set removal threshold are removed, and the candidate boxes after overlapping detection are finally obtained.

6. The method for detecting small targets of unmanned aerial vehicles in complex sea areas based on spatial transformation feature optimization according to claim 1 is characterized by: The tiny prediction head module first obtains the output feature map of the encoder, and extracts it after convolution processing to obtain a feature map suitable for small target detection. It then uses the convolved feature map to predict and output the category, bounding box coordinates, and existence confidence of each target.

7. The method for detecting small targets of unmanned aerial vehicles in complex sea areas based on spatial transformation feature optimization according to claim 1 is characterized in that: Step D includes the following specific steps: D1: Divide the preprocessed and converted ocean image dataset into a training set and a validation set; D2: Use the training set to iteratively train the neural network model based on spatial depth conversion feature fusion and adjust the model parameters to optimize performance; D3: Use the validation set to evaluate the performance of the neural network model based on spatial depth conversion feature fusion and save the trained model weights.

8. The method for detecting small targets of unmanned aerial vehicles in complex sea areas based on spatial transformation feature optimization according to claim 1 is characterized by: In step B, the images in the ocean image dataset are first preprocessed, and the preprocessing includes enhancement processing, noise reduction processing, edge detection processing, scaling processing and rotation processing; Then, a space-to-depth conversion module is constructed, and the space-to-depth conversion technology is used to convert the preprocessed image from the spatial dimension to the depth dimension.

9. The method for detecting small targets of unmanned aerial vehicles in complex sea areas based on spatial transformation feature optimization according to claim 1 is characterized by: The performance of the neural network model based on spatial depth conversion feature fusion is evaluated using accuracy, precision, recall and F1 score to determine whether the model is overfitting or underfitting.

10. The method for detecting small targets of unmanned aerial vehicles in complex sea areas based on spatial transformation feature optimization according to claim 1 is characterized in that: In step A, the ocean image dataset is annotated, including weather conditions, location and type of target objects.

Images