A remote sensing image target dynamic detection method based on a multi-kernel

Abstract

The application provides a remote sensing image target dynamic detection method based on a multi-core, and belongs to the field of remote sensing image detection.The method comprises the following steps: collecting a remote sensing image, optimizing a YOLOv8 model, using a cross-stage feature fusion module based on a multi-core in the optimized YOLOv8 model backbone network to extract information from the remote sensing image, and obtaining feature maps of different scales; using a neck network to dynamically up-sample the feature maps, adding a dynamic detection head in response to the neck network output up-sampled feature maps, and using an attention mechanism to optimize and fuse the up-sampled feature maps to obtain fused feature maps; and classifying the feature maps to obtain a classified target detection result.The application solves the problems of insufficient experimental effect on a multi-class data set, a large number of small targets, large target scale changes and dense distribution of remote sensing images, and detection under a complex background in the prior art.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image detection, and in particular relates to a remote sensing image target dynamic detection method based on a multi-kernel. Background Art

[0002] Remote sensing imagery utilizes sensors on various aviation and aerospace platforms to image distant targets. Based on the imaging method, it can be categorized into infrared, multispectral, synthetic aperture radar (SAR), and visible light imagery. Infrared remote sensing uses equipment to detect environmental parameters such as temperature to distinguish between objects, resulting in uncertainty in detection results. Multispectral imaging has lower resolution and can be affected by environmental conditions such as lighting and atmospheric conditions, requiring additional calibration steps to ensure data accuracy. While synthetic aperture radar (SAR) has strong penetration and can capture information about obscured objects on the ground, its data contains a large amount of information and requires specialized processing and analysis. Furthermore, the equipment is expensive, and terrain can affect imaging accuracy. Visible light imaging operates in the 380-760 nanometer wavelength band. Optical remote sensing images contain a wealth of detailed information, visually demonstrating the shape, color, and detailed features of targets. With the development of satellite and remote sensing technology, the continuous use of advanced equipment has enabled researchers to acquire large quantities of high-resolution remote sensing imagery with higher image quality and greater detail. This allows them to detect targets of interest in remote sensing images through target detection, thereby providing decision support for related tasks.

[0003] In recent years, with the development of deep learning, deep features extracted by neural networks have stronger semantic representation and discriminative capabilities, further improving the performance of object detection. However, remote sensing images are characterized by dense target distribution, a large number of small targets, large variations in target scale, blurred targets, and complex backgrounds. As a result, existing detection methods are unable to achieve good performance on remote sensing images.

[0004] In view of the characteristics of remote sensing images, the existing technology proposes a remote sensing image target detection method based on the improved YOLOv5, introduces a channel global attention mechanism in the backbone, and enhances the feature extraction capability of small targets. Although there is a certain effect improvement, the detection accuracy is low; the existing technology improves the loss function on YOLOv5, improves the accuracy of aircraft target detection in remote sensing images, and accelerates the convergence of the algorithm. However, this method only targets a single target and cannot fully prove its effect on the detection of other targets; the existing technology proposes a super-resolution and deep learning detection method in the direction of infrared small target detection, preprocesses the super-resolution image and performs multiple data enhancements. Although the dataset is enhanced, the processing is cumbersome and time-consuming, and cannot meet the real-time requirements; the existing technology performs lightweight design in the field of SAR image target detection, and proposes a SAR target detection algorithm enhanced by hybrid representation learning based on the unique features of SAR images. Although this method can be more suitable for SAR image datasets, the processing of such datasets is cumbersome and has low versatility.

[0005] For remote sensing image datasets, existing technologies have improved the Real-time Models for Object Detection (RTMDet) algorithm. Although this improves the algorithm's detection accuracy, the model has a large number of parameters and detection is time-consuming, failing to meet real-time requirements. Based on the existing YOLOv5, the proposed YOLO-RS algorithm introduces adaptive spatial feature fusion to improve feature fusion capabilities. However, this algorithm fails to account for issues such as large object scale variations and object blur in remote sensing images, resulting in limited detection effectiveness. Existing technologies have combined the latest YOLOv8 algorithm and, based on the good detection performance of the baseline model, designed a fast convolution module to reconstruct YOLOv8's C2f module, enhancing the network's feature extraction capabilities. This improves detection performance somewhat compared to mainstream algorithms, but the improvement is relatively small, and the model still has room for further improvement. Existing technologies have proposed an optical remote sensing aircraft target fast detection algorithm for embedded platforms. This algorithm improves on aircraft target features and is deployed in conjunction with hardware. However, its effectiveness on multiple categories has not been proven, and the algorithm lacks universal applicability.

[0006] The above algorithms have made certain improvements in the field of remote sensing image target detection, but the following problems still exist: 1. The selected dataset categories are relatively small or there are only a single category, which cannot fully prove the experimental effect on multi-category datasets; 2. The algorithm still has room for improvement on general optical remote sensing images and fails to solve the problem of large scale variations and dense distribution of remote sensing image targets; 3. The ground objects in remote sensing images often have complex backgrounds and diverse scale variations. These algorithms have not yet been able to solve the detection situation under complex backgrounds. Summary of the Invention

[0007] In response to the above-mentioned deficiencies in the prior art, the present invention provides a remote sensing image target dynamic detection method based on a multi-kernel, which solves the problems of the prior art in insufficient experimental results on multi-category data sets, a large number of small targets, large scale variations and dense distribution of remote sensing image targets, and detection in complex backgrounds.

[0008] In order to achieve the above objectives, the present invention adopts a technical solution: a remote sensing image target dynamic detection method based on a multi-kernel, comprising the following steps:

[0009] S1. Collect remote sensing images and optimize the YOLOv8 model to obtain an optimized YOLOv8 model;

[0010] S2. Use the optimized YOLOv8 model backbone network to extract information from remote sensing images and obtain feature maps of different scales through downsampling;

[0011] S3. Use the neck network of the optimized YOLOv8 model to dynamically upsample feature maps of different scales to generate upsampled feature maps, and use the neck network to output the upsampled feature maps;

[0012] S4. In response to the upsampled feature map output by the neck network, a dynamic detection head is added and the upsampled feature map is optimized using the attention mechanism, and feature fusion is performed on the optimized upsampled feature map to obtain a fused feature map;

[0013] S5. Detect and classify the fused feature map to obtain the classified target detection results, completing the dynamic detection of remote sensing image targets.

[0014] Furthermore, the optimized YOLOv8 model includes:

[0015] The polykernel network module is used to improve the cross-stage feature fusion module in the backbone network of the YOLOv8 model;

[0016] A cross-stage feature fusion module based on multi-core is used to extract information from remote sensing images and obtain feature maps of different scales through downsampling;

[0017] The dynamic feature pyramid network is used to dynamically upsample feature maps of different scales using the neck network to generate upsampled feature maps, and the neck network is used to output the upsampled feature maps. The obtained feature maps are input into the dynamic detection head to optimize and fuse the feature maps.

[0018] Furthermore, the loss function of the optimized YOLOv8 model includes:

[0019] Zoom loss function:

[0020]

[0021] Where VFL(·) represents the zoom loss function, p represents the predicted intersection-over-union (IoU) perceptual classification score, q represents the IoU ratio between the candidate box and the ground-truth box, α represents the balance factor, and γ represents the degree of easy defocusing of the hyperparameter adjustment.

[0022] Generalized intersection-over-union loss function:

[0023]

[0024] Among them, GIoU represents the generalized intersection-over-union loss function, A represents the predicted box, B represents the true box, C represents the minimum convex hull matrix containing the predicted box and the true box, and IoU represents the intersection-over-union ratio.

[0025] The beneficial effects of the present invention are as follows: the present invention improves the backbone network of the YOLOv8 model by adding a polykernel network, and improves the feature extraction capability of the backbone network for remote sensing images with large target scale changes and complex backgrounds through adaptive feature extraction of local and global context information; uses a dynamic upsampling strategy and adds a dynamic detection head to realize adaptive adjustment of the structure to detect targets of different sizes, optimizes blurred targets, and improves the accuracy of overall target detection; uses a zoom loss function to optimize the joint representation of target confidence and positioning accuracy, improves the effect of processing dense targets, and reduces the missed detection rate; uses generalized intersection-over-union (IoU) to increase the attention of the target detection model to difficult-to-detect targets, and improves the detection accuracy of the model.

[0026] Furthermore, the S2 includes the following steps:

[0027] S201, inputting the remote sensing image into the backbone network of the optimized YOLOv8 model, performing convolution processing using the convolution module to obtain convolution data;

[0028] S202, using the cross-stage feature fusion submodule in the cross-stage feature fusion module of the multi-core, dividing the convolution data into two branches for processing, wherein one branch outputs the convolution data, and the other branch processes the convolution data through multiple bottleneck submodules and then outputs the convolution data, and the outputs of the branches are combined to obtain the output data;

[0029] S203, using the polykernel network module to process the output data, using the polykernel submodule to capture local context information, and using the context-anchored attention submodule to capture long-range context information of the output data;

[0030] S204, based on the captured local context information and long-range context information, downsampling is performed using the backbone network of the optimized YOLOv8 model to obtain feature maps of different scales;

[0031] Furthermore, the expression of the convolution process is as follows:

[0032]

[0033] in, Represents a convolution operation with a kernel size of k*k, a step size of s, and a number of channels of c. represents a feature map with width w, height h, and number of channels c, SiLU(·) represents the activation function, and BN(·) represents batch normalization; Represents a two-dimensional convolution method with a convolution kernel size of k*k and a step size of s.

[0034] Furthermore, the expression of the cross-stage feature fusion module based on the multi-kernel is as follows:

[0035]

[0036]

[0037]

[0038]

[0039]

[0040] stk (m) =(m+1)×2+1

[0041]

[0042]

[0043] Among them, C2fPKI represents the cross-stage feature fusion module based on multi-core, PKINet represents the multi-core network module, C2f represents the cross-stage feature fusion submodule, and F c w×h Represents a feature map with a width of w, a height of h, and a number of channels of c, conv 1×1 represents the convolution operation with a convolution kernel size of 1*1, PKI represents the multi-scale local context information extracted by the poly kernel submodule, and CAA represents the feature information output by the upper and lower anchor attention submodule. represents primitive-by-primitive and concatenation operations, Represents element-by-element multiplication and connection operation, L represents local features, Z (m)Represents the contextual features extracted by the mth depth convolution, conv ks×ks Indicates the convolution operation with a convolution kernel size of ks*ks, DWConv 3×3 Indicates the depth convolution operation with a convolution kernel size of 3*3, k (m) Represents the convolution kernel size, m represents the depth convolution of size m, Sigmoid(·) represents the S-shaped growth curve, Represents the enhanced features, DWConv kb×1 Indicates the depth convolution operation with a convolution kernel size of kb*1, DWConv 1×kb Indicates a depthwise convolution operation with a convolution kernel size of 1*kb. Represents the feature map of the nth stage with width w and height h.

[0044] The beneficial effects of the above further scheme are as follows: in the cross-stage feature fusion module based on poly-kernels, the present invention adopts non-expanded multi-scale convolution kernels to extract target features of different scales and capture local context, and introduces a context-anchored attention module to capture remote context information, which promotes the adaptive feature extraction of local and global context information and solves the problems of large target scale changes and complex background in remote sensing images.

[0045] Furthermore, the S3 includes the following steps:

[0046] S301. Dynamically upsample feature maps of different scales using the optimized neck network of the YOLOv8 model to obtain an original sampling network.

[0047] S302: Processing feature maps of different scales using a first linear projection to generate an offset set, reshaping the offset set using an S-shaped growth curve function and a static factor to obtain reshaped features, and processing the reshaped features using a second linear projection to obtain an offset, summing the offset and the original sampling network to obtain a sampling set;

[0048] S303. Using the network sampling function, resample the assumed bilinear interpolation according to the position in the sampling set to obtain a content-aware upsampling result, generate an upsampling feature map, and use the neck network to output the upsampling feature map.

[0049] Furthermore, the expression of the upsampled feature map is as follows:

[0050] X′=gridsample(X,S)

[0051] S=G+O

[0052] O=0.5sigmoid(linear1(X))·linear2(X)

[0053] Where X′ represents the upsampled feature map, gridsample(·) represents the network sampling function, X represents the feature maps of different scales, S represents the sampling set, G represents the original sampling network, O represents the offset, 0.5 represents the static factor, sigmoid(·) represents the S-shaped growth curve, linear1(·) represents the first linear projection, and linear2(·) represents the second linear projection.

[0054] The beneficial effects of the above further scheme are as follows: the present invention constructs a dynamic feature pyramid network and uses dynamic upsampling, which reduces artifacts in the upsampling process, improves the ability to identify remote sensing images with a large number of targets and dense distribution, reduces the upsampling information loss rate and error rate, and improves the image quality during feature fusion and the accuracy of the sampling method.

[0055] Furthermore, the S4 includes the following steps:

[0056] S401, in response to the upsampled feature map output by the neck network, adding a dynamic detection head, unifying scale perception, spatial perception, and task perception, and obtaining a four-dimensional tensor of a rescaled feature pyramid;

[0057] S402. According to the attention mechanism, convolution is used to approximate the four-dimensional tensor and activated by a hard S-shaped curve function to reshape the four-dimensional tensor into a three-dimensional tensor to obtain scale-aware attention.

[0058] S403. Based on the attention mechanism, combined with the four-dimensional tensor, deformable convolution is used to sparse the attention learning, and features are integrated across scales to obtain spatial perception attention;

[0059] S404. Based on the attention mechanism, perform global average pooling on the four-dimensional tensor to reduce the dimensionality, use hyperparameters to control the threshold of the activation function, and use fully connected layers and normalization layers to process the reduced dimensionality tensor to obtain task-aware attention.

[0060] S405. Focus the scale-aware attention, space-aware attention, and task-aware attention on a unified dimension to obtain unified attention. Based on the upsampled feature map output by the neck network and the unified attention, the connection module is used to perform feature fusion to obtain a fused feature map.

[0061] Furthermore, the expression of the unified attention is as follows:

[0062] W(F)=π C (π S (π L (F)·F)·F)·F

[0063]

[0064]

[0065] π C (F)·F=max(α 1 (F)·F c +β 1 ,α 2 (F)·F c +β 2 )

[0066] Among them, W(F) represents unified attention, π L (F) represents scale-aware attention, π S (F) represents spatial perception attention, π C (F) represents task-aware attention, F represents a four-dimensional tensor, and the four dimensions include L, H, W, and C. σ(·) represents a hard S-shaped growth curve, f(·) represents a linear function approximated by convolution, S represents the product of dimension H and dimension W, K represents the number of sparse sampling positions, l represents the lth number of dimension L, k represents the kth number of sparse sampling positions, and w l,k represents the weight of the corresponding position, represents the self-learning space offset, p k represents the spatial position, c represents the number of channels, Δ mk Indicates p k The self-learning scalar at position, α 1 ,α 2 ,β 1 ,β 2 Both represent hyperparameters and are similar to dynamic linear rectification functions, F c Representing task-aware feature maps.

[0067] The beneficial effects of the above further scheme are: the present invention adds a dynamic detection head to identify remote sensing images with large changes in target scale, and optimizes blurred targets, thereby enhancing the multi-scale detection capability of the model, realizing adaptive adjustment of the structure to detect targets of different sizes, and improving the overall target detection accuracy.

[0068] Furthermore, the S5 includes the following steps:

[0069] S501, combining the confidence and positioning accuracy of the fused feature map to obtain a joint representation, optimizing the joint representation using a zoom loss function, and obtaining a classified fused feature map;

[0070] S502. Detect the classified fusion feature map using the classification function in combination with the generalized intersection-over-union loss function to obtain the classified target detection result, thereby completing the dynamic detection of the remote sensing image target.

[0071] The beneficial effects of the above further scheme are as follows: the present invention uses the zoom loss function and the generalized intersection-over-union loss function to optimize the joint representation of the confidence and positioning accuracy of the target features, and selects the minimum closed area of ​​the predicted box and the real box, thereby realizing the classification optimization of the target detection algorithm, improving the processing effect of the present invention on dense targets, reducing the missed detection rate, and improving the ability to distinguish difficult-to-detect targets, thereby improving the detection accuracy of the YOLOv8 model. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 Flow chart of the method of the present invention.

[0073] Figure 2 2 is the overall framework diagram of the dynamic detection algorithm based on the poly-kernel in this embodiment.

[0074] Figure 3 This is the overall framework diagram of the multi-core network in this embodiment.

[0075] Figure 4 This is a comparison chart of the original image, the YOLOv8 model detection result, and the detection result of the present invention in this embodiment. DETAILED DESCRIPTION

[0076] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0077] Before describing this embodiment, the following terms are explained:

[0078] C2f module: cross-stage feature fusion module;

[0079] PKI module: polycore module;

[0080] CAA module: upper and lower anchor attention module;

[0081] C2fPKI module: a cross-stage feature fusion module based on multi-core;

[0082] PKINet: PolyKernel Network;

[0083] Neck network: neck network;

[0084] Conv module: convolution module;

[0085] gridsample function: network sampling function;

[0086] Sigmoid function: S-shaped growth curve function;

[0087] hard-sigmoid function: hard S-shaped growth curve function;

[0088] Varifocal Loss function: zoom loss function;

[0089] IoU: intersection over union;

[0090] GIoU: Generalized Intersection over Union;

[0091] CIoU: Complete Intersection over Union;

[0092] AP: Average Precision;

[0093] HRRSD dataset: High-resolution remote sensing detection dataset;

[0094] RtmDets: Rotated object detection method.

[0095] Example 1

[0096] like Figure 1 As shown, the present invention provides a remote sensing image target dynamic detection method based on poly-kernel, and its implementation method is as follows:

[0097] S1. Collect remote sensing images and optimize the YOLOv8 model to obtain an optimized YOLOv8 model. The optimized YOLOv8 model includes:

[0098] The polykernel network module is used to improve the cross-stage feature fusion module in the backbone network of the YOLOv8 model;

[0099] A cross-stage feature fusion module based on multi-core is used to extract information from remote sensing images and obtain feature maps of different scales through downsampling;

[0100] The dynamic feature pyramid network is used to dynamically upsample feature maps of different scales using the neck network to generate upsampled feature maps, and the neck network is used to output the upsampled feature maps. The obtained feature maps are input into the dynamic detection head to optimize and fuse the feature maps.

[0101] In this embodiment, in the actual remote sensing image detection task, it is necessary to ensure the accuracy and speed of detection. In view of the characteristics of remote sensing images such as dense target distribution, a large number of small targets, large target scale variations, target blur and complex background, the present invention improves on YOLOv8 and proposes a dynamic remote sensing satellite image detection algorithm based on multi-core. First, the C2f module in the original YOLOv8 is improved and a new C2fPKI module is proposed to enhance the feature extraction capability of the backbone network; a dynamic pyramid network is designed and combined with a dynamic detection head network to enable the algorithm to dynamically adjust the target size in the remote sensing image and improve the network feature fusion capability; finally, the loss function is improved so that the model can optimize the detection capability of multiple dense targets and improve the detection accuracy of the algorithm;

[0102] Among them, YOLOv8 has the fastest detection speed and can meet the detection accuracy requirements. Therefore, based on this, we propose a dynamic detection algorithm based on multi-core that has stronger feature extraction capabilities and is more suitable for remote sensing image detection. The overall framework is as follows: Figure 2 shown.

[0103] In this embodiment, the YOLOv8 model is improved, specifically including the following tasks:

[0104] A. Improve the backbone network. The PKINet module is used to improve the backbone of the YOLOv8 model. Multi-scale convolution kernels without expansion are used to extract target features at different scales and capture local context. A contextual anchor attention module is introduced to capture remote context information.

[0105] B. Improve the upsampling method and build a dynamic feature pyramid network. Change the upsampling method in the original Neck network to dynamic upsampling, dynamically adjust the upsampling strategy according to the image content, input the obtained feature map into the dynamic detection head, optimize the feature map and perform feature fusion;

[0106] C. Improve classification loss and introduce Varifocal Loss and Giou Loss loss functions. Varifocal Loss is a loss function designed for dense target detection. It optimizes the joint representation of target confidence and positioning accuracy. Giou Loss considers the minimum closed area of ​​the predicted box and the true box to improve the accuracy of target detection.

[0107] S2. Use the optimized YOLOv8 model backbone network to extract information from remote sensing images and obtain feature maps of different scales through downsampling. The specific steps are as follows:

[0108] S201, inputting the remote sensing image into the backbone network of the optimized YOLOv8 model, performing convolution processing using the convolution module to obtain convolution data;

[0109] S202, using the cross-stage feature fusion submodule in the cross-stage feature fusion module of the multi-core, dividing the convolution data into two branches for processing, wherein one branch outputs the convolution data, and the other branch processes the convolution data through multiple bottleneck submodules and then outputs the convolution data, and the outputs of the branches are combined to obtain the output data;

[0110] S203, using the polykernel network module to process the output data, using the polykernel submodule to capture local context information, and using the context-anchored attention submodule to capture long-range context information of the output data;

[0111] S204: Based on the captured local context information and long-range context information, downsampling is performed using the optimized YOLOv8 model backbone network to obtain feature maps of different scales.

[0112] In this embodiment, the backbone network consists of five convolutional modules, four improved cross-stage feature fusion modules based on multi-core kernels, and a fast spatial pyramid pooling module. Feature maps of different scales are obtained through downsampling of the backbone network.

[0113] First, a remote sensing image of size 640×640 is input into the backbone network. The input image is convolved and feature extracted through the Conv layer of P1. The expression of the convolution process is as follows:

[0114]

[0115] in, Represents a convolution operation with a kernel size of k*k, a step size of s, and a number of channels of c. represents a feature map with width w, height h, and number of channels c, SiLU(·) represents the activation function, and BN(·) represents batch normalization; Represents a two-dimensional convolution method with a convolution kernel size of k*k and a step size of s;

[0116] The C2f module divides the input data into two branches for processing: one branch is directly passed to the output, and the other branch is processed by multiple bottleneck modules to obtain the output data. The output data of the C2f module is processed by the multi-kernel network, using convolution kernels of different sizes to capture target features of different sizes, enhancing the model's ability to recognize various targets. The CAA module is then combined to capture long-range contextual information, enabling the model to better understand the global context of the image. The expression of the multi-kernel network is shown below:

[0117]

[0118] Among them, PKINet represents the polykernel network module, PKI represents the multi-scale local context information extracted by the polykernel module, and CAA represents the feature information output by the upper and lower anchor attention modules. represents primitive-by-primitive and concatenation operations, Represents element-wise multiplication and concatenation operation;

[0119] like Figure 3 As shown in Figure 1, the polykernel network consists of two parts: a PKI module and a CAA module. The PKI module captures multi-scale texture features, obtains local information through small kernel convolution, and then a set of parallel depth convolutions to capture contextual information across multiple scales. Finally, the feature information of multiple scales is integrated through element-by-element summation. The calculation expression of the PKI module is as follows:

[0120]

[0121]

[0122]

[0123] stk (m) =(m+1)×2+1

[0124] Among them, L represents the local feature, Z (m) Represents the contextual features extracted by the mth depth convolution, conv ks×ks Indicates the convolution operation with a convolution kernel size of ks*ks, DWConv 3×3 Indicates the depth convolution operation with a convolution kernel size of 3*3, k (m) Represents the convolution kernel size, m represents the depth convolution of size m;

[0125] Since the PKI module focuses on extracting multi-scale local context information, the CAA module is integrated into PKINet to capture remote context information. The CAA module grasps the contextual interdependence between distant pixels, enhances the central features, and connects the two parts. The calculation expression of the CAA module is as follows:

[0126]

[0127]

[0128] Among them, Sigmoid(·) represents the S-shaped growth curve, Represents the enhanced features, DWConv kb×1 Indicates the depth convolution operation with a convolution kernel size of kb*1, DWConv 1×kbIndicates a depthwise convolution operation with a convolution kernel size of 1*kb. Represents the feature map of stage n with width w and height h;

[0129] Combining the C2f module with the PKINet module, we obtain an improved C2fPKI module, which can improve the backbone network's feature extraction capability and solve the problems of large target scale variations and complex backgrounds in remote sensing image target detection. The expression of the C2fPKI module is as follows:

[0130]

[0131] Among them, C2fPKI represents the cross-stage feature fusion module based on the multi-core, PKINet represents the multi-core network module, and C2f represents the cross-stage feature fusion module; and the data is output to the Neck network through the C2fPKI module.

[0132] S3. Use the neck network of the optimized YOLOv8 model to dynamically upsample feature maps of different scales to generate upsampled feature maps, and use the neck network to output the upsampled feature maps. The specific steps are as follows:

[0133] S301. Dynamically upsample feature maps of different scales using the optimized neck network of the YOLOv8 model to obtain an original sampling network.

[0134] S302: Processing feature maps of different scales using a first linear projection to generate an offset set, reshaping the offset set using an S-shaped growth curve function and a static factor to obtain reshaped features, and processing the reshaped features using a second linear projection to obtain an offset, summing the offset and the original sampling network to obtain a sampling set;

[0135] S303. Using the network sampling function, resample the assumed bilinear interpolation according to the position in the sampling set to obtain a content-aware upsampling result, generate an upsampling feature map, and use the neck network to output the upsampling feature map.

[0136] In this embodiment, since there are many targets and they are densely distributed, information is lost during feature fusion. Therefore, the upsampling method on the Neck network in YOLOv8 is improved, and the dynamic upsampling method is used to replace the original upsampling method to enhance the detection capability of dense targets. The expression of the dynamic upsampling is as follows:

[0137] X′=gridsample(X,S)

[0138] S=G+O

[0139] O=0.5sigmoid(linear1(X))·linear2(X)

[0140] Where X′ represents the upsampled feature map of size C×H2×W2, gridsample(·) represents the network sampling function, X represents the feature maps of different scales of size C×H1×W1, S represents the sampling set of size 2×H2×W2, 2 represents the coordinates of x and y, G represents the original sampling network, O represents the offset, 0.5 represents the static factor, sigmoid(·) represents the S-shaped growth curve, linear1(·) represents the first linear projection, and linear2(·) represents the second linear projection;

[0141] The flexibility of the offset is increased by generating a point-by-point "dynamic range factor" by linearly projecting the input features; by using the sigmoid function and a static factor of 0.5, the dynamic range value is between [0, 0.5], centered at 0.25, the same as the static range; s is generated using linear projection 2 The offset set is reshaped to meet the spatial size, and the feature X is reshaped into And linearly project it to 2g×sH×sW to get the offset O, and sum the offset O and the original sampling network G to get the sampling set S.

[0142] S4. In response to the upsampled feature map output by the neck network, a dynamic detection head is added and the upsampled feature map is optimized using the attention mechanism. The optimized upsampled feature map is then subjected to feature fusion to obtain a fused feature map. The specific steps are as follows:

[0143] S401, in response to the upsampled feature map output by the neck network, adding a dynamic detection head, unifying scale perception, spatial perception, and task perception, and obtaining a four-dimensional tensor of a rescaled feature pyramid;

[0144] S402. According to the attention mechanism, convolution is used to approximate the four-dimensional tensor and activated by a hard S-shaped curve function to reshape the four-dimensional tensor into a three-dimensional tensor to obtain scale-aware attention.

[0145] S403. Based on the attention mechanism, combined with the four-dimensional tensor, deformable convolution is used to sparse the attention learning, and features are integrated across scales to obtain spatial perception attention;

[0146] S404. Based on the attention mechanism, perform global average pooling on the four-dimensional tensor to reduce the dimensionality, use hyperparameters to control the threshold of the activation function, and use fully connected layers and normalization layers to process the reduced dimensionality tensor to obtain task-aware attention.

[0147] S405. Focus the scale-aware attention, space-aware attention, and task-aware attention on a unified dimension to obtain unified attention. Based on the upsampled feature map output by the neck network and the unified attention, the connection module is used to perform feature fusion to obtain a fused feature map.

[0148] In this embodiment, a dynamic detection head is added to the output of the Neck network to improve the detection head. By unifying scale perception, spatial perception and task perception, the attention mechanism is applied in different dimensions to improve the model's ability to handle large scale changes of targets in remote sensing images. At the same time, it is optimized for blurred targets, thereby improving the overall accuracy. The rescaled feature pyramid can be described as a four-dimensional tensor F∈R L×H×W×C , and define S = H × W to reshape the four-dimensional tensor into a three-dimensional tensor F∈R L×S×C , and calculate scale perception, spatial perception and task perception;

[0149] Scale-aware attention fuses features of different scales based on their semantic importance. The calculation expression is as follows:

[0150]

[0151] Among them, π L (F) represents scale-aware attention, f(·) represents a linear function approximated by 1*1 convolution, and σ(·) represents a hard S-shaped growth curve to reshape the four-dimensional tensor;

[0152] Spatial perception attention focuses on the ability to distinguish different spatial positions. Considering the high-dimensional S, S is decoupled, and deformation convolution is used to sparse the attention learning. Then, features are integrated across scales. The calculation expression is as follows:

[0153]

[0154] Among them, π S (F) represents spatial perception attention, K represents the number of sparse sampling positions, l represents the lth number of dimension L, k represents the kth number of sparse sampling positions, and w l,k represents the weight of the corresponding position, represents the self-learning space offset, p k represents the spatial position, c represents the number of channels, Indicates p k The self-learning scalar at position, and Both are learned from the median-level input features of the four-dimensional tensor F;

[0155] Task-aware attention promotes the generalization of joint learning and target expression capabilities. It can dynamically switch feature channels to assist different tasks, use hyperparameters to control the activation threshold, perform global average pooling on the L×S dimension to reduce the dimensionality, and use two fully connected layers and a normalization layer. Finally, a shifted sigmoid function is applied to normalize the output data to the interval [-1, 1]. The calculation expression of the task-aware attention is as follows:

[0156] π C (F)·F=max(α 1 (F)·F c +β 1 ,α 2 (F)·F c +β 2 )

[0157] Among them, π C (F) represents task-aware attention, α 1 ,α 2 ,β 1 ,β 2 Both represent hyperparameters and are similar to dynamic linear rectification functions, F c A feature map representing task perception;

[0158] Focusing scale-aware attention, space-aware attention, and task-aware attention on a unified dimension, we obtain unified attention. The expression of the unified attention is as follows:

[0159] W(F)=π C (π S (π L (F)·F)·F)·F

[0160] Among them, W(F) represents unified attention.

[0161] S5. Detect and classify the fused feature map to obtain the classified target detection results, and complete the dynamic detection of remote sensing image targets. The specific steps are as follows:

[0162] S501, combining the confidence and positioning accuracy of the fused feature map to obtain a joint representation, optimizing the joint representation using a zoom loss function, and obtaining a classified fused feature map;

[0163] S502. Detect the classified fusion feature map using the classification function in combination with the generalized intersection-over-union loss function to obtain the classified target detection result, thereby completing the dynamic detection of the remote sensing image target.

[0164] In this embodiment, the classification loss function of the YOLOv8 model is replaced by the Varifocal Loss function, which optimizes the joint representation of target confidence and positioning accuracy. It has better effects on dense targets in remote sensing images and reduces the missed detection rate. The expression of the Varifocal Loss function is as follows:

[0165]

[0166] Where VFL(·) represents the zoom loss function, p represents the predicted IoU perceptual classification score, q represents the IoU ratio between the candidate box and the ground-truth box, α represents the balance factor used to balance the ratio of positive and negative samples, and γ represents the degree of easy defocusing of the hyperparameter adjustment. For negative samples, that is, candidate boxes that are not responsible for the ground-truth box, their q value is 0.

[0167] The GIoU Loss function is used to consider the minimum closed area of ​​the predicted box and the true box, so that the model pays more attention to the target that is difficult to detect, thereby improving the detection accuracy of the model. The expression of the GIoU Loss function is as follows:

[0168]

[0169] GIoU stands for Generalized Intersection over Union (GIoU) ​​loss function, A represents the predicted bounding box, B represents the ground-truth bounding box, C represents the minimum convex hull matrix containing the predicted and ground-truth bounding boxes, and IoU stands for Intersection over Union (IoU). It is used to optimize target detection algorithms, especially when dealing with occlusion and size changes. It can provide more accurate matching assessments. A high GIoU value means that the two bounding boxes not only have a large overlap but also have similar shapes. These two loss functions can be used to optimize the algorithm, address the problem of densely packed targets in remote sensing images and the difficulty of target detection, reduce the target miss detection rate, and thus improve the detection accuracy of the model.

[0170] The target detection algorithm is used in combination with the generalized intersection-over-union loss function to detect the classified fusion feature map, obtain the classified target detection results, and complete the dynamic detection of remote sensing image targets.

[0171] Example 2

[0172] To ensure experimental fairness, all experiments in this example were conducted in the same experimental environment and training parameter configuration. The experimental environment was Windows 10 operating system, the CPU was an Intel Core i7-13700K 3.40GHz, the graphics card was an NVIDIA GeForce RTX 4080 with 16GB of video memory, and training was performed on a single GPU graphics card. The deep learning framework was PyTorch 1.12.1, the compilation environment was CUDA 11.3, and Python 3.8.18 was used. The training parameters were: 500 iterations, 16 batch size, 4 threads, and all other parameters kept their default values.

[0173] In order to evaluate the improvement effect of the present invention, the accuracy series index (AP) is used. The larger the value, the higher the detection accuracy. Among them, AP50 represents the average accuracy when the intersection-over-union ratio is 0.5, and AP75 represents the average accuracy when the intersection-over-union ratio is 0.75. Compared with the former, AP75 is more stringent on the detection results. The average precision mean is the most important indicator, which represents the average value of AP with a total of 10 thresholds between 0.5 and 0.95, and the threshold interval is 0.05. APs of different sizes are divided into APs, APm and AP1, which represent the detection results of small targets, medium targets and large targets respectively. The definition of a small target is a pixel area <32 2 , the definition of medium target is pixel area 32 2 <area<96 2 , the definition of a large target is a pixel area>96 2 .

[0174] In this example, the algorithm was validated using the HRRSD dataset, a general remote sensing image dataset. The dataset contains 13 categories: airplane, baseball field, basketball court, bridge, intersection, track and field, harbor, parking lot, ship, storage tank, T-junction, tennis court, and car. During the experiment, the HRRSD dataset was divided into a training set and a test set in a 4:1 ratio. The training set contained 17,409 images, and the test set contained 4,352 images.

[0175] In this embodiment, a comparative experiment is conducted on the HRRSD dataset, and the method of the present invention is compared with the target detection algorithm commonly used in recent years and the improved algorithm for remote sensing image detection; the comparative experiment of the HRRSD dataset is shown in Table 1. It can be seen from the table that the best value of each data is obtained by the method of the present invention, indicating that the present invention is the best; the average accuracy of the method proposed in the present invention is improved by 5.9 percentage points compared with the original YOLOv8. Compared with the same type of improved algorithm, the experimental results show that the present invention has more obvious improvement and higher detection accuracy than the same type of remote sensing image detection algorithm.

[0176] Table 1

[0177]

[0178]

[0179] In this embodiment, the YOLOv8 detection method and the detection method of the present invention are used to detect the test set images. Figure 4 The comparison of the detection results of the two detection methods is shown, where (a) is the original image, (b) is the detection result of the YOLOv8 algorithm, and (c) is the detection result of the proposed method. Different categories of targets are represented by prediction boxes of different colors, and the confidence scores are also shown.

[0180] As can be seen in the figure, in the first set of images, the proposed method more accurately identifies the aircraft target, with a confidence level of over 90, while the confidence level of the aircraft target detected by YOLOv8 is only around 83. In the second set of images, due to the blurriness of the image, the basketball court in the upper left corner was missed by YOLOv8, while the proposed method was able to accurately identify the target. In the third set of images, YOLOv8 mistakenly detected an open space as a baseball field, while the proposed method was able to accurately identify other targets. In summary, the proposed method has a higher detection effect, higher confidence scores for various target categories, and more accurate recognition and positioning of remote sensing image targets of various sizes.

Claims

1. A method for dynamic target detection in remote sensing images based on poly-kernels, characterized in that: The following steps are involved: S1. Collect remote sensing images and optimize the YOLOv8 model to obtain an optimized YOLOv8 model; The optimized YOLOv8 model includes: The polykernel network module is used to improve the cross-stage feature fusion module in the backbone network of the YOLOv8 model; A cross-stage feature fusion module based on multi-core is used to extract information from remote sensing images and obtain feature maps of different scales through downsampling; Dynamic feature pyramid network, which uses the neck network to dynamically upsample feature maps of different scales to generate upsampled feature maps. The neck network outputs the upsampled feature maps and inputs the obtained feature maps into the dynamic detection head for feature map optimization and feature fusion. The expression of the cross-stage feature fusion module based on poly-kernel is as follows: in, represents the cross-stage feature fusion module based on multi-kernel, Represents a multi-core network module, represents the cross-stage feature fusion submodule, Indicates width w Gao Wei h , the number of channels is c The feature map of Indicates that the convolution kernel size is The convolution operation, represents the multi-scale local context information extracted by the poly kernel submodule, represents the feature information output by the upper and lower anchor attention submodules, represents primitive-by-primitive and concatenation operations, represents the element-wise multiplication and concatenation operation, Represents local features, Indicates the m Context features extracted by deep convolution, Indicates that the convolution kernel size is The convolution operation, Indicates that the convolution kernel size is The depth convolution operation, represents the convolution kernel size, m Indicates size m The depth of the convolution, represents an S-shaped growth curve, represents the enhanced features, Indicates that the convolution kernel size is The depth convolution operation, Indicates that the convolution kernel size is The depth convolution operation, Indicates the n The stage width is w Gao Wei h Feature map of S2. Use the optimized YOLOv8 model backbone network to extract information from remote sensing images and obtain feature maps of different scales through downsampling; S3. Use the neck network of the optimized YOLOv8 model to dynamically upsample feature maps of different scales to generate upsampled feature maps, and use the neck network to output the upsampled feature maps; S4. In response to the upsampled feature map output by the neck network, a dynamic detection head is added and the upsampled feature map is optimized using the attention mechanism. The optimized upsampled feature map is then subjected to feature fusion to obtain a fused feature map, specifically: S401, in response to the upsampled feature map output by the neck network, adding a dynamic detection head, unifying scale perception, spatial perception, and task perception, and obtaining a four-dimensional tensor of a rescaled feature pyramid; S402. According to the attention mechanism, convolution is used to approximate the four-dimensional tensor and activated by a hard S-shaped curve function to reshape the four-dimensional tensor into a three-dimensional tensor to obtain scale-aware attention. S403. Based on the attention mechanism, combined with the four-dimensional tensor, deformable convolution is used to sparse the attention learning, and features are integrated across scales to obtain spatial perception attention; S404. Based on the attention mechanism, perform global average pooling on the four-dimensional tensor to reduce the dimensionality, use hyperparameters to control the threshold of the activation function, and use fully connected layers and normalization layers to process the reduced dimensionality tensor to obtain task-aware attention. S405, focusing the scale-aware attention, space-aware attention, and task-aware attention on a unified dimension to obtain a unified attention, and performing feature fusion using a connection module based on the upsampled feature map output by the neck network and the unified attention to obtain a fused feature map; S5. Detect and classify the fused feature map to obtain the classified target detection results, completing the dynamic detection of remote sensing image targets.

2. The method for dynamic detection of remote sensing image targets based on poly-kernel according to claim 1, characterized in that: The loss function of the optimized YOLOv8 model includes: Zoom loss function: in, represents the zoom loss function, represents the predicted IoU-aware classification score, represents the intersection-over-union ratio between the candidate box and the real box, represents the balance factor, Indicates how easy it is to lose focus in hyperparameter tuning; Generalized intersection-over-union loss function: in, represents the generalized intersection-over-union loss function, represents the prediction box, represents the true frame, Transform the minimum convex hull matrix containing the predicted box and the true box, Represents intersection and union ratio.

3. The method for dynamic target detection in remote sensing images based on poly-kernel according to claim 2, characterized in that: The S2 comprises the following steps: S201, inputting the remote sensing image into the backbone network of the optimized YOLOv8 model, performing convolution processing using the convolution module to obtain convolution data; S202, using the cross-stage feature fusion submodule in the cross-stage feature fusion module of the multi-core, dividing the convolution data into two branches for processing, wherein one branch outputs the convolution data, and the other branch processes the convolution data through multiple bottleneck submodules and then outputs the convolution data, and the outputs of the branches are combined to obtain the output data; S203, using the polykernel network module to process the output data, using the polykernel submodule to capture local context information, and using the context-anchored attention submodule to capture long-range context information of the output data; S204, based on the captured local context information and long-range context information, downsampling is performed using the backbone network of the optimized YOLOv8 model to obtain feature maps of different scales; The expression of the convolution process is as follows: in, Indicates that the convolution kernel size is , the step length is s , the number of channels is c The convolution operation, Indicates width w Gao Wei h , the number of channels is c The feature map of represents the activation function, represents batch normalization; Indicates that the convolution kernel size is , the step length is s Two-dimensional convolution method.

4. The method for dynamic target detection in remote sensing images based on poly-kernel according to claim 2, characterized in that: The S3 includes the following steps: S301. Dynamically upsample feature maps of different scales using the optimized neck network of the YOLOv8 model to obtain an original sampling network. S302: Processing feature maps of different scales using a first linear projection to generate an offset set, reshaping the offset set using an S-shaped growth curve function and a static factor to obtain reshaped features, and processing the reshaped features using a second linear projection to obtain an offset, summing the offset and the original sampling network to obtain a sampling set; S303. Using the network sampling function, resample the assumed bilinear interpolation according to the position in the sampling set to obtain a content-aware upsampling result, generate an upsampling feature map, and use the neck network to output the upsampling feature map.

5. The method for dynamic detection of remote sensing image targets based on poly-kernel according to claim 4, characterized in that: The expression of the up-sampled feature map is as follows: in, represents the upsampled feature map, represents the network sampling function, Represents feature maps of different scales, represents the sampling set, represents the original sampling network, Indicates the offset, 0.5 indicates the static factor, represents an S-shaped growth curve, represents the first linear projection, Represents the second linear projection.

6. The method for dynamic detection of remote sensing image targets based on poly-kernel according to claim 2, characterized in that: The expression of unified attention is as follows: in, Indicates unified attention, represents scale-aware attention, Indicates spatial perception attention, represents task-aware attention, Represents a four-dimensional tensor, the four dimensions include , represents a hard S-shaped growth curve, represents a linear function approximated by convolution, Representation Dimension and dimensions The product of represents the number of sparse sampling positions, Representation Dimension No. Number, The first sparse sampling position Number, represents the weight of the corresponding position, represents the self-learning space offset, Indicates spatial location, Indicates the number of channels, express The self-learning scalar at position, Both represent hyperparameters and are similar to dynamic linear rectifier functions, Representing task-aware feature maps.

7. The method for dynamic detection of remote sensing image targets based on poly-kernel according to claim 2, characterized in that: The S5 comprises the following steps: S501, combining the confidence and positioning accuracy of the fused feature map to obtain a joint representation, optimizing the joint representation using a zoom loss function, and obtaining a classified fused feature map; S502. Detect the classified fusion feature map using the classification function in combination with the generalized intersection-over-union loss function to obtain the classified target detection result, thereby completing the dynamic detection of the remote sensing image target.

Images