Remote sensing image target dynamic detection method based on multi-kernel

Abstract

The invention provides a remote sensing image target dynamic detection method based on a multi-kernel, and belongs to the field of remote sensing image detection.The method comprises the steps that a remote sensing image is collected, a YOLOv8 model is optimized, information extraction is conducted on the remote sensing image through a multi-kernel-based cross-stage feature fusion module in a backbone network of the optimized YOLOv8 model, and the remote sensing image is obtained; obtaining feature maps of different scales; performing dynamic up-sampling on the feature map by using a neck network, outputting an up-sampled feature map in response to the neck network, adding a dynamic detection head, and performing optimization and feature fusion on the up-sampled feature map by using an attention mechanism to obtain a fused feature map; and classifying the feature map to obtain a classified target detection result. According to the method, the problems of insufficient experimental effect on a multi-category data set, large number of small targets, large scale change and dense distribution of remote sensing image targets and detection conditions under a complex background in the prior art are solved.

Description

Technical Field

[0001] The invention belongs to the field of remote sensing image detection, and in particular relates to a remote sensing image target dynamic detection method based on a multi-kernel. Background Art

[0002] Remote sensing images are a technology that uses sensors on various aviation and aerospace platforms to image distant targets. According to different imaging methods, they are divided into infrared, multispectral, synthetic aperture radar SAR and visible light images. Among them, infrared remote sensing technology uses equipment to obtain environmental parameters such as the temperature of the target to distinguish different objects, which has uncertainty in the detection results. Multispectral imaging has lower resolution and may be affected by environmental conditions such as light and atmospheric conditions, which requires additional calibration steps to ensure the accuracy of the data. Although synthetic aperture radar has strong penetration and can obtain information on targets obscured by the surface, synthetic aperture radar SAR data contains a lot of information and requires professional processing and analysis before use. In addition, the cost of the equipment is high, and the large terrain undulations will affect the imaging accuracy. The visible light imaging band is between 380-760 nanometers. Optical remote sensing images contain a lot of detailed information and can intuitively display the shape, color and detailed features of the target. With the development of satellite and remote sensing technology, the continuous use of advanced equipment has enabled researchers to obtain a large number of high-resolution remote sensing images. These images have higher image quality and more detailed information, enabling them to detect targets of interest in remote sensing images through target detection, thereby providing decision support for related tasks.

[0003] In recent years, with the development of deep learning, the deep features extracted by neural networks have stronger semantic representation ability and discrimination, which further improves the performance of target detection. However, remote sensing images have the characteristics of dense target distribution, large number of small targets, large target scale variation, target blur and complex background, which makes the existing detection methods unable to achieve good performance on remote sensing images.

[0004] In view of the characteristics of remote sensing images, the prior art proposes a remote sensing image target detection method based on improved YOLOv5, introduces a channel global attention mechanism in the backbone, and enhances the feature extraction capability of small targets. Although there is a certain effect improvement, the detection accuracy is low; the prior art improves the loss function on YOLOv5, improves the detection accuracy of aircraft targets in remote sensing images, and accelerates the convergence of the algorithm, but this method only targets a single target and cannot fully prove an effect on the detection of other targets; the prior art proposes a super-resolution and deep learning detection method in the direction of infrared small target detection, pre-processes the super-resolution image and performs multiple data enhancements. Although the data set is enhanced, the processing is cumbersome and time-consuming, and cannot meet the real-time requirements; the prior art performs lightweight design in the field of SAR image target detection, and proposes a SAR target detection algorithm enhanced by hybrid representation learning based on the unique features of SAR images. Although this method can be more suitable for SAR image data sets, the processing of such data sets is cumbersome and the versatility is relatively low.

[0005] For remote sensing image data sets, the prior art improves the real-time target detection RTMDet (Real-time Models for Object Detection, RTMDet) algorithm. Although the algorithm detection accuracy is improved, the model has a large number of parameters and the detection takes a long time, which cannot meet the real-time requirements. Based on the prior art YOLOv5, the proposed YOLO-RS algorithm introduces adaptive spatial feature fusion to improve the feature fusion capability, but does not take into account the large target scale changes and target blurring in remote sensing images, and the detection effect of the algorithm is limited. The prior art combines the latest YOLOv8 algorithm, and designs a fast convolution module to reconstruct the C2f module of YOLOv8 on the basis of the good detection effect of the baseline model, which enhances the feature extraction ability of the network. Compared with the mainstream algorithm, the detection effect is improved to a certain extent, but the improvement is relatively small, and the model still has room for further improvement. The prior art proposes an optical remote sensing aircraft target fast detection algorithm for embedded platforms, which improves the characteristics of aircraft targets and is deployed in combination with hardware, but it cannot be proved whether it is effective in multiple categories, and the algorithm is not universal.

[0006] The above algorithms have made some improvements in the field of remote sensing image target detection, but there are still the following problems: 1. The selected data sets have few categories or only a single category, which cannot fully prove the experimental effect on multi-category data sets; 2. There is still room for improvement in the algorithms for general optical remote sensing images, and they have not been able to solve the problems of large scale variations and dense distribution of remote sensing image targets; 3. The ground objects in remote sensing images often have complex backgrounds and diverse scale variations, and these algorithms have not been able to solve the detection situation under complex backgrounds. Summary of the invention

[0007] In view of the above-mentioned deficiencies in the prior art, the present invention provides a remote sensing image target dynamic detection method based on poly-kernel, which solves the problems of insufficient experimental effect of the prior art on multi-category data sets, large number of small targets, large scale variation and dense distribution of remote sensing image targets, and detection in complex backgrounds.

[0008] In order to achieve the above objectives, the technical solution adopted by the present invention is: a remote sensing image target dynamic detection method based on poly-kernel, comprising the following steps:

[0009] S1, collect remote sensing images, and optimize the YOLOv8 model to obtain an optimized YOLOv8 model;

[0010] S2, use the optimized YOLOv8 model backbone network to extract information from remote sensing images, and obtain feature maps of different scales by downsampling;

[0011] S3. Use the neck network of the optimized YOLOv8 model to dynamically upsample feature maps of different scales, generate upsampled feature maps, and use the neck network to output the upsampled feature maps;

[0012] S4, in response to the upsampled feature map output by the neck network, adding a dynamic detection head and optimizing the upsampled feature map using an attention mechanism, and performing feature fusion on the optimized upsampled feature map to obtain a fused feature map;

[0013] S5. Detect and classify the fused feature map to obtain the classified target detection result, and complete the dynamic detection of the remote sensing image target.

[0014] Furthermore, the optimized YOLOv8 model includes:

[0015] The multi-kernel network module is used to improve the cross-stage feature fusion module in the backbone network of the YOLOv8 model;

[0016] A cross-stage feature fusion module based on poly-kernel is used to extract information from remote sensing images and obtain feature maps of different scales through downsampling;

[0017] The dynamic feature pyramid network is used to dynamically upsample feature maps of different scales using the neck network to generate upsampled feature maps, and the neck network is used to output the upsampled feature maps. The obtained feature maps are input into the dynamic detection head to optimize and fuse the feature maps.

[0018] Furthermore, the loss function of the optimized YOLOv8 model includes:

[0019] Zoom loss function:

[0020]

[0021] Where VFL(·) represents the zoom loss function, p represents the predicted IoU-perceptual classification score, q represents the IoU ratio between the candidate box and the true box, α represents the balance factor, and γ represents the degree of easy defocusing of the hyperparameter adjustment;

[0022] Generalized intersection-over-union loss function:

[0023]

[0024] Among them, GIoU represents the generalized intersection-over-union loss function, A represents the predicted box, B represents the true box, C represents the minimum convex hull matrix containing the predicted box and the true box, and IoU represents the intersection-over-union ratio.

[0025] The beneficial effects of the present invention are as follows: the present invention improves the backbone network of the YOLOv8 model by adding a poly-kernel network, and improves the feature extraction capability of the backbone network for remote sensing images with large target scale changes and complex backgrounds by adaptive feature extraction of local and global context information; uses a dynamic upsampling strategy and adds a dynamic detection head to realize adaptive adjustment of the structure to detect targets of different sizes, optimizes blurred targets, and improves the accuracy of overall target detection; uses a zoom loss function to optimize the joint representation of target confidence and positioning accuracy, improves the effect of processing dense targets, and reduces the missed detection rate; uses a generalized intersection-over-union ratio to increase the attention of the target detection model to difficult-to-detect targets, and improves the detection accuracy of the model.

[0026] Furthermore, the S2 comprises the following steps:

[0027] S201, inputting the remote sensing image into the backbone network of the optimized YOLOv8 model, performing convolution processing using a convolution module, and obtaining convolution data;

[0028] S202, using the cross-stage feature fusion submodule in the cross-stage feature fusion module of the multi-core, the convolution data is divided into two branches for processing, one branch outputs the convolution data, and the other branch outputs the convolution data after being processed by multiple bottleneck submodules, and the outputs of the branches are combined to obtain output data;

[0029] S203, using the poly-kernel network module to process the output data, using the poly-kernel sub-module to capture local context information, and using the context-anchored attention sub-module to capture remote context information of the output data;

[0030] S204, according to the captured local context information and remote context information, down-sampling is performed using the backbone network of the optimized YOLOv8 model to obtain feature maps of different scales;

[0031] Furthermore, the expression of the convolution process is as follows:

[0032]

[0033] in, It represents a convolution operation with a kernel size of k*k, a step size of s, and a number of channels of c. represents a feature map with width w, height h, and number of channels c, SiLU(·) represents the activation function, and BN(·) represents batch normalization; Represents a two-dimensional convolution method with a convolution kernel size of k*k and a step size of s.

[0034] Furthermore, the expression of the cross-stage feature fusion module based on poly-kernel is as follows:

[0035]

[0036]

[0037]

[0038]

[0039]

[0040] stk (m) =(m+1)×2+1

[0041]

[0042]

[0043] Among them, C2fPKI represents the cross-stage feature fusion module based on multi-core, PKINet represents the multi-core network module, C2f represents the cross-stage feature fusion submodule, and F c w×h Represents a feature map with a width of w, a height of h, and a number of channels of c, conv 1×1 represents the convolution operation with a kernel size of 1*1, PKI represents the multi-scale local context information extracted by the poly-kernel submodule, CAA represents the feature information output by the upper and lower anchor attention submodule, represents primitive and concatenation operations, represents the element-by-element multiplication operation, L represents the local feature, and Z (m)represents the contextual features extracted by the mth depth convolution, conv ks×ks Indicates the convolution operation with a convolution kernel size of ks*ks, DWConv 3×3 represents a depthwise convolution operation with a convolution kernel size of 3*3, k (m) represents the convolution kernel size, m represents the depth convolution of size m, Sigmoid(·) represents the S-shaped growth curve, Represents the enhanced features, DWConv kb×1 Indicates the depth convolution operation with a convolution kernel size of kb*1, DWConv 1×kb Indicates a deep convolution operation with a convolution kernel size of 1*kb. Represents the feature map of the nth stage with width w and height h.

[0044] The beneficial effects of the above further scheme are as follows: in the cross-stage feature fusion module based on poly-kernels, the present invention adopts a non-expanded multi-scale convolution kernel to extract target features of different scales and capture local context, and introduces a context anchor attention module to capture remote context information, which promotes the adaptive feature extraction of local and global context information and solves the problems of large target scale changes and complex background in remote sensing images.

[0045] Furthermore, the S3 comprises the following steps:

[0046] S301, dynamically up-sample feature maps of different scales using the neck network of the optimized YOLOv8 model to obtain an original sampling network;

[0047] S302, using the first linear projection to process feature maps of different scales to generate an offset set, and using the S-shaped growth curve function and the static factor to reshape the offset set to obtain reshaped features, and using the second linear projection to process the reshaped features to obtain an offset, and summing the offset and the original sampling network to obtain a sampling set;

[0048] S303, using the network sampling function, re-sample the assumed bilinear interpolation according to the position in the sampling set to obtain a content-aware upsampling result, generate an upsampling feature map, and output the upsampling feature map using the neck network.

[0049] Furthermore, the expression of the up-sampled feature map is as follows:

[0050] X′=gridsample(X,S)

[0051] S=G+O

[0052] O=0.5sigmoid(linear1(X))·linear2(X)

[0053] Among them, X′ represents the upsampled feature map, gridsample(·) represents the network sampling function, X represents the feature maps of different scales, S represents the sampling set, G represents the original sampling network, O represents the offset, 0.5 represents the static factor, sigmoid(·) represents the S-shaped growth curve, linear1(·) represents the first linear projection, and linear2(·) represents the second linear projection.

[0054] The beneficial effects of the above further scheme are as follows: the present invention constructs a dynamic feature pyramid network and uses dynamic upsampling, which reduces artifacts in the upsampling process, improves the ability to identify remote sensing images with a large number of targets and dense distribution, reduces the upsampling information loss rate and error rate, and improves the image quality during feature fusion and the accuracy of the sampling method.

[0055] Furthermore, the S4 comprises the following steps:

[0056] S401, in response to the upsampled feature map output by the neck network, adding a dynamic detection head, unifying scale perception, space perception and task perception, and obtaining a four-dimensional tensor of a rescaled feature pyramid;

[0057] S402, according to the attention mechanism, convolve the four-dimensional tensor using convolution approximation, and activate it using a hard S-curve function to reshape the four-dimensional tensor into a three-dimensional tensor to obtain scale-aware attention;

[0058] S403. Based on the attention mechanism, combined with the four-dimensional tensor, the deformable convolution is used to sparse the attention learning, and the features are integrated across scales to obtain spatial perception attention;

[0059] S404. According to the attention mechanism, global average pooling is performed on the four-dimensional tensor to reduce the dimension, a threshold of the activation function is controlled using a hyperparameter, and the reduced-dimensional tensor is processed using a fully connected layer and a normalization layer to obtain task-aware attention.

[0060] S405. Focus the scale-aware attention, space-aware attention, and task-aware attention on a unified dimension to obtain unified attention. Based on the upsampled feature map and the unified attention output by the neck network, a connection module is used to perform feature fusion to obtain a fused feature map.

[0061] Furthermore, the expression of the unified attention is as follows:

[0062] W(F)=π C (π S (π L (F)·F)·F)·F

[0063]

[0064]

[0065] π C (F)·F=max(α 1 (F)·F c +β 1 ,α 2 (F)·F c +β 2 )

[0066] Among them, W(F) represents unified attention, π L (F) represents scale-aware attention, π S (F) represents spatial perception attention, π C (F) represents task-aware attention, F represents a four-dimensional tensor, and the four dimensions include L, H, W, and C. σ(·) represents a hard S-shaped growth curve, f(·) represents a linear function approximated by convolution, S represents the product of dimension H and dimension W, K represents the number of sparse sampling positions, l represents the lth number of dimension L, k represents the kth number of sparse sampling positions, and w l,k represents the weight of the corresponding position, represents the self-learning space offset, p k represents the spatial position, c represents the number of channels, Δ mk Indicates p k The self-learning scalar at position, α 1 ,α 2 ,β 1 ,β 2 Both represent hyperparameters and are similar to dynamic linear rectification functions, F c Representing task-aware feature maps.

[0067] The beneficial effects of the above further scheme are as follows: the present invention adds a dynamic detection head to identify remote sensing images with large changes in target scale, and optimizes blurred targets, thereby enhancing the multi-scale detection capability of the model, realizing an adaptive adjustment structure to detect targets of different sizes, and improving the overall target detection accuracy.

[0068] Furthermore, the S5 comprises the following steps:

[0069] S501, combining the confidence and positioning accuracy of the fused feature map to obtain a joint representation, optimizing the joint representation using a zoom loss function, and obtaining a classified fused feature map;

[0070] S502, using the classification function and combining it with the generalized intersection-over-union loss function to detect the classified fusion feature map, obtain the classified target detection result, and complete the dynamic detection of the remote sensing image target.

[0071] The beneficial effects of the above further scheme are as follows: the present invention utilizes the zoom loss function and the generalized intersection-over-union loss function to optimize the joint representation of the confidence and positioning accuracy of the target features, and selects the minimum closed area of ​​the prediction box and the real box, thereby realizing the classification optimization of the target detection algorithm, improving the processing effect of the present invention on dense targets, reducing the missed detection rate, and improving the ability to distinguish difficult-to-detect targets, thereby improving the detection accuracy of the YOLOv8 model. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 The present invention is a flow chart of the method.

[0073] Figure 2 4 is an overall framework diagram of the dynamic detection algorithm based on the multi-core in this embodiment.

[0074] Figure 3 4 is an overall framework diagram of the multi-core network in this embodiment.

[0075] Figure 4 It is a comparison chart of the original image, the detection result of the YOLOv8 model and the detection result of the present invention in this embodiment. DETAILED DESCRIPTION

[0076] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0077] Before describing this embodiment, the following terms are explained:

[0078] C2f module: cross-stage feature fusion module;

[0079] PKI module: poly kernel module;

[0080] CAA module: upper and lower anchor attention module;

[0081] C2fPKI module: a cross-stage feature fusion module based on multi-core;

[0082] PKINet: PolyKernel Network;

[0083] Neck network: Neck network;

[0084] Conv module: convolution module;

[0085] gridsample function: network sampling function;

[0086] Sigmoid function: S-shaped growth curve function;

[0087] hard-sigmoid function: hard S-shaped growth curve function;

[0088] Varifocal Loss function: zoom loss function;

[0089] IoU: Intersection over Union;

[0090] GIoU: Generalized Intersection-over-Union;

[0091] CIoU: Complete Intersection over Union;

[0092] AP: Average Precision;

[0093] HRRSD dataset: High-resolution remote sensing detection dataset;

[0094] RtmDets: A rotated object detection method.

[0095] Example 1

[0096] like Figure 1 As shown, the present invention provides a remote sensing image target dynamic detection method based on poly-kernel, and its implementation method is as follows:

[0097] S1. Collect remote sensing images and optimize the YOLOv8 model to obtain an optimized YOLOv8 model. The optimized YOLOv8 model includes:

[0098] The multi-kernel network module is used to improve the cross-stage feature fusion module in the backbone network of the YOLOv8 model;

[0099] A cross-stage feature fusion module based on poly-kernel is used to extract information from remote sensing images and obtain feature maps of different scales through downsampling;

[0100] The dynamic feature pyramid network is used to dynamically upsample feature maps of different scales using the neck network to generate upsampled feature maps, and the neck network is used to output the upsampled feature maps. The obtained feature maps are input into the dynamic detection head to optimize and fuse the feature maps.

[0101] In this embodiment, in the actual remote sensing image detection task, it is necessary to ensure the accuracy and speed of detection. In view of the characteristics of remote sensing image targets such as dense distribution, large number of small targets, large changes in target scale, blurred targets and complex background, the present invention improves on YOLOv8 and proposes a dynamic remote sensing satellite image detection algorithm based on multi-core. First, the C2f module in the original YOLOv8 is improved, and a new C2fPKI module is proposed to enhance the feature extraction capability of the backbone network; and a dynamic pyramid network is designed, combined with a dynamic detection head network, so that the algorithm can dynamically adjust the target size in the remote sensing image and improve the network feature fusion capability; finally, the loss function is improved so that the model can optimize the detection capability of multiple dense targets and improve the detection accuracy of the algorithm;

[0102] Among them, YOLOv8 has the fastest detection speed and can meet the detection accuracy requirements. Therefore, based on this, we propose a dynamic detection algorithm based on multi-core that has stronger feature extraction capabilities and is more suitable for remote sensing image detection. The overall framework is as follows: Figure 2 shown.

[0103] In this embodiment, the YOLOv8 model is improved, specifically including the following tasks:

[0104] A. Improve the backbone network. The PKINet module is used to improve the backbone of the YOLOv8 model. The multi-scale convolution kernel without expansion is used to extract target features of different scales and capture local context. The contextual anchor attention module is introduced to capture remote context information.

[0105] B. Improve the upsampling method and build a dynamic feature pyramid network. Change the upsampling method in the original Neck network to dynamic upsampling, dynamically adjust the upsampling strategy according to the image content, input the obtained feature map into the dynamic detection head, optimize the feature map and fuse the features;

[0106] C. Improve the classification loss and introduce the Varifocal Loss and Giou Loss loss functions. Varifocal Loss is a loss function designed for dense target detection. It optimizes the joint representation of target confidence and positioning accuracy. Giou Loss considers the minimum closed area of ​​the predicted box and the true box to improve the accuracy of target detection.

[0107] S2. Use the optimized YOLOv8 model backbone network to extract information from remote sensing images and obtain feature maps of different scales by downsampling. The specific steps are as follows:

[0108] S201, inputting the remote sensing image into the backbone network of the optimized YOLOv8 model, performing convolution processing using a convolution module, and obtaining convolution data;

[0109] S202, using the cross-stage feature fusion submodule in the cross-stage feature fusion module of the multi-core, the convolution data is divided into two branches for processing, one branch outputs the convolution data, and the other branch outputs the convolution data after being processed by multiple bottleneck submodules, and the outputs of the branches are combined to obtain output data;

[0110] S203, using the poly-kernel network module to process the output data, using the poly-kernel sub-module to capture local context information, and using the context-anchored attention sub-module to capture remote context information of the output data;

[0111] S204: According to the captured local context information and long-range context information, down-sampling is performed using the backbone network of the optimized YOLOv8 model to obtain feature maps of different scales.

[0112] In this embodiment, the backbone network consists of five convolution modules, four improved cross-stage feature fusion modules based on multi-core, and a fast spatial pyramid pooling module, and feature maps of different scales are obtained by downsampling the backbone network;

[0113] First, a remote sensing image of size 640×640 is input into the backbone network, and the input image is convolved and feature extracted through the Conv layer of P1. The expression of the convolution processing is as follows:

[0114]

[0115] in, It represents a convolution operation with a kernel size of k*k, a step size of s, and a number of channels of c. represents a feature map with width w, height h, and number of channels c, SiLU(·) represents the activation function, and BN(·) represents batch normalization; Represents a two-dimensional convolution method with a convolution kernel size of k*k and a step size of s;

[0116] Since the C2f module divides the input data into two branches for processing: one branch is directly passed to the output, and the other branch is processed by multiple bottleneck modules to obtain the output data, the output data of the C2f module is processed by the poly-kernel network, and the convolution kernels of different sizes are used to capture the target features of different sizes, so as to enhance the model's recognition ability for various targets, and then combined with the CAA module to capture long-distance context information, so that the model can better understand the global context of the image; the expression of the poly-kernel network is shown as follows:

[0117]

[0118] Among them, PKINet represents the polykernel network module, PKI represents the multi-scale local context information extracted by the polykernel module, CAA represents the feature information output by the upper and lower anchor attention modules, represents primitive and concatenation operations, Represents an element-by-element multiplication join operation;

[0119] like Figure 3 As shown in the figure, the poly-kernel network consists of two parts: a PKI module and a CAA module. The PKI module captures multi-scale texture features, obtains local information through small kernel convolution, and then a set of parallel deep convolutions to capture contextual information across multiple scales, and then fuses feature information of multiple scales through element-by-element summation. The calculation expression of the PKI module is as follows:

[0120]

[0121]

[0122]

[0123] stk (m) =(m+1)×2+1

[0124] Among them, L represents the local feature, Z (m) represents the contextual features extracted by the mth depth convolution, conv ks×ks Indicates the convolution operation with a convolution kernel size of ks*ks, DWConv 3×3 represents a depthwise convolution operation with a convolution kernel size of 3*3, k (m) Represents the convolution kernel size, and m represents the depth convolution of size m;

[0125] Since the PKI module focuses on extracting multi-scale local context information, in order to capture remote context information, the CAA module is integrated into PKINet. The CAA module grasps the contextual interdependence between distant pixels, enhances the central features, and connects the two parts. The CAA module calculation expression is as follows:

[0126]

[0127]

[0128] Among them, Sigmoid(·) represents the S-shaped growth curve, Represents the enhanced features, DWConv kb×1 Indicates the depth convolution operation with a convolution kernel size of kb*1, DWConv 1×kbIndicates a deep convolution operation with a convolution kernel size of 1*kb. Represents the feature map of stage n with width w and height h;

[0129] Combining the C2f module with the PKINet module, an improved C2fPKI module is obtained, which can improve the feature extraction capability of the backbone network and solve the problems of large target scale changes and complex background in remote sensing image target detection; the expression of the C2fPKI module is as follows:

[0130]

[0131] Among them, C2fPKI represents the cross-stage feature fusion module based on the multi-core, PKINet represents the multi-core network module, and C2f represents the cross-stage feature fusion module; and the data is output to the Neck network through the C2fPKI module.

[0132] S3. Use the neck network of the optimized YOLOv8 model to dynamically upsample feature maps of different scales, generate upsampled feature maps, and use the neck network to output the upsampled feature maps. The specific steps are as follows:

[0133] S301, dynamically up-sample feature maps of different scales using the neck network of the optimized YOLOv8 model to obtain an original sampling network;

[0134] S302, using the first linear projection to process feature maps of different scales to generate an offset set, and using the S-shaped growth curve function and the static factor to reshape the offset set to obtain reshaped features, and using the second linear projection to process the reshaped features to obtain an offset, and summing the offset and the original sampling network to obtain a sampling set;

[0135] S303, using the network sampling function, re-sample the assumed bilinear interpolation according to the position in the sampling set to obtain a content-aware upsampling result, generate an upsampling feature map, and output the upsampling feature map using the neck network.

[0136] In this embodiment, since there are many targets and they are densely distributed, information is lost during feature fusion. Therefore, the upsampling method on the Neck network in YOLOv8 is improved, and the dynamic upsampling method is used to replace the original upsampling method to enhance the detection capability of dense targets. The expression of the dynamic upsampling is as follows:

[0137] X′=gridsample(X,S)

[0138] S=G+O

[0139] O=0.5sigmoid(linear1(X))·linear2(X)

[0140] Where X′ represents the upsampled feature map of size C×H2×W2, gridsample(·) represents the network sampling function, X represents the feature maps of different scales of size C×H1×W1, S represents the sampling set of size 2×H2×W2, 2 represents the coordinates of x and y, G represents the original sampling network, O represents the offset, 0.5 represents the static factor, sigmoid(·) represents the S-shaped growth curve, linear1(·) represents the first linear projection, and linear2(·) represents the second linear projection;

[0141] The input features are linearly projected to generate a point-by-point "dynamic range factor" to increase the flexibility of the offset; by using the sigmoid function and a static factor of 0.5, the dynamic range is between [0, 0.5], centered at 0.25, the same as the static range; s is generated using linear projection 2 offset set, and reshape the offset set to meet the spatial size, so that feature X is reshaped into And linearly project it to 2g×sH×sW to get the offset O, and sum the offset O and the original sampling network G to get the sampling set S.

[0142] S4. In response to the upsampled feature map output by the neck network, a dynamic detection head is added and the upsampled feature map is optimized using the attention mechanism, and the optimized upsampled feature map is subjected to feature fusion to obtain a fused feature map. The specific steps are as follows:

[0143] S401, in response to the upsampled feature map output by the neck network, adding a dynamic detection head, unifying scale perception, space perception and task perception, and obtaining a four-dimensional tensor of a rescaled feature pyramid;

[0144] S402, according to the attention mechanism, convolve the four-dimensional tensor using convolution approximation, and activate it using a hard S-curve function to reshape the four-dimensional tensor into a three-dimensional tensor to obtain scale-aware attention;

[0145] S403. Based on the attention mechanism, combined with the four-dimensional tensor, the deformable convolution is used to sparse the attention learning, and the features are integrated across scales to obtain spatial perception attention;

[0146] S404. According to the attention mechanism, global average pooling is performed on the four-dimensional tensor to reduce the dimension, a threshold of the activation function is controlled using a hyperparameter, and the reduced-dimensional tensor is processed using a fully connected layer and a normalization layer to obtain task-aware attention.

[0147] S405. Focus the scale-aware attention, space-aware attention, and task-aware attention on a unified dimension to obtain unified attention. Based on the upsampled feature map and the unified attention output by the neck network, a connection module is used to perform feature fusion to obtain a fused feature map.

[0148] In this embodiment, a dynamic detection head is added at the output of the Neck network to improve the detection head. By unifying scale perception, spatial perception and task perception, the attention mechanism is applied in different dimensions to improve the model's ability to handle large changes in target scale in remote sensing images. At the same time, the model is optimized for blurred targets, thereby improving the overall accuracy. The rescaled feature pyramid can be described as a four-dimensional tensor F∈R L×H×W×C , and define S = H × W to reshape the four-dimensional tensor into a three-dimensional tensor F∈R L×S×C , and calculate scale perception, spatial perception, and task perception;

[0149] Scale-aware attention fuses features of different scales based on their semantic importance. The calculation expression is as follows:

[0150]

[0151] Among them, π L (F) represents scale-aware attention, f(·) represents a linear function approximated by 1*1 convolution, and σ(·) represents a hard S-shaped growth curve to reshape the four-dimensional tensor;

[0152] Spatial perception attention focuses on the ability to distinguish different spatial positions. Considering the high-dimensional S, S is decoupled, and deformation convolution is used to sparse the attention learning, and then the features are integrated across scales. The calculation expression is as follows:

[0153]

[0154] Among them, π S (F) represents spatial perception attention, K represents the number of sparse sampling positions, l represents the lth number of dimension L, k represents the kth number of sparse sampling positions, and w l,k represents the weight of the corresponding position, represents the self-learning space offset, p k represents the spatial position, c represents the number of channels, Indicates p k The self-learning scalar at position, and They are all learned from the median-level input features of the four-dimensional tensor F;

[0155] Task-aware attention promotes the generalization of joint learning and target expression ability. It can dynamically switch feature channels to assist different tasks, use hyperparameters to control the activation threshold, perform global average pooling on the L dimension × S dimension to reduce the dimension, and use two fully connected layers and a normalization layer. Finally, a shifted S-type function is applied to normalize the output data to the interval [-1,1]. The calculation expression of the task-aware attention is as follows:

[0156] π C (F)·F=max(α 1 (F)·F c +β 1 ,α 2 (F)·F c +β 2 )

[0157] Among them, π C (F) represents task-aware attention, α 1 ,α 2 ,β 1 ,β 2 Both represent hyperparameters and are similar to dynamic linear rectification functions, F c A feature map representing task awareness;

[0158] Focusing scale-aware attention, space-aware attention, and task-aware attention on a unified dimension, unified attention is obtained; the expression of the unified attention is as follows:

[0159] W(F)=π C (π S (π L (F)·F)·F)·F

[0160] Among them, W(F) represents unified attention.

[0161] S5. Detect and classify the fused feature map to obtain the classified target detection result, and complete the dynamic detection of the remote sensing image target. The specific steps are as follows:

[0162] S501, combining the confidence and positioning accuracy of the fused feature map to obtain a joint representation, optimizing the joint representation using a zoom loss function, and obtaining a classified fused feature map;

[0163] S502, using the classification function and combining it with the generalized intersection-over-union loss function to detect the classified fusion feature map, obtain the classified target detection result, and complete the dynamic detection of the remote sensing image target.

[0164] In this embodiment, the classification loss function of the YOLOv8 model is replaced by a Varifocal Loss function, which optimizes the joint representation of the target confidence and positioning accuracy, has a better effect on dense targets in remote sensing images, and reduces the missed detection rate. The expression of the Varifocal Loss function is as follows:

[0165]

[0166] Among them, VFL(·) represents the zoom loss function, p represents the predicted IoU-perceptual classification score, q represents the IoU ratio between the candidate box and the real box, α represents the balance factor used to balance the ratio of positive and negative samples, and γ represents the degree of easy defocusing of the hyperparameter adjustment; for negative samples, that is, candidate boxes that are not responsible for the real box, their q value is 0;

[0167] The GIoU Loss function is used to consider the minimum closed area of ​​the predicted box and the real box, so that the model pays more attention to the difficult-to-detect targets, thereby improving the detection accuracy of the model. The expression of the GIoU Loss function is as follows:

[0168]

[0169] Among them, GIoU stands for the generalized intersection over union loss function, A stands for the predicted box, B stands for the true box, C stands for the minimum convex hull matrix containing the predicted box and the true box, and IoU stands for intersection over union; it is used to optimize the target detection algorithm, especially when dealing with object occlusion, size changes, etc., it can provide a more accurate matching evaluation. A high GIOU value means that the two boxes not only have a large overlap, but also have similar shapes. Through these two loss functions, the algorithm can be optimized to solve the problem of dense targets and difficult target detection in remote sensing images, reduce the target missed detection rate, and thus improve the detection accuracy of the model;

[0170] The target detection algorithm is used in combination with the generalized intersection-over-union loss function to detect the classified fusion feature map, obtain the classified target detection result, and complete the dynamic detection of remote sensing image targets.

[0171] Example 2

[0172] In this embodiment, in order to ensure the fairness of the experiment, all experiments of the present invention are carried out under the same experimental environment and training parameter configuration. The experimental environment is Windows 10 operating system, the CPU is Intel Core i7-13700K 3.40GHz, the graphics card is NVIDIA GeForce RTX 4080, 16GB video memory, and the training is carried out on a single GPU graphics card, the deep learning framework is PyTorch1.12.1, the compilation environment is CUDA11.3, python3.8.18; training parameters: number of iterations 500, batch size 16, number of threads 4, and the rest of the parameters keep the default values;

[0173] In order to evaluate the improvement effect of the present invention, the accuracy series index (AP) is used. The larger the value, the higher the detection accuracy. Among them, AP50 represents the average accuracy when the intersection-over-union ratio is 0.5, and AP75 represents the average accuracy when the intersection-over-union ratio is 0.75. Compared with the former, AP75 is more stringent on the detection results. The average precision mean is the most important indicator, which represents the average value of AP with 10 thresholds between 0.5 and 0.95, and the threshold interval is 0.05. APs of different sizes are divided into APs, APm and AP1, which respectively represent the detection results of small targets, medium targets and large targets. The definition of a small target is a pixel area <32 2 , the definition of a medium target is a pixel area of ​​32 2 <area<96 2 , the definition of a large target is a pixel area>96 2 .

[0174] In this embodiment, the present invention uses a general remote sensing image dataset HRRSD to verify the algorithm, and the HRRSD dataset has 13 categories, which are: airplane, baseball field, basketball court, bridge, intersection, track and field, port, parking lot, ship, storage tank, T-junction, tennis court and car. During the experiment, the HRRSD dataset is divided into a training set and a test set according to a ratio of 4:1, wherein the training set contains 17409 pictures and the test set contains 4352 pictures.

[0175] In this embodiment, a comparative experiment is conducted in the HRRSD data set, and the method of the present invention is compared with the target detection algorithm commonly used in recent years and the algorithm improved for remote sensing image detection; the comparative experiment of the HRRSD data set is shown in Table 1. It can be seen from the table that the best value of each data is obtained by the method of the present invention, indicating that the present invention is the best; the average accuracy of the method proposed in the present invention is improved by 5.9 percentage points compared with the original YOLOv8. Compared with the same type of improved algorithms, the experimental results show that the present invention is more significantly improved than the same type of remote sensing image detection algorithm and has higher detection accuracy.

[0176] Table 1

[0177]

[0178]

[0179] In this embodiment, the YOLOv8 detection method and the detection method of the present invention are used to detect the test set images. Figure 4 The comparison of the detection effects of the two detection methods is shown, where (a) is the original image, (b) is the detection result of the YOLOv8 algorithm, and (c) is the detection result of the method of the present invention. Targets of different categories are represented by prediction boxes of different colors, and the confidence scores are also shown;

[0180] It can be seen from the figure that in the first group of pictures, the method of the present invention is more accurate in identifying the aircraft target, with a confidence level of more than 90, while the confidence level of the aircraft target detected by YOLOv8 is only about 83; in the second group of pictures, because the pictures are more blurred, the basketball court in the upper left corner is missed during the YOLOv8 detection, while the method of the present invention can accurately identify the target; in the third group of pictures, YOLOv8 mistakenly detects an open space as a baseball field, while the method of the present invention can accurately identify other targets. In summary, the method of the present invention has a higher detection effect, and has higher confidence scores for various target categories, and is more accurate in identifying and locating remote sensing image targets of various sizes.

Claims

1. A remote sensing image target dynamic detection method based on poly-kernel, characterized in that: The following steps are involved: S1, collect remote sensing images, and optimize the YOLOv8 model to obtain an optimized YOLOv8 model; S2, use the optimized YOLOv8 model backbone network to extract information from remote sensing images, and obtain feature maps of different scales by downsampling; S3. Use the neck network of the optimized YOLOv8 model to dynamically upsample feature maps of different scales, generate upsampled feature maps, and use the neck network to output the upsampled feature maps; S4, in response to the upsampled feature map output by the neck network, adding a dynamic detection head and optimizing the upsampled feature map using an attention mechanism, and performing feature fusion on the optimized upsampled feature map to obtain a fused feature map; S5. Detect and classify the fused feature map to obtain the classified target detection result, and complete the dynamic detection of the remote sensing image target.

2. The remote sensing image target dynamic detection method based on poly-kernel according to claim 1, characterized in that: The optimized YOLOv8 model includes: The multi-kernel network module is used to improve the cross-stage feature fusion module in the backbone network of the YOLOv8 model; A cross-stage feature fusion module based on poly-kernel is used to extract information from remote sensing images and obtain feature maps of different scales through downsampling; The dynamic feature pyramid network is used to dynamically upsample feature maps of different scales using the neck network to generate upsampled feature maps, and the neck network is used to output the upsampled feature maps. The obtained feature maps are input into the dynamic detection head to optimize and fuse the feature maps.

3. The remote sensing image target dynamic detection method based on poly-kernel according to claim 2 is characterized in that: The loss function of the optimized YOLOv8 model includes: Zoom loss function: Where VFL(·) represents the zoom loss function, p represents the predicted IoU-perceptual classification score, q represents the IoU ratio between the candidate box and the true box, α represents the balance factor, and γ represents the degree of easy defocusing of the hyperparameter adjustment; Generalized intersection-over-union loss function: Among them, GIoU represents the generalized intersection-over-union loss function, A represents the predicted box, B represents the true box, C represents the minimum convex hull matrix containing the predicted box and the true box, and IoU represents the intersection-over-union ratio.

4. The remote sensing image target dynamic detection method based on poly-kernel according to claim 3 is characterized in that: The S2 comprises the following steps: S201, inputting the remote sensing image into the backbone network of the optimized YOLOv8 model, performing convolution processing using a convolution module, and obtaining convolution data; S202, using the cross-stage feature fusion submodule in the cross-stage feature fusion module of the multi-core, the convolution data is divided into two branches for processing, one branch outputs the convolution data, and the other branch outputs the convolution data after being processed by multiple bottleneck submodules, and the outputs of the branches are combined to obtain output data; S203, using the poly-kernel network module to process the output data, using the poly-kernel sub-module to capture local context information, and using the context-anchored attention sub-module to capture remote context information of the output data; S204, according to the captured local context information and remote context information, down-sampling is performed using the backbone network of the optimized YOLOv8 model to obtain feature maps of different scales; The expression of the convolution process is as follows: in, F represents a convolution operation with a kernel size of k*k, a step size of s, and a number of channels of c. c w×h represents a feature map with width w, height h, and number of channels c, SiLU(·) represents the activation function, and BN(·) represents batch normalization; Represents a two-dimensional convolution method with a convolution kernel size of k*k and a step size of s.

5. The remote sensing image target dynamic detection method based on poly-kernel according to claim 4, characterized in that: The expression of the cross-stage feature fusion module based on poly-kernel is as follows: C2fPKI=PKINet(C2f(F c w×h )) L=Conv ks×ks (DWConv 3×3 (F c w×h )) s.t.k (m) =(m+1)×2+1 Among them, C2fPKI represents the cross-stage feature fusion module based on multi-core, PKINet represents the multi-core network module, C2f represents the cross-stage feature fusion submodule, and F c w×h Represents a feature map with a width of w, a height of h, and a number of channels of c, conv 1×1 represents the convolution operation with a kernel size of 1*1, PKI represents the multi-scale local context information extracted by the poly-kernel submodule, CAA represents the feature information output by the upper and lower anchor attention submodule, represents primitive and concatenation operations, represents the element-by-element multiplication operation, L represents the local feature, and Z (m) represents the contextual features extracted by the mth depth convolution, conv ks×ks Indicates the convolution operation with a convolution kernel size of ks*ks, DWConv 3×3 represents a depthwise convolution operation with a convolution kernel size of 3*3, k (m) represents the convolution kernel size, m represents the depth convolution of size m, Sigmoid(·) represents the S-shaped growth curve, Represents the enhanced features, DWConv kb×1 Indicates the depth convolution operation with a convolution kernel size of kb*1, DWConv 1×kb Indicates a deep convolution operation with a convolution kernel size of 1*kb. Represents the feature map of the nth stage with width w and height h.

6. The remote sensing image target dynamic detection method based on poly-kernel according to claim 3, characterized in that: The S3 comprises the following steps: S301, dynamically up-sample feature maps of different scales using the neck network of the optimized YOLOv8 model to obtain an original sampling network; S302, using the first linear projection to process feature maps of different scales to generate an offset set, and using the S-shaped growth curve function and the static factor to reshape the offset set to obtain reshaped features, and using the second linear projection to process the reshaped features to obtain an offset, and summing the offset and the original sampling network to obtain a sampling set; S303, using the network sampling function, re-sample the assumed bilinear interpolation according to the position in the sampling set to obtain a content-aware upsampling result, generate an upsampling feature map, and output the upsampling feature map using the neck network.

7. The remote sensing image target dynamic detection method based on poly-kernel according to claim 6, characterized in that: The expression of the up-sampled feature map is as follows: X′=gridsample(X,S) S=G+O O=0.5sigmoid(linear1(X))·linear2(X) Among them, X′ represents the upsampled feature map, gridsample(·) represents the network sampling function, X represents the feature maps of different scales, S represents the sampling set, G represents the original sampling network, O represents the offset, 0.5 represents the static factor, sigmoid(·) represents the S-shaped growth curve, linear1(·) represents the first linear projection, and linear2(·) represents the second linear projection.

8. The remote sensing image target dynamic detection method based on poly-kernel according to claim 3 is characterized in that: The S4 comprises the following steps: S401, in response to the upsampled feature map output by the neck network, adding a dynamic detection head, unifying scale perception, space perception and task perception, and obtaining a four-dimensional tensor of a rescaled feature pyramid; S402, according to the attention mechanism, convolve the four-dimensional tensor using convolution approximation, and activate it using a hard S-curve function to reshape the four-dimensional tensor into a three-dimensional tensor to obtain scale-aware attention; S403. Based on the attention mechanism, combined with the four-dimensional tensor, the deformable convolution is used to sparse the attention learning, and the features are integrated across scales to obtain spatial perception attention; S404. According to the attention mechanism, global average pooling is performed on the four-dimensional tensor to reduce the dimension, a threshold of the activation function is controlled using a hyperparameter, and the reduced-dimensional tensor is processed using a fully connected layer and a normalization layer to obtain task-aware attention. S405. Focus the scale-aware attention, space-aware attention, and task-aware attention on a unified dimension to obtain unified attention. Based on the upsampled feature map and the unified attention output by the neck network, a connection module is used to perform feature fusion to obtain a fused feature map.

9. The remote sensing image target dynamic detection method based on poly-kernel according to claim 8, characterized in that: The expression of unified attention is as follows: W(F)=π C (p S (p L (F)·F)·F)·F p C (F)·F=max(α 1 (F)·F c +b 1 ,a 2 (F)·F c +b 2 ) Among them, W(F) represents unified attention, π L (F) represents scale-aware attention, π S (F) represents spatial perception attention, π C (F) represents task-aware attention, F represents a four-dimensional tensor, and the four dimensions include L, H, W, and C. σ(·) represents a hard S-shaped growth curve, f(·) represents a linear function approximated by convolution, S represents the product of dimension H and dimension W, K represents the number of sparse sampling positions, l represents the lth number of dimension L, k represents the kth number of sparse sampling positions, and w l,k represents the weight of the corresponding position, Δ pk represents the self-learning space offset, p k represents the spatial position, c represents the number of channels, Δ mk Indicates p k The self-learning scalar at position, α 1 ,α 2 ,β 1 ,β 2 Both represent hyperparameters and are similar to dynamic linear rectification functions, F c Representing task-aware feature maps.

10. The remote sensing image target dynamic detection method based on poly-kernel according to claim 3, characterized in that: The S5 comprises the following steps: S501, combining the confidence and positioning accuracy of the fused feature map to obtain a joint representation, optimizing the joint representation using a zoom loss function, and obtaining a classified fused feature map; S502, using the classification function and combining it with the generalized intersection-over-union loss function to detect the classified fusion feature map, obtain the classified target detection result, and complete the dynamic detection of the remote sensing image target.

Images