A fine-grained fog detection method for remote sensing images based on multi-layer collaborative network

Abstract

The present invention belongs to the technical field of remote sensing image intelligent processing, and in particular, relates to a fine-grained foggy remote sensing image detection method based on a multi-layer collaborative network, comprising: inputting a foggy remote sensing visible light image to be detected into a pre-established and trained detection model, outputting a ship detection result, thereby realizing end-to-end fine-grained target detection; the detection model comprises: a backbone network, a neck structure and a multi-detection head module; the backbone network, based on multi-layer collaboration, adopts a multi-purpose convolution kernel to compress the volume of the detection model while restoring the image degradation effect; the neck structure adopts a feature pyramid network and a pyramid attention network, and introduces an attention mechanism to cascade multi-scale feature maps, and pay attention to the fine-grained differences of the image to improve the accuracy of category recognition; the multi-detection head module is used to output feature maps of different resolutions to improve the perception ability of the target. The entire network directly realizes end-to-end fine-grained target detection under foggy conditions.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image intelligent processing, and in particular relates to a fine-grained foggy remote sensing image detection method based on a multi-layer collaborative network. Background Art

[0002] The rapid development of contemporary remote sensing technology has provided abundant high-resolution remote sensing images for daily life and military applications. In marine applications, accurate ship positioning and tracking are essential for maritime safety and rescue. In military applications, effective ship positioning can assist in intelligence collection and analysis. Ship target detection is the basis and prerequisite for realizing these applications, so it has attracted much attention in the field of remote sensing image processing. Compared with synthetic aperture radar (SAR) images, optical images have richer texture information and clearer structures, and can provide more detailed information of the observed scene, which makes the use of visible light remote sensing images for ship target detection an important research direction. Although optical remote sensing images face many challenges in practical applications, such as weather changes and wave interference, by using advanced image processing techniques, such as convolutional neural networks (CNN) and multi-scale feature extraction, as well as data enhancement and transfer learning methods, quite good detection performance can be achieved. Therefore, the field of optical remote sensing ship target detection has theoretical feasibility and practical application value, and its research will help to further improve the ability of remote sensing image processing and promote the development of ship positioning and tracking technology.

[0003] Object detection technology plays an important role in marine environment monitoring and control. However, in complex sea conditions, factors such as reduced image contrast lead to a decline in the performance of traditional object detection models. The data distribution of the detection model in training and testing is different, resulting in poor performance of the model in different data distributions. In marine environment monitoring, domain shift in the input image can cause a large amount of detail to be lost, thus affecting the performance of the object detection model. Existing object detection methods perform well in detecting ships under normal sea conditions, but fail to obtain satisfactory results under complex sea conditions.

[0004] There are currently two main methods for traditional ship detection under complex conditions. The first method is a ship detection method based on background modeling. This method is relatively simple, has strong real-time performance, and has a relatively high target detection rate. However, on the one hand, this method has a relatively poor detection effect on stationary targets, and on the other hand, there will be problems such as artifacts, shadow foregrounds, and incomplete moving targets. The second type of algorithm is a ship detection method based on manually extracted features. This method is highly interpretable and can make rapid detections under simple background conditions, but this method also has problems such as poor recognition effect, low accuracy, large amount of calculation, slow operation speed, and the possibility of multiple recognition results.

[0005] Although deep learning methods have been widely used in target detection, there has been no in-depth exploration of fast ship detection under complex conditions, and ship detection under complex conditions still faces major challenges. A simple idea is to split this task into two tasks: image enhancement preprocessing and target detection, and study these two tasks separately. For image enhancement tasks, traditional algorithms need to rely on researchers' strong prior knowledge and a series of assumptions and estimates of parameters in the degradation process, and perform poorly under certain extreme conditions. Deep learning-based methods for enhancement rely on a large number of paired images as data sources for training. In actual situations, these data are often difficult to obtain, and some restored images have obvious artifacts. The performance of sending the restored image to the detection network depends to a certain extent on the performance of the enhancement network, which increases uncertainty for the required detection results. In addition, the two-stage approach will also increase the complexity of the algorithm, resulting in reduced detection efficiency. For the above reasons, researchers introduced the concept of multi-task learning into deep learning, using a more complex network to learn common feature expressions, and then each task separately learns its own output on the common features. This type of method improves data usage efficiency, reduces overfitting through shared representations, and accelerates learning by using auxiliary tasks. However, negative transfer or destructive interference will inevitably occur, and the performance of different tasks will constrain each other. Therefore, how to design a high-performance complex condition ship detection network is still an urgent problem to be solved. Summary of the invention

[0006] In view of the technical defects of the existing ship target detection methods under complex conditions: ① Ship data sets are difficult to obtain under complex conditions, and real scene data are limited; ② Traditional detection networks perform poorly under complex conditions; ③ The existing ship detection network structure under complex conditions is complex, the amount of calculation is large, and the efficiency is low. The purpose of the present invention is to overcome the above-mentioned defects of the prior art and propose a fine-grained remote sensing image detection method based on a multi-layer collaborative network to achieve end-to-end detection of ship targets under complex conditions and reduce the complexity of the network while ensuring accuracy.

[0007] In order to achieve the above object, the present invention proposes a fine-grained foggy remote sensing image detection method based on a multi-layer collaborative network, comprising:

[0008] The remote sensing visible light image to be detected in foggy weather is input into the pre-established and trained detection model, and the ship detection result is output, thereby achieving end-to-end fine-grained target detection;

[0009] The detection model includes: a backbone network, a neck structure and a multi-detection head module; wherein,

[0010] The backbone network, based on multi-layer collaboration, uses multi-purpose convolution kernels to compress the volume of the detection model while restoring the degradation effect of the image;

[0011] The neck structure adopts a feature pyramid network and a pyramid attention network, and introduces an attention mechanism to cascade multi-scale feature maps and focus on fine-grained differences in images to improve category recognition accuracy;

[0012] The multi-detection head module is used to output feature maps with different resolutions to improve the perception capability of the target.

[0013] Preferably, the backbone network includes: multiplexed convolution, 3 stacked Block modules, a PCA attention module, 2 CBL modules and a linear rectification function. The processing process of the backbone network is: the input foggy remote sensing visible light image to be detected is sequentially subjected to multiplexed convolution and 3 Block modules to obtain feature maps P0, P1, P2 and P3. After splicing operation, attention module and 2 CBL modules, P1, P2 and P3 are added pixel by pixel with P0 after passing through the CBL module, and then the feature map is output after passing through the linear rectification function.

[0014] Preferably, the Block module includes: multiplexed convolution, linear rectification function, PCA attention module and residual connection.

[0015] Preferably, the CBL module comprises cascaded convolution, normalization operations and linear rectification functions.

[0016] Preferably, the processing of the feature pyramid network includes: starting from the top layer, gradually adding low-resolution features to bottom-layer features by upsampling, and fusing the low-resolution features of the bottom-layer feature map with the high semantic information of the top-layer feature map.

[0017] Preferably, the processing process of the pyramid attention network includes: transferring semantic information from low dimension to high dimension again through convolution operation, and fusing with the corresponding layer of the feature pyramid network to realize information interaction between different levels.

[0018] Preferably, the introduced attention mechanism includes an average pooling layer and four convolutional layers, wherein the first convolutional layer and the second convolutional layer are connected to output a linear rectification function feature map. The second convolutional layer and the third convolutional layer are connected to the sigmoid activation function to output the feature map Input and Connect the Multiply function to get the feature map The third convolutional layer and the fourth convolutional layer are connected to the linear rectification function output feature map After the fourth convolutional layer, the sigmoid activation function is connected to output the feature map and Connect the Multiply function between them.

[0019] Preferably, the multi-detection head module includes 4 detection heads, outputs 4 feature maps, and filters and organizes the merged detection results through non-maximum suppression technology.

[0020] Preferably, the method further comprises a step of training the detection model, comprising:

[0021] Construct a degraded image ship detection dataset based on physical models;

[0022] A point spread function is obtained according to the modulation transfer function, and a foggy image is obtained by simulating the point spread function to form a training set including light fog, medium fog and dense fog;

[0023] The training set is input into the detection model in sequence for training and parameter adjustment until the training requirements are met to obtain a trained detection model.

[0024] Preferably, the total loss function L loss for:

[0025] L loss =α k (α box L ciou +α obj L obj +α cls L cls )

[0026] Among them, α k is the weight coefficient of the output feature map, α box , α obj , α cls are the weights for balancing target positioning loss, target confidence loss, and target classification loss, respectively. ciou The target positioning loss is expressed as CIOU loss:

[0027]

[0028] Among them, b and b gt Represent the predicted box and the true box respectively, It represents the ratio of the square of the Euclidean distance between the center points of the two boxes to the square of the diagonal distance of the minimum enclosed area of ​​the two boxes. IOU is the overlap ratio of the predicted box and the real box. v is the normalized difference in the length and width ratio of the predicted box and the real box.

[0029] L obj The target confidence loss is expressed as binary cross entropy loss:

[0030] L obj (p 0 ,p iou )=-∑[p iou ·log p 0 +(1-p iou )·log(1-p 0 )]

[0031] Among them, p 0 Represents the prediction box confidence score, p iou Represents the CIOU value of the predicted box and the corresponding target box;

[0032] L cls The target classification loss is expressed as cross entropy loss:

[0033]

[0034] Among them, p i ∈{0,1} represents the true value of the i-th class, which is 1 if it belongs to the i-th class and 0 if it does not. Represents the probability of the current category prediction.

[0035] Compared with the prior art, the advantages of the present invention are:

[0036] 1. Use the atmospheric modulation transfer function to simulate and then construct the foggy ship data set by the point spread function;

[0037] 2. Design a multi-layer collaborative network as the target detection backbone network and introduce multiple detection heads;

[0038] 3. The backbone uses multiple convolution kernels to compress the model size and computational complexity without affecting accuracy.

[0039] 4. The entire network directly realizes end-to-end fine-grained object detection under foggy conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of the fine-grained foggy remote sensing image detection method based on a multi-layer collaborative network of the present invention;

[0041] Figure 2 A network structure diagram of the detection model constructed for the present invention;

[0042] Figure 3 A diagram of the backbone network structure based on multi-layer collaboration constructed for the present invention;

[0043] Figure 4 This is a comparison diagram after applying the degradation effect;

[0044] Figure 5 Structural diagram of the PCA attention module constructed for the present invention. DETAILED DESCRIPTION

[0045] This application proposes a fine-grained remote sensing image detection method based on a multi-layer collaborative network, which includes the following technical framework:

[0046] Step 1: Construction of degraded image dataset based on physical model

[0047] The original data is the ShipRSImageNet data of Level 2, which has 25 categories of ships, 17,573 ship instances, and 3,436 visible light images. After the data set is divided, the training set has 10,860 instances, the validation set has 3,103 instances, and the test set has 3,610 instances. The modulation transfer function (MTF) of the superimposed multiple scattering effect of fog droplets is used to achieve the image degradation effect, and a fine-grained ship target detection dataset of degraded images is constructed.

[0048] Step 2: Backbone network construction based on multi-layer collaboration modules

[0049] The backbone network first passes through an SCConv to extract shallow features and output a feature map of size 64*256*256. Then it passes through three consecutive Block modules to obtain p 1 ,p 2 ,p 3 The three feature maps are then concat together as the input of the PCA (Pixel and Channel Attention) module, and the output feature map is 192*256*256 in size. Then, after a series of convolution operations and feature map p 0 After the convolution operation, the corresponding elements are added, and finally a rectified linear unit (ReLU) is used to prevent gradient diffusion, and the output feature map is 128*64*64.

[0050] Step 3: Neck structure design based on attention mechanism

[0051] The neck structure based on the attention mechanism adopts PAN (Pyramid Attention Network) + FPN (Feature Pyramid Network) as a whole. This structure combines the advantages of the two feature pyramid methods. By cascading multi-scale feature maps, a feature pyramid suitable for target detection of different scales is constructed. At the same time, the PCA attention mechanism is introduced in the neck part, so that the network pays more attention to the features of each pixel, suppresses irrelevant features, and guides the update of feature maps.

[0052] Step 4: Multi-detection head module design

[0053] The feature map resolutions of the output layers of the four detection heads are 64*64, 32*32, 16*16, and 8*8 respectively. This multi-scale detection strategy helps improve the perception of targets of different sizes, allowing the model to better adapt to the detection needs of targets of different sizes. Each detection head can focus on target detection tasks within a specific scale range to adapt to targets of different scales. Multiple detection heads can separate the predictions of targets of different categories, reduce interference between categories, improve detection accuracy, improve the multi-scale perception capabilities of target detection, and category-specific prediction capabilities, and allow customization for specific task requirements. They provide the model with greater flexibility and accuracy, helping to solve target detection challenges in different scenarios and applications.

[0054] Step 5: Optimize the end-to-end fine-grained object detection network

[0055] The sum of target localization loss, target confidence loss and target classification loss is used as the loss function of the entire network. The comprehensive consideration of these three loss functions is the accuracy, confidence and classification ability of target detection. loss To optimize the network so that it can better fit the target location, target confidence and target category information on the training data.

[0056] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0057] Example

[0058] The embodiment of the present invention discloses a fine-grained foggy remote sensing image detection method based on a multi-layer collaborative network. Figure 1 , the method of the present invention comprises:

[0059] The remote sensing visible light image to be detected in foggy weather is input into the pre-established and trained detection model, and the ship detection results are output, thus achieving end-to-end fine-grained target detection.

[0060] The detection model adopts a fine-grained remote sensing ship target detection network based on a multi-layer collaborative network, which is mainly composed of a backbone network based on a multi-layer collaborative module, a neck structure based on an attention mechanism, and multiple detection heads. Figure 2 , where the backbone network is composed of SCConv, Block module and PCA attention module; the neck adopts FPN+PAN structure as a whole, and cascades multi-scale feature maps to construct a feature pyramid suitable for target detection of different scales. Through an efficient feature transfer mechanism, it effectively integrates shallow semantic information with deep semantic information, overcomes the problems of insufficient low-level feature extraction capabilities and insufficient utilization of high-level semantic information in traditional target detection algorithms, and introduces PCA attention module to make the network pay more attention to the features of each pixel, suppress irrelevant features, and guide the update of feature maps; finally, four detection heads are used to better handle targets of different sizes and aspect ratios, and can produce more accurate target bounding boxes and classification results, which can capture more detailed information of the target while maintaining high resolution, and use different receptive fields and scales on feature maps at different levels to improve the model's perception of the target. The input is a remote sensing visible light image degraded by the physical model, and the output is the ship detection result of the degraded remote sensing image;

[0061] The method of the present invention is based on the target detection network proposed above and is implemented according to the following steps:

[0062] Step 1: Construction of degraded image dataset based on physical model

[0063] The present invention constructs a degraded image ship detection dataset based on a physical model. The generation process of foggy images can be described by the atmospheric modulation transfer function. Using the atmospheric transfer functions of different foggy conditions, foggy images under corresponding conditions can be simulated. The general steps are divided into two steps: (1) obtaining a point spread function based on the modulation transfer function. (2) obtaining a foggy image by simulating the point spread function.

[0064] The modulation transfer function measures the amplitude response of the system to the sine wave input and can be used to analyze the image quality of the imaging link. Since general remote sensing systems are spatially invariant systems, they can be divided linearly. The total MTF of the system is obtained by multiplying the sub-unit MTFs. For foggy days, the multiple scattering of fog droplets plays a dominant role. To simplify the calculation, only the effect of fog droplets is considered, and the attenuation characteristics of air molecules, water vapor and aerosols are ignored. The MTF in a one-dimensional plane is as follows:

[0065]

[0066] in, l is the particle correlation distance, which is related to the concentration and diameter of the particles.

[0067] Because the point spread function is two-dimensional, the MTF in one-dimensional space is expanded to two dimensions. Using the central symmetry of MTF, the point spread function range is used as the diameter of the modulation transfer function, and the one-dimensional MTF is rotated around the center and expanded to two dimensions, and then an inverse Fourier transform is performed to obtain the point spread function (PSF). For the ShipRSImageNet dataset, we use the above algorithm to randomly generate foggy images under three conditions: light fog, medium fog, and dense fog.

[0068] Light mist: particle correlation distance l = 90 μm, visibility distance R m =1200m;

[0069] Medium fog: particle correlation distance l = 60 μm, visibility distance R m =700m;

[0070] Thick fog: particle correlation distance l = 25 μm, visibility distance R m =100m;

[0071] The basic idea of ​​the point spread function model is similar to that of the MTF model, and the two can be converted to each other through Fourier transform:

[0072] PSF(x,y)=IFFT[MTF(f x ,f y )]

[0073] In the frequency domain, the degraded image of the imaging system can be obtained by multiplying the original image and the MTF and then performing an inverse Fourier transform:

[0074] f out (x,y)=f in (x,y)*PSF(x,y)

[0075] PSF(x,y) is the point spread function of the system, f in (x,y) is the input image of the system. The example of the dataset is as follows Figure 4 There are 700 light fog images, 700 medium fog images, and 798 heavy fog images in the training set.

[0076] Step 2: Building a backbone network based on multi-layer collaboration

[0077] Build a backbone network based on multi-layer collaboration. For specific structure, refer to Figure 3, including: multiplexed convolution, 3 stacked Block modules, PCA attention module, 2 CBL modules and linear rectification function. The processing process of the backbone network is as follows: the input foggy remote sensing visible light image to be detected is sequentially subjected to multiplexed convolution and 3 Block modules to obtain feature maps P0, P1, P2 and P3. After the splicing operation, attention module and 2 CBL modules, P1, P2 and P3 are added pixel by pixel with P0 after the CBL module, and then the feature map is output after the linear rectification function.

[0078] Specifically, after the degraded image → SCConv → after three Block stacking, the feature maps P1, P2 and P3 are restored → Concat operation → PCA attention module → 2*CBL module → pixel-by-pixel addition operation → ReLU → output 128*64*64 feature map P4. Among them, SCConv is a multiplexed convolution, assuming that the size of the main convolution kernel and the traditional convolution kernel are equal, and they are represented as d×d. Performing the sampling process on the main convolution kernel will produce [d / 2] secondary convolution kernels, where [] means rounding up. Therefore, the dimension of the i-th secondary convolution kernel can be expressed as (d-2i+2)×(d-2i+2). Compared with the results of the main convolution kernel, the spatial dimensions of the feature maps generated by applying these secondary convolution kernels remain unchanged.

[0079] Assume that the number of secondary convolution kernels is s for the convolution kernel The output is The total space complexity of the traditional convolution kernel is O(d 2 сп), the computational complexity is The spatial complexity of SCConv convolution is O(d 2 cn / s), the computational complexity is In addition, following the idea of ​​convolution kernel reuse in the channel, multiple secondary convolution kernels are generated. More feature maps can be obtained without increasing the number of convolution kernels. SCConv convolution can be expressed as:

[0080]

[0081] Where g is the step size on the channel, f i represents the convolution kernel of the i-th sliding window, Indicates the number of channels of the secondary convolution kernel, c indicates the number of channels, and a convolution kernel is used n times to generate feature maps. Then, three blocks are used to further extract features. Assume that the input feature map After convolution, we get the feature map p 0 :

[0082]

[0083] in,

[0084] Block Structure Reference Figure 3 , including: multiplexed convolution, linear rectification function, PCA attention module and residual connection.

[0085] The input of each Block first passes through SCConv, followed by a ReLU layer to prevent gradient diffusion, and then two attention modules are cascaded and combined with residual connections to obtain the final refined feature map to prevent gradient disappearance. The Block structure can be expressed as:

[0086]

[0087] The input of each subsequent Block layer comes from the output of the previous Block:

[0088] p i =f block (p i-1 )i=1,2,3

[0089] The stacking of multiple blocks and the combination of their output feature maps enable us to better transmit features at different levels. The output after Concat is cascaded through two attention modules and then through two CBL convolutions. Each CBL block is cascaded by Conv 1*1 convolution, BatchNorm and ReLU. The obtained feature map is added to the result of P0 after CBL convolution, and finally a linear rectification function (ReLU) is passed to obtain the final feature map.

[0090] Step 3: Neck structure design based on attention mechanism

[0091] The feature pyramid network and pyramid attention network are used, and the attention mechanism is introduced to perform cascade processing on multi-scale feature maps and focus on the fine-grained differences in images to improve the accuracy of category recognition.

[0092] First, FPN starts from the top layer and uses upsampling to gradually add low-resolution features to the bottom-layer features, fusing the low-resolution features of the bottom-layer feature map with the high semantic information of the top-layer feature map.

[0093] Secondly, PAN transfers semantic information from low dimension to high dimension through convolution operation. The purpose of downsampling is to build a feature pyramid, increase the network's receptive field and perception ability, and integrate with the corresponding layer of FPN to achieve information interaction between different levels. The PCA attention mechanism is introduced in PAN to make the network pay more attention to the features of each pixel, suppress irrelevant features, and guide the update of feature maps.

[0094] The introduced attention mechanism includes an average pooling layer and four convolutional layers. The linear rectification function is connected between the first convolutional layer and the second convolutional layer to output the feature map F1c. The sigmoid activation function is connected between the second and third convolutional layers to output the feature map F2c. The Multiply function is connected between the input and F2c to obtain the feature map F1p. The linear rectification function is connected between the third and fourth convolutional layers to output the feature map F2p. The sigmoid activation function is connected after the fourth convolutional layer to output the feature map F3p. The Multiply function is connected between F1p and F3p.

[0095] Specifically, refer to Figure 5 , the structure of the PCA attention module is as follows: feature map F c As input → Averagepooling → Conv 1*1 convolution → ReLU → Conv 1*1 convolution → Sigmoid → Multiply → Conv 1*1 convolution → ReLU → Conv 1*1 convolution → Sigmoid → Multiply → and F c The channel attention layer strengthens the feature representation between multiple scales and can fully explore the correlation between channels. The pixel attention layer is used to capture the contextual information between adjacent pixels and can more accurately identify the target information when the difference between the background and the target is small.

[0096] Step 4: Multi-detection head module design

[0097] The detection head uses predefined anchor boxes to represent the approximate shape of the object in the data sample. These anchor boxes are composed of different widths and heights. They use the pixel size of the original image and are used to predict the position and size of the object. The detection head in this invention uses four groups of anchor boxes of different sizes. Each group of anchor boxes of different sizes will produce channels of different sizes in the feature map. There are a total of twelve different anchor boxes, namely:

[0098] [10,12,28,17,22,34],[56,53,29,130,140,37],[63,144,125,99,28,81],[105,192,207,175,311,254]. Finally, four feature maps of size 32*39168*30, 32*9792*30, 32*2448*30, and 32*612*30 are obtained. The output is concat-operated to merge the prediction results of different detection heads. Then the non-maximum suppression (NMS) technique is used to filter and sort the merged detection results. The IOU threshold in the training process is 0.6, which is used to control the overlap between bounding boxes. When the intersection-over-union (IOU) of two bounding boxes is higher than the threshold, the bounding box with lower confidence is removed to eliminate overlapping detection results. The confidence threshold is 0.001, which is used to filter out bounding boxes with lower confidence scores to reduce the possibility of false detection.

[0099] Step 5: Optimize the end-to-end fine-grained object detection network

[0100] The loss function of this method consists of target positioning loss, target confidence loss and target classification loss. The target positioning loss is expressed as CIOU loss:

[0101]

[0102] Among them, b and b gt Represent the predicted box and the true box respectively, It represents the ratio of the square of the Euclidean distance between the center points of the two boxes to the square of the diagonal distance of the minimum closure area of ​​the two boxes. IOU is the overlap ratio of the predicted box and the real box. v is the normalized difference in the length and width ratio of the predicted box and the real box:

[0103]

[0104] Target confidence loss L obj Expressed as binary cross entropy loss:

[0105] L obj (p 0 ,p iou )=-∑[p iou ·logp 0 +(1-p iou )·log(1-p 0 )]

[0106] p 0 Represents the prediction box confidence score, p iou Represents the CIOU value of the predicted box and the corresponding target box (not completely 0, 1).

[0107] The target classification loss is expressed as cross entropy loss:

[0108]

[0109] p i ∈{0,1} represents the true value of the i-th class, which is 1 if it belongs to the i-th class and 0 if it does not. Represents the probability of the current category prediction. The total loss function can be expressed as:

[0110] L loss =α k (α box L ciou +α obj L obj +α cls L cls )

[0111] α k is the weight coefficient of the output feature map, α box , α obj , α cls The weights used to balance the loss functions are set to 0.05, 1.0, and 0.5 in the experiment. k The weight used to balance the output feature maps of each scale is set to 1 in the experiment.

[0112] To evaluate the generalization ability of the proposed model, we conducted experiments under Windows 10 system, with PyTorch 2.0 as the software platform and NVIDIA GeForce RTX 3090 as the hardware platform. To verify the effectiveness of the present invention, the proposed model was compared with several benchmark detectors (Faster R-CNN, SSD, FSAF, RepPoints, YOLOF, YOLOv5, YOLOv7, and YOLOv8). All experiments were conducted under the same training settings. The batch size was set to 10 and the number of training rounds was set to 800.

[0113] Comparing the results in Table 1, the proposed algorithm has achieved significant improvements over YOLOv5s and RepPoints, with a mAP that is 2% higher (46.7%) than YOLOv5s and 2.5% higher than RepPoints' 46.2%. In addition, the proposed algorithm also has significant improvements in mAP compared to YOLOF and SSD512, increasing by 3.9% and 6.6%, respectively. These improvements highlight the efficacy of the proposed algorithm in target detection tasks, especially in achieving higher accuracy than the above algorithms. In terms of processing speed, the proposed algorithm achieves 33.3 FPS (frames Per Second), which is fast enough to meet the needs of real-time image or video stream processing and generate target detection results in a timely manner.

[0114] In addition, an ablation study was conducted to investigate the optimal performance of the Block module model and the role of SCConv in these modules. The experimental results are shown in Table 2. When the number of Block modules increases from two to three, the relative improvement reaches 1.25%. In addition, in this experiment, the number of modules is considered as an independent variable, while the accuracy of the model is the dependent variable. By analyzing the response of the dependent variable to the independent variable, it can be concluded that adding this module has a positive impact on the model performance. In addition, the computation and performance of the entire model in this module are compared with those of ordinary convolution kernels and SCConv, where GFLOPs refers to the amount of computation when training the model with the same input size. The experiments show that the proposed model design can achieve the same performance as ordinary convolution blocks with reduced computation.

[0115] Table 1 Performance index experimental results of the implementation method of the present invention compared with the existing target detection method

[0116]

[0117] Table 2 Experimental results of different numbers of Block modules of the present invention and performance indicators compared with the original Conv

[0118]

[0119] The beneficial effect of the present invention is that the physical mechanism is introduced into the data set construction, so that the data set can better fit the real scene. The backbone network is improved, and the degradation effect of the image can be restored through multi-layer collaborative modules to directly realize end-to-end detection while meeting the real-time requirements of detection; the PCA attention mechanism is introduced in the neck, so that the model pays attention to the fine-grained differences of the image, improves the accuracy of category recognition, and uses four detection heads to achieve more flexible and accurate detection and positioning on different scale features, completing the effective modeling of fine-grained ship recognition under foggy conditions.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention is described in detail with reference to the embodiments, it should be understood by those skilled in the art that any modification or equivalent replacement of the technical solutions of the present invention does not depart from the spirit and scope of the technical solutions of the present invention and should be included in the scope of the claims of the present invention.

Claims

1. A fine-grained foggy remote sensing image detection method based on a multi-layer collaborative network, comprising: The remote sensing visible light image to be detected in foggy weather is input into the pre-established and trained detection model, and the ship detection result is output, thereby achieving end-to-end fine-grained target detection; The detection model includes: a backbone network, a neck structure and a multi-detection head module; wherein, The backbone network, based on multi-layer collaboration, uses multi-purpose convolution kernels to compress the volume of the detection model while restoring the degradation effect of the image; The neck structure adopts a feature pyramid network and a pyramid attention network, and introduces an attention mechanism to cascade multi-scale feature maps and focus on fine-grained differences in images to improve category recognition accuracy; The multi-detection head module is used to output feature maps of different resolutions to improve the perception ability of the target; The backbone network includes: multiplexed convolution, 3 stacked Block modules, PCA attention module, 2 CBL modules and linear rectification function. The processing process of the backbone network is: the input foggy remote sensing visible light image to be detected is sequentially subjected to multiplexed convolution and 3 Block modules to obtain feature maps P0, P1, P2 and P3. After the splicing operation, attention module and 2 CBL modules, P1, P2 and P3 are added pixel by pixel with P0 after passing through the CBL module, and then the feature map is output after the linear rectification function. The Block module includes: multiplexed convolution, linear rectification function, PCA attention module and residual connection; The CBL module includes cascaded convolution, normalization operations and linear rectification functions.

2. The fine-grained foggy remote sensing image detection method based on a multi-layer collaborative network according to claim 1 is characterized in that: The processing of the feature pyramid network includes: starting from the top layer, using upsampling to gradually add low-resolution features to bottom-layer features, and fusing the low-resolution features of the bottom-layer feature map with the high semantic information of the top-layer feature map.

3. The fine-grained foggy remote sensing image detection method based on a multi-layer collaborative network according to claim 2 is characterized in that: The processing process of the pyramid attention network includes: through convolution operation, transferring semantic information from low dimension to high dimension again, and fusing with the corresponding layer of the feature pyramid network to realize information interaction between different levels.

4. The fine-grained foggy remote sensing image detection method based on a multi-layer collaborative network according to claim 3 is characterized in that: The attention mechanism introduced includes an average pooling layer and four convolutional layers, wherein the first convolutional layer and the second convolutional layer are connected to output a linear rectification function feature map. The second convolutional layer and the third convolutional layer are connected to the sigmoid activation function to output the feature map Input and Connect the Multiply function to get the feature map The third convolutional layer and the fourth convolutional layer are connected to the linear rectification function output feature map After the fourth convolutional layer, the sigmoid activation function is connected to output the feature map and Connect the Multiply function between them.

5. The fine-grained foggy remote sensing image detection method based on a multi-layer collaborative network according to claim 1 is characterized in that: The multi-detection head module includes 4 detection heads, outputs 4 feature maps, and filters and organizes the merged detection results through non-maximum suppression technology.

6. The fine-grained foggy remote sensing image detection method based on a multi-layer collaborative network according to claim 1 is characterized in that: The method also includes a step of training a detection model, including: Construct a degraded image ship detection dataset based on physical models; A point spread function is obtained according to the modulation transfer function, and a foggy image is obtained by simulating the point spread function to form a training set including light fog, medium fog and dense fog; The training set is input into the detection model in sequence for training and parameter adjustment until the training requirements are met to obtain a trained detection model.

7. The fine-grained foggy remote sensing image detection method based on a multi-layer collaborative network according to claim 6 is characterized in that: The total loss function L loss for: L loss =a k (a box L ciou +a obj L obj +a cls L cls ) Among them, α k is the weight coefficient of the output feature map, α box , α obj , α cls are the weights for balancing target positioning loss, target confidence loss, and target classification loss, respectively. ciou The target positioning loss is expressed as CIOU loss: Among them, b and b gt Represent the predicted box and the true box respectively, It represents the ratio of the square of the Euclidean distance between the center points of the two boxes to the square of the diagonal distance of the minimum enclosed area of ​​the two boxes. IOU is the overlap ratio of the predicted box and the real box. v is the normalized difference in the length and width ratio of the predicted box and the real box. L obj The target confidence loss is expressed as binary cross entropy loss: L obj (p0,p iou )=-∑[p iou ·logp0+(1-p iou )·log(1-p0)] Among them, p0 represents the confidence score of the prediction box, p iou Represents the CIOU value of the predicted box and the corresponding target box; L cls The target classification loss is expressed as cross entropy loss: Among them, p i ∈{0,1} represents the true value of the i-th class, which is 1 if it belongs to the i-th class and 0 if it does not. Represents the probability of the current category prediction.

Images