SAR and optical image adaptive adversarial feature fusion target detection method

Abstract

The invention discloses an SAR and optical image adaptive adversarial feature fusion target detection method, and relates to the field of deep learning image fusion and the field of target detection. Comprising the following steps: performing speckle filtering processing on an SAR image by adopting a Refine Lee filter to obtain a filtered SAR image; performing image registration processing on the filtered SAR image and the filtered optical image to obtain a registered SAR image and a registered optical image; performing adaptive feature fusion on the registered SAR image and optical image by using a channel and space attention mechanism and a double-discriminator generative adversarial network to obtain a target fusion image; and performing target detection on the target fusion image by using a position attention mechanism and a YOLOv8 algorithm to obtain a target detection result. The problem that target detection of a neural network in a complex scene SAR image or an optical image is difficult to achieve ideal detection performance is solved.

Description

Technical Field

[0001] The present application relates to the fields of deep learning image fusion and target detection, and in particular to a SAR and optical image adaptive adversarial feature fusion target detection method. Background Art

[0002] Synthetic aperture radar (SAR) and optical images are two commonly used imaging methods in the field of remote sensing, each with its own unique advantages. SAR images use microwave imaging technology to observe the surface of the earth in all weather conditions and at all times of the day, and have the ability to penetrate natural interference such as clouds and haze. Therefore, SAR images can provide stable observation data under complex weather and environmental conditions. However, due to the imaging mechanism of SAR images, their images are often affected by coherent speckle noise, resulting in complex image textures and difficult to be used directly for target detection. In contrast, optical images can clearly capture the color, texture and shape information of objects through visible light band imaging, with high resolution and intuitive visual effects, so they are widely used in target recognition. However, the imaging effect of optical images is easily affected by factors such as weather conditions and changes in lighting, and there are certain limitations.

[0003] In recent years, how to combine the advantages of SAR images and optical images has become a research hotspot in the field of remote sensing image processing. By fusing the penetration of SAR images with the rich visual information of optical images, more accurate target detection can be achieved in complex scenes. However, how to effectively fuse these two different modal images remains a challenging topic. Currently, generative adversarial networks (GANs) are gradually being introduced into the field of remote sensing image processing as an effective feature extraction and fusion tool. The accuracy of target detection can be significantly improved by adaptively fusing features of SAR images and optical images. However, when dealing with complex scenes, traditional fusion methods tend to ignore important local information in the image, resulting in a decrease in detection accuracy. Summary of the invention

[0004] The purpose of this application is to provide a SAR and optical image adaptive adversarial feature fusion target detection method to solve the problem that neural networks are difficult to achieve ideal detection performance in target detection in complex scene SAR images or optical images.

[0005] To achieve the above objectives, this application adopts the following technical solutions: On the one hand, the present application provides a SAR and optical image adaptive adversarial feature fusion target detection method, comprising: The Refine Lee filter is used to perform speckle filtering on the SAR image to obtain the filtered SAR image; Performing image registration processing on the filtered SAR image and the optical image to obtain a registered SAR image and optical image; Adaptively fuse the registered SAR image and the optical image using a channel and space attention mechanism and a dual discriminator generative adversarial network to obtain a target fused image; The position attention mechanism and the YOLOv8 algorithm are used to perform target detection on the target fusion image to obtain a target detection result.

[0006] On the other hand, the present application also provides a SAR and optical image adaptive adversarial feature fusion target detection device, comprising: A filtering module is used to perform speckle filtering on the SAR image using a Refine Lee filter to obtain a filtered SAR image; A registration module, used for performing image registration processing on the filtered SAR image and the optical image to obtain a registered SAR image and optical image; A fusion module is used to perform adaptive feature fusion on the registered SAR image and the optical image by using a channel and spatial attention mechanism and a dual discriminator generative adversarial network to obtain a target fused image; The detection module is used to perform target detection on the target fusion image by using the position attention mechanism and the YOLOv8 algorithm to obtain a target detection result.

[0007] On the other hand, the present application also provides an electronic device: A processor; and a memory arranged to store computer executable instructions, which when executed cause the processor to perform the steps of the method as claimed in any one of the preceding claims.

[0008] Based on the above technical solution, this application can achieve the following technical effects: By using the Refine-Lee filter to filter the SAR image for speckle filtering, the noise in the SAR image is effectively reduced, the image quality is improved, and a better data basis is provided for subsequent feature fusion and target detection. The dual discriminator generative adversarial network (DDcGAN) for multi-resolution image fusion is used to adaptively fuse the features of SAR and optical images. The channel and spatial attention module (CBAM) is introduced in the fusion process; more weights are given to the target to be detected, so that the fused image can more accurately reflect the characteristic information of the target; the YOLOv8 network is used for target detection on the generated multimodal fusion image, and the position information is introduced in combination with the coordinate attention mechanism (CA), which further improves the accuracy of target detection; by dynamically combining the global attention mechanism and the Ghost module, a new lightweight detection head is constructed, which greatly improves the efficiency of SAR target detection and adapts to the requirements of computing resources in complex scenes. This method makes full use of the all-weather and strong penetrating characteristics of SAR images and the color and texture information of optical images. It can achieve high-precision remote sensing image target detection under different weather conditions and environments, and has good robustness and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a flowchart of a SAR and optical image adaptive adversarial feature fusion target detection method provided by an embodiment of the present application; Figure 2 It is a schematic diagram of a target detection process of SAR and optical image adaptive adversarial feature fusion provided by an embodiment of the present application; Figure 3 Schematic diagram of the overall network framework of DDcGAN provided in one embodiment of the present application; Figure 4 is a schematic diagram of a generator of a DDcGAN provided in an embodiment of the present application; Figure 5 It is a schematic diagram of a network block diagram of a CBAM attention mechanism provided in an embodiment of the present application; Figure 6 is a schematic diagram of a discriminator of a DDcGAN provided in an embodiment of the present application; Figure 7 This is a YOLOv8 network block diagram provided by an embodiment of the present application; Figure 8 is a network block diagram of a CA attention mechanism provided by an embodiment of the present application; Fig. 9 is a network block diagram of a detection layer provided in an embodiment of the present application; Fig.10 is a network block diagram of a Swin Transformer provided in an embodiment of the present application; Fig.11 is a network block diagram of a Ghost module provided in an embodiment of the present application; Fig.12 is a SAR original image provided by an embodiment of the present application; Fig.13 is an optical original image provided by an embodiment of the present application; Fig.14 is the fusion result provided by an embodiment of the present application; Fig.15 This is a target detection result provided by an embodiment of the present application. DETAILED DESCRIPTION

[0010] The present application is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present application will become clearer according to the following description and claims. It should be noted that the drawings are all in a very simplified form and are not in precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present application.

[0011] It should be noted that, in order to clearly explain the content of the present application, the present application specifically cites multiple embodiments to further illustrate different implementation methods of the present application, wherein the multiple embodiments are enumerated rather than exhaustive. In addition, for the sake of brevity of explanation, the contents mentioned in the previous embodiments are often omitted in the subsequent embodiments. Therefore, the contents not mentioned in the subsequent embodiments can refer to the previous embodiments accordingly. Example 1

[0012] Please refer to Figure 1 , Figure 1 The figure is a flow chart of a SAR and optical image adaptive adversarial feature fusion target detection method provided by this embodiment. The method specifically comprises the following steps: Step 102: Use the Refine Lee filter to perform speckle filtering on the SAR image to obtain a filtered SAR image.

[0013] It should be noted that one implementation of step 102 may be: Decompose the SAR image window into 9 sub-panes of 3×3, and calculate the mean value in each sub-pane; Four edge gradient operators are used for detection. Each edge gradient operator corresponds to two edge direction windows. The edge direction window whose mean is closer to the mean of the central pixel among the two edge direction windows corresponding to the operator with the largest gradient is selected and recorded as the calibration edge direction window. In the calibrated edge direction window, the Refine Lee filter is used to filter and obtain the filtered SAR image.

[0014] Based on this, the Refine-Lee filter is used to perform speckle filtering on SAR images, which effectively reduces the noise in SAR images, improves the image quality, and provides a better data basis for subsequent feature fusion and target detection.

[0015] Step 104: performing image registration processing on the filtered SAR image and the optical image to obtain a registered SAR image and optical image.

[0016] It should be noted that one implementation of step 104 may be: Use OpenCV to resize the filtered SAR image and optical image to the same size; The Oriented FAST and Rotated BRIEF (ORB) algorithm is used to perform image registration on the filtered SAR image and the optical image to obtain the registered SAR image and optical image.

[0017] Based on this, the all-weather and strong penetrating characteristics of SAR images and the color and texture information of optical images are fully utilized, and high-precision remote sensing image target detection can be achieved under different weather conditions and environments. It has good robustness and broad application prospects.

[0018] Step 106: Use a channel and space attention mechanism and a dual discriminator generative adversarial network (DDcGAN) to perform adaptive feature fusion on the registered SAR image and the optical image to obtain a target fused image.

[0019] It should be noted that one implementation of step 106 may be: Input the registered SAR image and the optical image into the generator to generate an initial fused image; Inputting the initial fused image and the registered SAR image into one of the discriminators, inputting the initial fused image and the registered optical image into another discriminator, using the registered SAR image and the optical image as true value images to discriminate the initial fused image, and obtaining updated generator parameters; Based on the updated generator parameters, a target fused image is generated, the specific steps comprising: The registered SAR image and optical image are input into the generator with updated parameters, and the Densenet network structure is used to extract features to obtain initial features. Based on the initial features, CBAM is used to jointly focus on the spatial information and channel information of the feature map to extract enhanced features; Perform feature fusion based on enhanced features to obtain a fused feature map; The fused feature map is subjected to feature reconstruction to obtain a target fused image.

[0020] Specifically, DDcGAN updates the parameters of the generator through adversarial training of two pairs of discriminators and generators, so that the fused image obtains more detailed information of the source image, so that the discriminator cannot distinguish between the two, and then generates a fused image through the generator. In the fusion process, CBAM is introduced to jointly focus on the spatial information and channel information of the feature map, so that the network gives more fusion weights to the target to be detected during fusion, thereby obtaining a fusion result that is more conducive to target detection.

[0021] Based on this, DDcGAN is used to adaptively fuse the features of SAR and optical images, which can achieve high-quality image fusion effects and perform well in detail preservation and resolution improvement of multimodal fusion images. CBAM is introduced in the fusion process to give more weight to the target to be measured, so that the fused image can more accurately reflect the characteristic information of the target.

[0022] Step 108: Use the position attention mechanism and the YOLOv8 algorithm to perform target detection on the target fusion image to obtain a target detection result.

[0023] It should be noted that one implementation of step 108 may be: The YOLOv8 algorithm includes a backbone network module, a feature fusion module, and a detection head module; Extracting first feature information from the target fused image through the backbone network module; Using a position attention mechanism to introduce the position information of the target into the feature fusion module, fusing the first feature information through the feature fusion module to obtain feature fusion information; Based on the feature fusion information, the detection head module performs classification prediction and position regression prediction respectively to obtain the target detection result. The specific steps include: Based on the feature fusion information, the Swin Transformer model is used to perform global feature association and extract global feature information; The global feature information is divided into two branches according to the classification task and the position regression task. The classification prediction and position regression prediction are performed by dynamically adjusting the Ghost network and the convolutional network. The specific steps include: Each branch is predicted in turn through the Ghost network, 3×3 convolutional network and 1×1 convolutional network to obtain classification prediction and position regression prediction results; The classification prediction and position regression prediction results are concatenated to obtain the target detection results.

[0024] Specifically, the fused image is passed through the backbone network of YOLOv8 to extract its preliminary feature information, and then the feature information is further refined and extracted through the feature fusion network of YOLOv8. Finally, the decoupling head of the detection network predicts the category and target position respectively, avoiding the loss of accuracy caused by the conflict between the two. In order to enable the network to accurately detect the target position in the fused image of complex scenes, CA is integrated into YOLOv8 to inject the corresponding position information into the target, thereby improving the network's detection performance of the target. At the YOLOv8 decoupling head, the Swin Transformer module is used to perform global feature association on the decision features to enhance the network's extraction of deep semantic information of the target. After global feature extraction, the Ghost network and convolution module are dynamically combined to make feature extraction more efficient and reduce the risk of model overfitting.

[0025] In summary, compared with the prior art, the proposed adaptive adversarial feature fusion target detection method for SAR images and optical images in complex scenes has the following beneficial effects: by using the Refine-Lee filter to filter the SAR image for coherent speckle, the noise in the SAR image is effectively reduced, the image quality is improved, and a better data basis is provided for subsequent feature fusion and target detection; using DDcGAN, the features of SAR and optical images are adaptively fused. CBAM is introduced in the fusion process to give more weight to the target to be detected, so that the fused image can more accurately reflect the characteristic information of the target; YOLOv8 network is used for target detection on the generated multimodal fusion image, and position information is introduced in combination with CA to further improve the accuracy of target detection; by dynamically combining the global attention mechanism and the Ghost module, a new lightweight detection head is constructed, which greatly improves the efficiency of SAR target detection and adapts to the requirements for computing resources in complex scenes; this method makes full use of the all-weather and strong penetration characteristics of SAR images and the color and texture information of optical images, and can achieve high-precision remote sensing image target detection under different weather conditions and environments, with good robustness and broad application prospects. Example 2

[0026] See also Figure 2 , this embodiment provides another SAR and optical image adaptive adversarial feature fusion target detection method.

[0027] Step 1: At the input stage, the Refine Lee filter is used to filter the coherent speckle noise of the SAR image. The filtering process is as follows: decompose the filter window into 9 sub-panes of 3×3, calculate the mean in each sub-pane, and then use 4 edge gradient operators for detection. Each edge gradient operator corresponds to 2 edge direction windows, and select the one with the largest gradient in the two edge direction windows whose mean is close to the central pixel mean. Then, the Lee filter is used for filtering in the edge direction window. After that, the size of the SAR image and the optical image are adjusted to be consistent, and the ORB algorithm is further used to align the two images. The process of aligning the feature space of the image is as follows: use OpenCV to adjust the optical image and the SAR image to the same size, and use ORB to match the feature points of the two images to align the images.

[0028] Step 2: In the fusion stage, this method uses DDcGAN to perform information complementation fusion on multi-source images. The fusion network mainly consists of a generator G and two discriminators D i , D v The overall network framework is as follows: Figure 3 As shown in Figure 2, DDcGAN inputs the infrared image i and the visible light image v into the generator G to generate the fused image f; the infrared image i and the fused image f are input into the discriminator D i , input the visible light image v and the fused image f into the discriminator D v ; Use i and v as the true value images to discriminate the fused image f, and update the parameters of the generator through adversarial training between the discriminator and the generator, so that the fused image f can obtain more information from the source image, so that the discriminator cannot distinguish between the two. Among them, the generator network block diagram is as follows Figure 4 As shown in the figure. The generator G adopts a codec structure with strong feature reconstruction capability in unsupervised learning. After the infrared and visible light images are connected by channels, a dual-channel image is formed and input into the encoder for feature extraction. The extracted features are input into the convolutional attention mechanism module to find key features, and then feature fusion and feature reconstruction are performed. The convolution layer of the feature extraction module uses the Densenet structure instead of the conventional convolution layer connection method, where n is the number of convolution layer output channels, the convolution layer step size is 1, and 3×3 convolution kernels are used to obtain 48 feature maps.

[0029] Furthermore, in order to obtain a fusion result that is more suitable for target detection, this method introduces more focused target information of CBAM in the fusion process and gives more fusion weight to the target to be detected. Figure 5 As shown in Figure 1, CBAM uses the convolutional attention mechanism to give more weight to the significant pixels in the input feature map, thereby outputting a feature enhancement map that can better express the original image. The formula of CBAM is as follows:

[0030]

[0031]

[0032]

[0033] in, Represents input; is the intermediate feature map generated by F through the channel attention mechanism, for The intermediate feature map generated by the spatial attention mechanism; is a fully connected layer; is the maximum pooling layer; is the average pooling layer; and Represent channel and spatial attention mechanisms respectively; represents the activation function, Represents convolution.

[0034] The feature map output by the encoder enters the feature fusion module for image fusion and then enters the decoder. Figure 6 As shown in the figure, the size of the convolutional layers in the decoder is 3×3, the activation function of the last convolutional layer is the tanh function, and the fused feature map is reconstructed in the decoder. Finally, the number of channels is reduced to 1, and the fused image is output. In the entire encoder-decoder network, each convolutional layer except the last convolutional layer is followed by a batch normalization layer and a rectification function (LeakyReLU) layer to improve the stability of the model during training.

[0035] Step 3: In the target detection stage, the YOLOv8 algorithm is used for detection. Figure 7 As shown in the figure, the YOLOv8 algorithm is mainly composed of three parts: backbone network module, feature fusion module and detection head module.

[0036] The main feature extraction module of YOLOv8 still uses the CSP-DarkNet structure, and replaces the C3 module with the C2f module with richer gradient flow. The corresponding number of channels of the multi-scale model is adjusted, which reduces the model calculation amount and accelerates the convergence of the network. The feature fusion module adopts the Path Aggregation Network (PANet) structure. Through upsampling, downsampling and channel feature information fusion, the multi-scale information extracted by the network is connected up and down, so that the shallow features have deep semantic information, and the deep features have shallow spatial detail information, which effectively enhances the network's detection performance for multi-scale targets. Finally, the three output branches of PANet are sent to the detection module. In order to better detect targets in complex scenes, this method introduces the CA attention mechanism. Figure 8 As shown in Figure 2, CA captures long-range dependencies in one direction while preserving precise position information in the other direction. The generated feature maps are encoded into a pair of direction-aware and position-sensitive attention maps, respectively. These attention maps can be applied to the input feature maps to enhance the representation of the attention target. The formula is as follows:

[0037]

[0038]

[0039]

[0040]

[0041]

[0042] in, represents the input feature F, Indicates height, represents the height variable, Indicates width, Represents the width variable, Represents the output of the average pooling layer at a fixed height, Represents the output of the average pooling layer when the width is fixed, F 1 represents concatenation and convolution operations, and Represent BN and activation function respectively, represents a highly branched convolution, represents the input features at the height branch, represents the convolution in the width branch, represents the input feature in the width branch, and Respectively represent the focusing results of the elements in the height branch and the width branch, Represents the output of CA.

[0043] In order to further improve the network's detection performance for dense small targets in complex remote sensing scenes, this method improves the detection head of YOLOv8. The new detection head network structure is as follows: Fig. 9 The input features first pass through the Swin Transformer module to achieve long-term feature extraction by introducing a moving window mechanism, thereby enhancing the network's global perception of feature maps. The network structure of Swin Transformer is shown in Fig.10 As shown. Then, the decoupling head network is combined to separate the classification task from the position regression task. The feature information is classified and regressed independently along the two branches, avoiding the accuracy loss caused by the conflict between the two tasks. The features of each branch are sent to the Ghost module to further extract the feature information efficiently. Fig.11 As shown in the figure, the Ghost module generates half of the main feature map by using a set of depth-separable convolutions, and then uses a linear transformation to generate the other half of the feature map. This method efficiently extracts the information of the feature map and significantly reduces the amount of calculation and parameters.

[0044] A perfect detection network should not only consider the full utilization of feature information, but also the lightweight of the detection network itself. Since the feature information is processed by multiple convolutions and pooling layers in the detection layer stage, the spatial information is less and the semantic information is highly concentrated. If the detection network is too complex and the amount of information is small, it is easy to cause overfitting of the detection. Therefore, in the first feature extraction module of each branch, the detection head network uses the Ghost module to replace the ordinary convolution to convert the number of channels of the previous high-dimensional feature information into the number of channels required for detection. Since the amount of calculation at this stage is large, Ghost is used to replace ordinary convolution, which effectively reduces the number of parameters and calculations of the detection layer network. The dynamic adjustment range of the channel after this is small. 3×3 ordinary convolution is used to further fuse the feature information and 1×1 convolution is used to adjust the number of prediction channels to retain the detection performance of the network. Finally, the classification prediction and regression prediction of the decoupling head are spliced ​​through the splicing layer to obtain the output of the detection layer. It can be seen that the detection head network combines the global domain feature information perception ability. At the same time, combined with the distribution of feature channel information, the use of Ghost module and ordinary convolution is dynamically adjusted to maximize the detection performance of the network and maintain a certain lightweight of the detection layer to prevent overfitting.

[0045] Figure 12-Figure 15 The experimental results of this method on SAR images and visible light images are listed. Fig.12 is the original SAR image, Fig.13 is an optical image, Fig.14 As the fusion result, Fig.15 is the target detection result. From the above fusion examples, it can be seen that the fusion result of the above method has good texture fidelity and target intensity information. The fused image combines the target information in the SAR image and the visible light image, and better supports the detection and recognition of the target detection algorithm. In summary, the adaptive adversarial feature fusion target detection method for SAR images and optical images for complex scenes proposed in this application still has ideal detection performance in complex scenes, which proves the robustness of the method and its broad application prospects. Example 3

[0046] This embodiment provides a SAR and optical image adaptive adversarial feature fusion target detection device. The device specifically includes: A filtering module is used to perform speckle filtering on the SAR image using a Refine Lee filter to obtain a filtered SAR image; A registration module, used for performing image registration processing on the filtered SAR image and the optical image to obtain a registered SAR image and optical image; A fusion module is used to perform adaptive feature fusion on the registered SAR image and the optical image by using a channel and spatial attention mechanism and a dual discriminator generative adversarial network to obtain a target fused image; The detection module is used to perform target detection on the target fusion image by using the position attention mechanism and the YOLOv8 algorithm to obtain a target detection result.

[0047] In summary, compared with the prior art, the proposed adaptive adversarial feature fusion target detection device for SAR images and optical images in complex scenes has the following beneficial effects: by using the Refine-Lee filter to filter the SAR image for coherent speckle, the noise in the SAR image is effectively reduced, the image quality is improved, and a better data basis is provided for subsequent feature fusion and target detection; using DDcGAN, the features of SAR and optical images are adaptively fused. CBAM is introduced in the fusion process to give more weight to the target to be detected, so that the fused image can more accurately reflect the characteristic information of the target; YOLOv8 network is used for target detection on the generated multimodal fusion image, and position information is introduced in combination with CA to further improve the accuracy of target detection; by dynamically combining the global attention mechanism and the Ghost module, a new lightweight detection head is constructed, which greatly improves the efficiency of SAR target detection and adapts to the requirements for computing resources in complex scenes; this method makes full use of the all-weather and strong penetration characteristics of SAR images and the color and texture information of optical images, and can achieve high-precision remote sensing image target detection under different weather conditions and environments, with good robustness and broad application prospects. Example 4

[0048] In another feasible embodiment, this embodiment provides a device for SAR and optical image adaptive adversarial feature fusion target detection, and the device may specifically include: A processor; and a memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the steps in any of the above method embodiments. Example 5

[0049] In another feasible embodiment, this embodiment provides a storage medium for SAR and optical image adaptive adversarial feature fusion target detection, and the storage medium may specifically include: The storage medium stores a processing program for SAR and optical image adaptive adversarial feature fusion target detection. When the processing program for SAR and optical image adaptive adversarial feature fusion target detection is executed by the processor, the steps in any of the above method embodiments are implemented.

[0050] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can be modified and varied in various ways. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A SAR and optical image adaptive adversarial feature fusion target detection method, characterized in that: include: The Refine Lee filter is used to perform speckle filtering on the SAR image to obtain the filtered SAR image; Performing image registration processing on the filtered SAR image and the optical image to obtain a registered SAR image and optical image; Adaptively fuse the registered SAR image and the optical image using a channel and space attention mechanism and a dual discriminator generative adversarial network to obtain a target fused image; The position attention mechanism and the YOLOv8 algorithm are used to perform target detection on the target fusion image to obtain a target detection result.

2. The method according to claim 1, characterized in that The method of using the Refine Lee filter to perform speckle filtering on the SAR image to obtain a filtered SAR image includes: Decompose the SAR image window into 9 sub-panes of 3×3, and calculate the mean value in each sub-pane; Four edge gradient operators are used for detection. Each edge gradient operator corresponds to two edge direction windows. The edge direction window whose mean is closer to the mean of the central pixel among the two edge direction windows corresponding to the operator with the largest gradient is selected and recorded as the calibration edge direction window. In the calibrated edge direction window, a Refine Lee filter is used to perform filtering to obtain a filtered SAR image.

3. The method according to claim 1, characterized in that The performing image registration processing on the filtered SAR image and the optical image to obtain the registered SAR image and the optical image includes: Adjusting the filtered SAR image and the optical image to the same size; The filtered SAR image and the optical image are subjected to image registration processing by using the ORB algorithm to obtain the registered SAR image and the optical image.

4. The method according to claim 1, characterized in that: The dual discriminator generative adversarial network includes two discriminators and a generator. The channel and spatial attention mechanism and the dual discriminator generative adversarial network are used to perform adaptive feature fusion on the registered SAR image and the optical image to obtain a target fused image, including: Inputting the registered SAR image and the optical image into a generator to generate an initial fused image; Input the initial fused image and the registered SAR image into one of the discriminators, input the initial fused image and the registered optical image into another discriminator, use the registered SAR image and the optical image as true value images to discriminate the initial fused image, and obtain updated generator parameters; Based on the updated generator parameters, a target fused image is generated.

5. The method according to claim 4, characterized in that The generating a target fused image based on the updated parameters of the generator comprises: Input the registered SAR image and the optical image into the generator with updated parameters, and extract features using the Densenet network structure to obtain initial features; Based on the initial features, the spatial information and channel information of the feature map are jointly focused by using the channel and spatial attention mechanism to extract enhanced features; Perform feature fusion based on the enhanced features to obtain a fused feature map; The fused feature map is subjected to feature reconstruction to obtain a target fused image.

6. The method according to claim 1, characterized in that The target detection is performed on the target fusion image using the position attention mechanism and the YOLOv8 algorithm to obtain the target detection result, including: The YOLOv8 algorithm includes a backbone network module, a feature fusion module and a detection head module; Extracting first feature information from the target fused image through the backbone network module; Using a position attention mechanism to introduce the position information of the target into the feature fusion module, fusing the first feature information through the feature fusion module to obtain feature fusion information; Based on the feature fusion information, the detection head module performs classification prediction and position regression prediction respectively to obtain the target detection result.

7. The method according to claim 6, characterized in that Based on the feature fusion information, the detection head module performs classification prediction and position regression prediction respectively to obtain the target detection result, including: Based on the feature fusion information, a Swin Transformer model is used to perform global feature association and extract global feature information; The global feature information is divided into two branches according to the classification task and the position regression task, and the classification prediction and the position regression prediction are performed by dynamically adjusting the Ghost network and the convolutional network; The classification prediction and position regression prediction results are concatenated to obtain the target detection result.

8. The method according to claim 7, characterized in that The classification prediction and position regression prediction are performed by dynamically adjusting the Ghost network and the convolutional network, including: Each branch is predicted in turn through the Ghost network, 3×3 convolutional network and 1×1 convolutional network to obtain classification prediction and position regression prediction results.

9. A SAR and optical image adaptive adversarial feature fusion target detection device, characterized in that: include: A filtering module is used to perform speckle filtering on the SAR image using a Refine Lee filter to obtain a filtered SAR image; A registration module, used for performing image registration processing on the filtered SAR image and the optical image to obtain a registered SAR image and optical image; A fusion module is used to perform adaptive feature fusion on the registered SAR image and the optical image by using a channel and spatial attention mechanism and a dual discriminator generative adversarial network to obtain a target fused image; The detection module is used to perform target detection on the target fusion image by using the position attention mechanism and the YOLOv8 algorithm to obtain a target detection result.

10. An electronic device, characterized in that: include: processor; and a memory arranged to store computer executable instructions which, when executed, cause the processor to perform the steps of the method as claimed in any one of claims 1 to 8.

Images