Laser interference suppression method based on generative adversarial network

Abstract

The invention discloses a laser interference suppression method based on a generative adversarial network, and relates to the technical field of image processing, and the method comprises the steps: obtaining a laser interference image; inputting the laser interference image into a laser suppression model, and outputting an image after laser suppression; the laser suppression model is a generator obtained by training a generative adversarial network, and the generator in the generative adversarial network adopts an improved Swinin-Unet network structure. The laser interference can be suppressed, and the optical imaging quality can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a laser interference suppression method based on generative adversarial networks. Background Technology

[0002] Optoelectronic imaging systems are widely used in target recognition, industrial automation, remote sensing, and optoelectronic countermeasures. However, the focal plane structure and autofocus mechanism of these systems cause incident light to form a focused spot at the image sensor, which is susceptible to blinding or glare under laser radiation. Laser glare can create point, line, or area laser interference spots, obscuring key image information and features, which can severely impact the performance of detection and tracking systems. Therefore, in addition to excellent imaging capabilities, optoelectronic imaging systems urgently need to recover target features obscured by laser spots.

[0003] With the development of computer vision technology and its widespread application in fields such as image restoration and image inpainting, traditional image restoration methods mainly rely on computational techniques to simulate and replicate the imaging process. These methods perform well in handling some common image degradation problems, such as noise and blur. However, the effect of laser light on the overall image grayscale is uneven. It not only causes overexposure but can also alter the image's contrast and color balance, while simultaneously creating glare at different points, lines, and surfaces, resulting in poor image restoration outcomes. Summary of the Invention

[0004] The purpose of this application is to provide a laser interference suppression method based on generative adversarial networks, which can suppress laser interference and improve optical imaging quality.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] In a first aspect, this application provides a laser interference suppression method based on generative adversarial networks (GANs), the laser interference suppression method based on GANs comprising:

[0007] Acquire laser interference images;

[0008] The laser interference image is input into the laser suppression model, and the laser-suppressed image is output. The laser suppression model is a generator obtained by training a generative adversarial network. The generator in the generative adversarial network adopts an improved Swin-Unet network. The improvement of the improved Swin-Unet network is that each skip connection is modified to a DSTB module and a multi-scale attention module connected in sequence. The DSTB module includes multiple SwinTransformer blocks.

[0009] Optionally, the generative adversarial network is trained, specifically including:

[0010] A training set is constructed by collecting laser interference image data pairs; each laser interference image data pair includes a laser interference image sample and a reference image, wherein the reference image is an uninterrupted image sample corresponding to the laser interference image sample.

[0011] During the training of the generative adversarial network using the training set, laser interference image samples are input into the generator to output a predicted image. A discriminator is used to distinguish between the predicted image and the corresponding reference image, and the trained generator is used as a laser suppression model.

[0012] Optionally, the laser interference image data centering image is an image processed to a uniformly set size; the uniformly set size is 224×224 pixels.

[0013] Optionally, the discriminator in the generative adversarial network comprises multiple sequentially connected basic convolutional modules, each of which includes a convolutional layer, a batch normalization layer, and a LeakyReLU activation function connected in sequence.

[0014] Alternatively, the loss function for training a generative adversarial network can be expressed as:

[0015]

[0016] in, The total loss value is represented by α, which is a coefficient. and These are respectively adversarial loss, perceptual loss, and content loss;

[0017]

[0018] Where G() is the generator, D() is the discriminator, MSE is the mean square error, n is the spatial size of the output feature map of the discriminator, x is the input laser interference image, and D(G(x)) pq This represents the output value of the discriminator at position (p, q) in the laser interference image;

[0019]

[0020] Among them, W (i,j) and H (i,j) Let φ be the width and height of the feature map extracted from the laser interference image in the j-th convolutional layer of the i-th basic convolutional module, respectively. i,j () represents the feature map extracted by the discriminator in the j-th convolutional layer of the i-th basic convolutional module. The weight parameter is θ G generator;

[0021]

[0022] Where m represents different scale levels. and Let represent the mean pixel intensity in the local region of the predicted image and the reference image at the m-th scale, respectively. and Let these represent the standard deviations of the predicted image and the reference image at the m-th scale, respectively. β represents the covariance between the predicted image and the reference image at the m-th scale. m and γ m c1 and c2 are constants; the predicted image is the image output by the generator.

[0023] Optionally, the discriminator in the generative adversarial network may be a VGG16 network, a VGG19 network, a ResNet18 network, or a ResNet50 network.

[0024] Optionally, the laser interference image is an image output by an optoelectronic imaging system.

[0025] Optionally, the DSTB module includes four transformation units connected in sequence. Each transformation unit includes a Win Transformer block, a connection layer, and a convolutional layer connected in sequence. In the first transformation unit, the connection layer connects the input and output of the first Win Transformer block. In the second transformation unit, the connection layer connects the input, output, and output of the first and second Win Transformer blocks. In the third transformation unit, the connection layer connects the input, output, output, and output of the first and second Win Transformer blocks. In the fourth transformation unit, the connection layer connects the input, output, output, and output of the first and second Win Transformer blocks.

[0026] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0027] This application provides a laser interference suppression method based on generative adversarial networks (GANs). The method inputs a laser interference image into a laser suppression model and outputs a laser-suppressed image. The laser suppression model is obtained by training a GAN. The generator in the GAN adopts an improved Swin-Unet network structure. The improved Swin-Unet network structure introduces a hierarchical Transformer structure into the Unet generator network and modifies each skip connection therein into a sequentially connected DSTB module and a multi-scale attention module. This effectively solves the long-distance dependency problem in visual tasks, has efficient feature extraction capabilities and spatial information preservation advantages, and can suppress laser interference and improve optical imaging quality. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A schematic flowchart of a laser interference suppression method based on generative adversarial networks provided in an embodiment of this application;

[0030] Figure 2 This is a schematic diagram of the generative adversarial network training process provided in an embodiment of this application;

[0031] Figure 3 This is a schematic diagram of the Swin-Unet network structure provided in an embodiment of this application;

[0032] Figure 4 A schematic diagram of an improved Swin-Unet network structure provided in an embodiment of this application;

[0033] Figure 5 This is a schematic diagram of the SwinTransformer block structure provided in an embodiment of this application;

[0034] Figure 6 This is a schematic diagram of the DSTB module structure provided in an embodiment of this application;

[0035] Figure 7 This is a schematic diagram of a multi-scale attention module structure provided in an embodiment of this application;

[0036] Figure 8 This is a schematic diagram of a discriminator structure provided in an embodiment of this application;

[0037] Figure 9Laser interference images of targets with different power lasers provided in an embodiment of this application;

[0038] Figure 10 A schematic diagram illustrating various metrics of the generator and discriminator of a generative adversarial network provided in an embodiment of this application on the training set;

[0039] Figure 11 This is a schematic diagram illustrating the laser suppression effect of a laser suppression model provided in an embodiment of this application. Detailed Implementation

[0040] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] This application provides a laser interference suppression method based on generative adversarial networks, such as... Figure 1 As shown, the laser interference suppression method based on generative adversarial networks includes steps 101-102.

[0043] Step 101: Obtain the laser interference image.

[0044] Step 102: Input the laser interference image into the laser suppression model and output the laser-suppressed image; the laser suppression model is a generator obtained by training a generative adversarial network, and the generator in the generative adversarial network adopts an improved Swin-Unet network; the improvement of the improved Swin-Unet network is that each skip connection is modified to a DSTB module and a multi-scale attention module connected in sequence, and the DSTB module includes multiple Swin Transformer blocks.

[0045] The laser interference image is an image output by the photoelectric imaging system.

[0046] This application's Swin-Unet network structure effectively addresses the long-range dependency problem in visual tasks by introducing a hierarchical Transformer structure. It boasts efficient feature extraction capabilities and spatial information preservation advantages. Furthermore, by modifying skip connections to sequentially connected DSTB modules and multi-scale attention modules, it further enhances the network's learning ability for different spatial regions, enabling it to handle complex scenes and larger-scale images. It can suppress laser interference, improve optical imaging quality, and outperform traditional optical imaging quality. It effectively solves the problem of existing optical imaging systems losing target tracking when subjected to laser interference, and has broad application adaptability.

[0047] Most existing methods for suppressing laser interference focus on hardware-level protection, primarily through material improvements, new optical path designs, or anti-reflective coatings. In contrast to physical protection measures, computer vision-based methods, implemented directly on a computer using algorithms, eliminate the need for additional hardware, reducing system complexity and cost.

[0048] In one exemplary embodiment, such as Figure 2 As shown, training a generative adversarial network specifically includes:

[0049] A training set is constructed by collecting laser interference image data pairs. Each laser interference image data pair includes a laser interference image sample and a reference image, wherein the reference image is an uninterrupted image sample corresponding to the laser interference image sample. More specifically, laser interference and uninterrupted image data pairs are collected using a camera. The distance between the camera and the target sample is a predetermined distance, for example, 3 meters.

[0050] During the training of the generative adversarial network using the training set, laser interference image samples are input into the generator to output a predicted image. A discriminator is used to distinguish between the predicted image and the corresponding reference image, and the trained generator is used as a laser suppression model.

[0051] The trained model was validated for actual performance, and more complex scenario data was added to further optimize the laser suppression model.

[0052] In one exemplary embodiment, the Swin-Unet network structure is as follows: Figure 3As shown, the Swin-Unet network architecture is an improved image segmentation network that integrates the Swin Transformer model and the classic U-Net architecture. Swin Transformer is a deep neural network based on a self-attention mechanism. This network effectively addresses the long-range dependency problem in visual tasks by introducing a hierarchical Transformer structure. U-Net, on the other hand, is an encoder-decoder network specifically designed for image segmentation, characterized by the use of skip connections to preserve spatial information during the downsampling process. The Swin-Unet network architecture combines the efficient feature extraction capabilities of Swin Transformer with the spatial information preservation advantages of U-Net.

[0053] The Swin-Unet network includes encoders and decoders.

[0054] In one exemplary embodiment, this application employs an improved Swin-Unet network, such as... Figure 4 As shown, the improvement of the improved Swin-Unet network is that each skip connection is modified to be a DSTB module and an Efficient Multi-Scale Attention (EMA) module connected in sequence, wherein the DSTB module includes multiple SwinTransformer blocks.

[0055] The DSTB module includes four transformation units connected in sequence. Each transformation unit includes a WinTransformer block, a connection layer, and a convolutional layer connected in sequence. In the first transformation unit, the connection layer connects the input and output of the first WinTransformer block. In the second transformation unit, the connection layer connects the input, output, and output of the first and second WinTransformer blocks. In the third transformation unit, the connection layer connects the input, output, output, and output of the first and second WinTransformer blocks. In the fourth transformation unit, the connection layer connects the input, output, output, and output of the first and second WinTransformer blocks.

[0056] The Swin-Unet network employs an innovative encoder structure that initially processes the input image, transforming it into a sequential embedded representation. Specifically, the input image is first segmented into several non-overlapping 4×4 pixel blocks. This segmentation process reduces the feature dimension of each block to 4×4×3 = 48. Then, a linear embedding layer is applied to map the feature dimensions to the model-defined dimension C. The transformed patch tokens undergo feature extraction through several Swin Transformer blocks and a patch merging layer. The patch merging layer performs downsampling and increases the feature dimension, resulting in a hierarchical feature representation. The primary function of the patch merging layer is to integrate local features and reduce sequence length, while the task of the Swin Transformer blocks is to extract rich feature information.

[0057] In the Swin Transformer model, image processing tasks are transformed into a problem of interaction and information flow between a series of image patches. The Swin Transformer model consists of several Swin Transformer blocks, each containing multiple parallel attention heads and feedforward network layers, forming the basic computational unit. Within the block, image patches interact through local attention mechanisms, promoting the exchange and fusion of global information. However, Swin Transformer blocks differ from conventional multi-head self-attention (MSA) modules because they are based on shifted windows. In the Swin-Unet model, two consecutive Swin Transformer blocks are used sequentially. Each Swin Transformer block consists of a LayerNorm (LN) layer, a multi-head self-attention module, residual connections, and a two-layer multilayer perceptron (MLP) with a GELU activation function. Figure 5 As shown, in two consecutive Swing Transformer blocks, window-based multi-head self-attention (W-MSA) and shift-window-based multi-head self-attention (SW-MSA) modules are applied respectively, and their specific calculation formulas are as follows:

[0058]

[0059] Among them, z l and Let represent the outputs of the SW-MSA module and the MLP in the l-th block, respectively. The self-attention formula for each window is:

[0060]

[0061] in, M represents the query, key, and value matrix, respectively. 2 This indicates the number of patches in the window, where d represents the dimension of Q or K. To offset relative positions and further enrich the semantic information of the encoder features in the enhanced model, this application replaces the original simple skip connections with dense Swing Transformer blocks (DSTBs), where the Dense Swing Transformer Block (DSTB) structure is as follows: Figure 6 As shown.

[0062] Figure 6 middle The output for each Swing Transformer is given by the formula:

[0063]

[0064] in, and The outputs of the 1st, (t-1)th, and tth Swing Transformer blocks are respectively, F i P is the input to the first Swing Transformer block. ST and P FF These represent the Swing Transformer operation and the feature fusion operation, respectively.

[0065] Furthermore, the design of connecting the dense Swing Transformer block to an efficient multi-scale attention module that learns across spatial dimensions using EMA aims to further enhance the network's ability to capture multi-scale features and strengthen its stability and robustness. The dense Swing Transformer block extracts global and local information from images through multiple transformations and multi-scale feature extraction. However, relying solely on self-attention within local windows may limit the network's understanding of the overall image structure, especially when dealing with complex scenes and large-scale images.

[0066] Therefore, introducing an efficient multi-scale attention module based on EMA cross-spatial learning can effectively solve this problem. This module adaptively weights attention across multiple scales, dynamically adjusting the weights of features at each scale, further enhancing the network's learning ability for different spatial regions. Through cross-spatial learning, channels are grouped without channel dimensionality reduction, preserving information in each channel while reducing computational overhead. Its network structure is as follows: Figure 7 As shown.

[0067] In the EMA module, the feature maps extracted by the DSTB module are first processed. As input, the feature map is divided into G groups of feature maps along the channel dimension of X: X = [X0, X1, ..., X...].i ,...,X G ], where sub-feature maps Subsequently, in the 1×1 branch (i.e. Figure 7 On the right side, a two-dimensional global average pooling operation (AvgPool) is performed along the horizontal and vertical directions to encode the channels, establishing the interaction between the channels and their spatial locations, and simultaneously generating two channel attention maps connected vertically. The calculation formula for this operation is as follows:

[0068]

[0069] In the formula, H and W are the height and width of the feature map, respectively. and These are the channel attention maps generated along the horizontal and vertical axes, respectively, where i and j are the width and height of the C-th channel input, respectively. and The input features are the spatial positions (j, W) and (H, i) in the Cth channel, respectively.

[0070] Subsequently, a non-linear Sigmoid activation function is used to combine the two channel attention maps processed by the 1×1 convolution within each group. After normalization, a two-dimensional average pooling operation and Softmax are further calculated to obtain an intermediate feature map of dimension C / G×1. The formula for calculating the two-dimensional global average pooling is:

[0071]

[0072] Among them, Z C This represents the feature map of the C-th channel after two-dimensional average pooling (i.e., Avg Pooling). The input feature represents the spatial location (i,j) within the C-th channel. The intermediate feature map after Softmax operation is multiplied by the sub-feature map after 3×3 convolution to obtain the first spatial attention weight map with dimension 1×H×W.

[0073] The output from the 3×3 branch is concatenated with the feature map from the 1×1 branch after undergoing 3×3 convolution, 2D flat pooling, and softmax operations, resulting in a second spatial attention map with dimensions 1×H×W. Finally, the two spatial attention maps are added and normalized using the sigmoid activation function to obtain the final attention map. This result is then mapped onto the original feature map, thus focusing the model's attention on key regions.

[0074] In the encoder section, the C-dimensional input image with a resolution of H / 4 × W / 4 is divided into several non-overlapping image patches through a patch partitioning operation, and then channel mapping is performed through a linear embedding layer. Next, the feature maps are sequentially fed into two consecutive Swing Transformer blocks for initial feature representation learning. During the learning process, the feature dimension and resolution remain unchanged. Simultaneously, the patch merging layer compresses the spatial resolution by a 2x downsampling while doubling the number of channels, reducing computational complexity and enhancing feature expressiveness. This operation is repeated three times in the encoder, working in conjunction with the Swing Transformer blocks to achieve multi-level semantic modeling.

[0075] The decoder is built upon the SwingTransformer block and is designed to correspond to the encoder. Unlike the patch merging layer used in the encoder, the decoder employs a patch expanding layer to perform feature map upsampling. The patch expanding layer restores a higher-resolution feature map by rearranging adjacent feature map patches, achieving a 2x resolution upsampling while halving the number of feature channels to ensure feature dimension consistency.

[0076] In one exemplary embodiment, the discriminator is structured as follows: Figure 8 As shown, the specific configuration is described below:

[0077] 1) The input layer receives an image input of size (C, H, W), where C represents the number of channels in the image, and H and W represent the height and width of the image, respectively.

[0078] 2) This discriminator contains several basic convolutional modules. Each basic convolutional module consists of a convolutional layer, a batch normalization layer, and a LeakyReLU activation function, connected sequentially. In the convolutional layer, the number of input channels is converted to a specified number of output channels. The convolutional layer uses a 3×3 kernel with a stride of 1 and edge padding of 1. Following the convolution is a batch normalization layer, followed by the LeakyReLU activation function. In subsequent convolutional layers, the number of channels remains the same, and a 3×3 kernel is still used, but the stride becomes 2 and the edge padding is 1. Each convolutional layer is followed by a batch normalization layer and a LeakyReLU activation function.

[0079] The stacked convolutional blocks (basic convolutional modules) are configured as follows:

[0080] The first convolutional block does not contain a batch normalization layer, has C input channels (C=1 for grayscale images), and 64 output channels.

[0081] The second convolutional block contains two basic convolutional modules with 64 input channels and 128 output channels.

[0082] The third convolutional block contains two basic convolutional modules with 128 input channels and 256 output channels.

[0083] The fourth convolutional block contains two basic convolutional modules with 256 input channels and 512 output channels.

[0084] Finally, the output layer is a convolutional layer with 512 input channels and 1 output channel, using a 3×3 convolutional kernel with a stride of 1 and padding of 1. The output size generated by this layer is (1, patch_h, patch_w), where patch_h = H / 2^4, patch_w = W / 2^4, patch_h is the height of the output image, and patch_w is the width of the output image.

[0085] In one exemplary embodiment, the images in each laser interference image data pair are images processed to a uniformly set size; the uniformly set size is 224×224 pixels.

[0086] In an exemplary embodiment, the loss function for training a generative adversarial network is expressed as:

[0087]

[0088] in, The total loss value is the loss function defined in the task of recovering laser-interferenced images using generative adversarial networks, where α is a coefficient. and These represent adversarial loss, perceptual loss, and content loss, respectively; α = 0.01.

[0089] The generator loss function is defined as:

[0090]

[0091] Where G(·) is the generator, D(·) is the discriminator, MSE is the mean squared error, and n represents the spatial size of the discriminator's output feature map, that is, the side length of the two-dimensional scoring map output by the discriminator after convolution processing the input image (if the discriminator output is an n×n feature map, then it contains n 2 The score value at each position (p,q) corresponds to the discriminator's true or false prediction for a local region (patch) in the image; x is the input laser interference image, D(G(x)). pq This represents the output value of the discriminator at position (p, q) in the laser interference image (row p, column q).

[0092] The discriminator loss function is defined as:

[0093]

[0094] Among them, L disc-MSE Let D(y) be the loss of the discriminator, y be the reference image, and D(y) be the loss of the discriminator. pq This represents the output value of the discriminator at position (p, q) in the reference image.

[0095] Perceptual loss: Measures the structural and textural similarity between the generated and original images. Mean squared error loss is used to calculate the distance between the original and generated images in the feature space, and a pre-trained discriminator is used for feature extraction. By training the model using this objective function, the reconstructed image can continuously approach the original image in the feature space.

[0096] When the discriminator uses VGG16, the discriminator's knowledge loss function is defined as:

[0097]

[0098] Among them, W (i,j) For and H (i,j) Let φ be the width and height of the feature map extracted from the laser interference image in the j-th convolutional layer of the i-th basic convolutional module, respectively. i,j (·) represents the feature map extracted by the discriminator in the j-th convolutional layer of the i-th basic convolutional module. The weight parameter is θ G The generator, θ G This represents all the parameters that make up the generator network G, such as convolution kernel weights, BN layer parameters, etc., with the goal of restoring the disturbed image to an interference-free image that is close to the reference image.

[0099] Content loss: The Multi-Scale Structural Similarity Index Measure (MSSSIM) loss function can evaluate the visual quality of the generated image at multiple scales, thereby driving the generative network to produce restoration results that are closer to the original image in terms of color and texture details. This design aims to ensure the naturalness and realism of the restored image in visual perception, while maintaining the richness and integrity of image details.

[0100]

[0101] Where M represents different scale levels. and Let represent the mean pixel intensity in the local region of the predicted image and the reference image at the m-th scale, respectively. and Let these represent the standard deviations of the predicted image and the reference image at the m-th scale, respectively. β represents the covariance between the predicted image and the reference image at the m-th scale. m and γ m β is a coefficient. m and γ m The terms c1 and c2 are constants used to represent the relative importance between the two terms. These constants are introduced as smoothing terms to avoid the denominator approaching zero. The predicted image is the image output by the generator. In an exemplary embodiment, the discriminator in the generative adversarial network uses a VGG16 network, a VGG19 network, a ResNet18 network, or a ResNet50 network.

[0102] In one exemplary embodiment, the parameters for acquiring images from the training set are shown in Table 1. The optical imaging system is aimed at the target, and the position of the laser is fixed to interfere with the camera lens. Images are acquired in real time using a computer, resulting in 4800 images with and without laser interference. The target is approximately 3 meters away from the camera.

[0103] Table 1 List of Instruments and Equipment

[0104] Instruments and equipment Specific parameters Photoelectric imaging system Resolution: 1440*1080 laser Wavelength: 532nm computer Model: Legion Y7000PIRH8 Target car Size: 30cm*20cm*20cm Sorebo beam expander GBE10-A

[0105] An optoelectronic imaging system is used to acquire images of the target vehicle. A laser is used as the laser interference source, and a beam expander is used to expand the laser output from the laser. The image output by the optoelectronic imaging system, i.e., the laser interference image, is acquired by a computer.

[0106] By adjusting the laser power, images of laser interference at different power levels were acquired. Figure 9 In (a), (b), (c), (d), (e), and (f), as the laser power increases, the light spot of the target scene continues to expand. The 4800 sets of collected images are divided into training set and test set in an 8:2 ratio, with 3840 images in the training set and 960 images in the test set.

[0107] The effectiveness of laser suppression was verified using a generative adversarial network (GAN) algorithm. Experimental environment parameters are shown in Table 2. GAN training was conducted on a system equipped with an NVIDIA GeForce RTX 3090 graphics card. The optimizer used was Adam, with a specific AdamW variant set. The first moment decay rate (β1) was set to 0.5, the second moment decay rate (β2) to 0.999, and the learning rate to 5e-4. Due to hardware resource constraints, the batch size was configured to 16, and a learning rate decay strategy was introduced from the 6th training epoch. All input images were preprocessed to a uniform size of 224×224 pixels. The training dataset underwent 105 training epochs, and the entire training process took approximately 5 hours.

[0108] Table 2 Experimental Environment Parameters

[0109] Experimental System Windows 10 CPU 3.00GHzIntel(R)Xeon(R)Gold6248R GPU NVIDIA GeForce RTX 3090 Memory 32GB Development Environment Python 3.9 Deep learning framework Pytorch

[0110] The aforementioned test set was used to evaluate the effectiveness of model training. Specifically, metrics such as Peak Signal-to-Noise Ratio (PSNR) and Color Difference Index (CIEDE2000) were used for evaluation, and their calculation formulas are as follows:

[0111]

[0112] Peak Signal-to-Noise Ratio (PSNR) is a reconstruction quality evaluation metric based on pixel-level error. MAX represents the maximum value of a pixel (usually 255), MSE is the mean square error between the reconstructed image and the reference image, and I(i',j') and K(i',j') represent the reference image and the reconstructed image, respectively. A higher PSNR value indicates less image distortion and better quality.

[0113]

[0114] Where CIEDE2000 represents color difference, and H and W represent the height and width of the image, respectively. This represents the color difference between the generated image and the reference image at position (i', j'). ΔE 00 The smaller the value, the closer the colors of the two images are, and the less obvious the visual difference is.

[0115] like Figure 10 To generate curves showing the changes in loss value, MSSSIM, PSNR, and CIDDE2000 metrics of the adversarial network on the training set. Figure 10 The discriminator in (a) stabilizes after 60 batches, while the generator stabilizes after 90 batches. Multi-scale structural similarity evaluates the visual quality of an image across multiple scales, thus providing a more comprehensive measure of image quality. A similarity score closer to 1 indicates a better image generated by the model. Figure 10 In (b), (c), and (d), the MSSSIM of the model stabilized at around 0.98 after 60 batches. Simultaneously, the PSNR improved from around 15 to around 34, an increase of approximately 1.26 times, indicating that the GAN model has good suppression performance. Furthermore, the model processes laser-interferenced images in approximately 600ms, demonstrating good real-time performance. In addition, the CIDDE2000 error decreased from approximately 25% to approximately 2%. It can be seen that the laser suppression model in this application maintains the high imaging quality of the optical imaging system while improving the system's resistance to laser interference.

[0116] The glare effect observed in the experiment was generated by the laser's normal incident detector. Figure 11 The results of the image suppression process are shown, including a series of images with varying background edge complexity. The columns from left to right present the laser-interference image, the suppressed image, and the control image without laser interference. The laser-interference image is... Figure 11 In (a), (d), and (g), the suppressed images are: Figure 11 In (b), (e), and (h), the control images without laser interference are... Figure 11 (c), (f), and (k). The impact of lasers on optical imaging systems is not limited to damage to the central bright spot, but also includes diffuse reflection interference on all sensing elements of the detector, which leads to a decrease in image quality.

[0117] Comparative analysis of the suppressed images with those without laser interference demonstrates that a well-trained generative adversarial network can effectively eliminate diffuse reflection interference caused by lasers and skillfully recover overexposed areas on the imaging device affected by interfering lasers. Tests on images with complex backgrounds further confirm the high robustness of the proposed method. Figure 11 The second row of images demonstrates the high structural complexity of the glare region, while the first row shows low structural complexity; both achieve good suppression results. Experimental results show that when the laser causing glare creates an overexposed region on the photosensitive surface of the detector, information in that region is primarily lost. If the target image has low structural complexity within the overexposed region, and there is a certain semantic similarity between the pixels in that region and surrounding pixels, the designed generative adversarial network can infer the lost boundary information and texture details through edges, thus achieving superior restoration results. Conversely, if the structural complexity within the overexposed region is high, although the restored image can generate a reasonable texture structure, some corresponding details will inevitably be lost.

[0118] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A laser interference suppression method based on generative adversarial networks, characterized in that, The laser interference suppression method based on generative adversarial networks includes: Acquire laser interference images; The laser interference image is input into the laser suppression model, and the laser-suppressed image is output. The laser suppression model is a generator obtained by training a generative adversarial network. The generator in the generative adversarial network adopts an improved Swin-Unet network. The improvement of the improved Swin-Unet network is that each skip connection is modified to a DSTB module and a multi-scale attention module connected in sequence. The DSTB module includes multiple SwinTransformer blocks.

2. The laser interference suppression method based on generative adversarial networks according to claim 1, characterized in that, Training a generative adversarial network specifically includes: A training set is constructed by collecting laser interference image data pairs; each laser interference image data pair includes a laser interference image sample and a reference image, wherein the reference image is an uninterrupted image sample corresponding to the laser interference image sample. During the training of the generative adversarial network using the training set, laser interference image samples are input into the generator to output a predicted image. A discriminator is used to distinguish between the predicted image and the corresponding reference image, and the trained generator is used as a laser suppression model.

3. The laser interference suppression method based on generative adversarial networks according to claim 2, characterized in that, The laser interference image data centering images are images that have been processed to a uniformly set size; the uniformly set size is 224×224 pixels.

4. The laser interference suppression method based on generative adversarial networks according to claim 1, characterized in that, The generative adversarial network (GAN) has a discriminator consisting of multiple sequentially connected basic convolutional modules. Each basic convolutional module includes a convolutional layer, a batch normalization layer, and a LeakyReLU activation function, all connected in sequence.

5. The laser interference suppression method based on generative adversarial networks according to claim 4, characterized in that, The loss function for training a generative adversarial network is expressed as: in, The total loss value is represented by α, which is a coefficient. and These are respectively adversarial loss, perceptual loss, and content loss; Where G() is the generator, D() is the discriminator, MSE is the mean square error, n is the spatial size of the output feature map of the discriminator, x is the input laser interference image, and D(G(x)) pq This represents the output value of the discriminator at position (p, q) in the laser interference image; Among them, W (i,j) For and H (i,j) Let φ be the width and height of the feature map extracted from the laser interference image in the j-th convolutional layer of the i-th basic convolutional module, respectively. (i,j) () represents the feature map extracted by the discriminator in the j-th convolutional layer of the i-th basic convolutional module. The weight parameter is θ G generator; Where M represents different scale levels. and Let represent the mean pixel intensity in the local region of the predicted image and the reference image at the m-th scale, respectively. and Let these represent the standard deviations of the predicted image and the reference image at the m-th scale, respectively. β represents the covariance between the predicted image and the reference image at the m-th scale. m and γ m c1 and c2 are constants; the predicted image is the image output by the generator.

6. The laser interference suppression method based on generative adversarial networks according to claim 1, characterized in that, The discriminator in the generative adversarial network adopts a VGG16 network, a VGG19 network, a ResNet18 network, or a ResNet50 network.

7. The laser interference suppression method based on generative adversarial networks according to claim 1, characterized in that, The laser interference image is an image output by the photoelectric imaging system.

8. The laser interference suppression method based on generative adversarial networks according to claim 1, characterized in that, The DSTB module includes four transformation units connected in sequence. Each transformation unit includes a Swing Transformer block, a connection layer, and a convolutional layer connected in sequence. In the first transformation unit, the connection layer connects the input and output of the first Swing Transformer block. In the second transformation unit, the connection layer connects the input, output, and output of the first Swing Transformer block. In the third transformation unit, the connection layer connects the input, output, output, and output of the first and second Swing Transformer blocks. In the fourth transformation unit, the connection layer connects the input, output, output, and output of the first and second Swing Transformer blocks.

Images