A Deep Learning-Based Method for Suppressing Stray Light Noise in Large Field-of-View Images
By constructing a pyramid-shaped deformable convolutional large kernel attention mechanism denoising model, the problem of stray light noise suppression in large field-of-view small aperture telescopes is solved, improving image quality and target positioning accuracy, and is applicable to large field-of-view optical systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHANGCHUN SATELLITE OBSERVATORY OF NAT ASTRONOMICAL OBSERVATORY OF CHINESE ACAD OF SCI
- Filing Date
- 2024-07-16
- Publication Date
- 2026-06-30
AI Technical Summary
Existing technologies struggle to effectively suppress stray light noise, especially moonlight and thin cloud noise, in large field-of-view small-aperture telescopes, leading to a decrease in the signal-to-noise ratio of the observed target and affecting detection performance.
A denoising model based on a deep learning-based pyramid-shaped deformable convolutional kernel attention mechanism is adopted. The feature pyramid is constructed by multiple downsampling and upsampling, and combined with the deformable convolutional kernel attention mechanism, the capture and removal of stray light features are enhanced.
High-quality stray light noise suppression was achieved, improving the image signal-to-noise ratio and target positioning accuracy, and ensuring the detection performance of the large field-of-view small aperture telescope.
Smart Images
Figure CN118761927B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of optoelectronic telescope technology, and more specifically to a method for suppressing stray light noise in large field-of-view images based on deep learning. Background Technology
[0002] Large field-of-view, small-aperture telescopes are survey-type optoelectronic devices characterized by high search efficiency, strong detection capabilities, and long detection arcs, and are also inexpensive, making them widely used for space target observation. To achieve rapid searching and long-arc tracking of space targets over large sky areas, a large field-of-view optical system is essential, and it also represents one of the development trends in space target detection systems.
[0003] However, large field-of-view optical systems are highly sensitive to stray light. Once stray light reaches the image plane, it will generate stray radiation noise on the detector surface. This can cause inhomogeneities in the image plane and reduce the signal-to-noise ratio of the observed target. In severe cases, it can completely drown out the observed target, rendering the telescope unable to function properly.
[0004] Stray light refers to non-imaging rays radiating onto the detector surface in an optical system, as well as imaging rays that travel through abnormal paths to reach the detector surface. Stray light can be categorized into two types based on its source: internal stray light and external stray light. Internal stray light is mainly thermal radiation generated by control motors, temperature-controlled heat sources, and high-temperature optical components, primarily present in infrared imaging systems. For visible light detection systems, external stray light, caused by direct illumination from external radiation sources or by multiple reflections and scatterings from internal components, plays a major role. These radiation sources include moonlight and atmospheric scattered light (i.e., thin clouds). Therefore, effective suppression of stray light, especially moonlight and thin clouds, is one of the key technologies for ensuring the detection performance of small-aperture, large-field-of-view telescopes.
[0005] Currently, research on stray light suppression from moonlight and thin clouds can be divided into two categories: one is from the perspective of the optical system, which involves analyzing the sources of stray light and improving the design of the optical system to fundamentally suppress stray light; the other is from the perspective of the image, which involves recovering a clear image from a degraded image based on the characteristics of stray light in the image. Typical stray light suppression structures include hoods, baffles, and apertures. While using stray light suppression structures is the most direct and effective method, stray light suppression structures for large field-of-view small-aperture telescopes must meet the requirements of being lightweight and compact. This leads to a significant increase in the cost of large field-of-view small-aperture telescopes using such structures, making them unsuitable for application. Clearly, improving image processing capabilities is the most important approach.
[0006] Suppressing moonlight and thin clouds has always been a challenging problem in spatial target image processing. The noise generated by moonlight and thin clouds in spatial target images is randomly distributed, complex and varied in shape, and belongs to the high-frequency part of the image, constituting a major component of the entire image. Furthermore, the gray values of the spatial target and background stars are not higher than its gray value, and the correlation between noise in adjacent frames is weak. Traditional denoising methods, including spatial domain denoising, transform domain denoising, sparse transform-based denoising, and hybrid denoising methods, are difficult to cope with the complex and varied noise in real-world scenes, and their ability to suppress moonlight and thin cloud noise is limited.
[0007] In recent years, deep learning-based denoising methods have emerged. Deep learning, with its powerful learning capabilities and adaptability, has achieved great success in image processing. Compared to traditional denoising methods, deep learning can learn complex nonlinear mapping relationships from large amounts of image data, thus better capturing the features and structural information in images. By constructing end-to-end deep neural network models, stray light features and denoising patterns can be learned directly from image data. This makes deep learning methods more flexible and applicable, better able to handle complex and varied stray light noise, and plays an important role in improving image quality and scientific analysis. This invention aims to use deep learning methods to suppress moonlight and thin cloud noise. Summary of the Invention
[0008] To address the problems that existing stray light suppression methods are unsuitable for large field-of-view, small-aperture telescopes, and that denoising methods struggle to handle complex and varied noise in real-world scenes, as well as having limited ability to suppress moonlight and thin cloud noise, this invention provides a deep learning-based stray light noise suppression method for large field-of-view images.
[0009] A deep learning-based method for suppressing stray light noise in large field-of-view images is proposed. This method utilizes a pyramid-shaped deformable convolutional large kernel attention mechanism to achieve noise reduction. The specific steps are as follows:
[0010] Step 1: Downsampling operation;
[0011] The original spatial target image under stray light interference is input into the upsampling layer of the pyramid-shaped deformable convolutional large kernel attention mechanism denoising model for two consecutive downsampling operations to obtain the downsampled sub-image.
[0012] Step 2: Feature extraction;
[0013] The downsampled sub-image obtained in step one Feature extraction is performed to obtain feature maps at scale l2. downsampled sub-images Feature extraction is performed to obtain feature maps at scale l1.
[0014] Step 3: Upsampling operation;
[0015] For feature maps A double upsampling operation was performed, and the feature map was compared with the feature map. The images are stitched together, convolutionally processed, and then upsampled twice to obtain the denoised output image y. i .
[0016] Furthermore, the specific process of performing two downsampling operations in step one is as follows:
[0017] Step 11: First downsampling by 2x to obtain the original spatial target image at scale l0 under conditions of stray light interference. Perform a 2x downsampling to obtain four downsampled sub-images at scale l1. Size is Expressed as a formula:
[0018]
[0019] In the formula, c is the number of channels in the image, h is the height of the image, w is the width of the image, and ↓2 indicates a double downsampling operation, resulting in four downsampled sub-images. The size is reduced to the original spatial target image under conditions of stray light interference. i 1 / 4;
[0020] Step 2. Second downsampling: Apply the second 2x downsampling to the four downsampled sub-images at scale l1. All images were downsampled by a factor of 2 to obtain 16 downsampled sub-images at scale l2. Size is Expressed as a formula:
[0021]
[0022] In the formula, there are 16 downsampled sub-images. The size is reduced to the original spatial target image under conditions of stray light interference. i 1 / 16 of.
[0023] Furthermore, in step two, a feature extraction module is used. and feature extraction module Implement downsampling of sub-images and downsampled sub-images Feature extraction;
[0024] The feature extraction module It consists of dual convolutional activation layers;
[0025] The feature extraction module Downsampled sub-images After processing through n convolutional activation layers, then through m convolutional normalization activation layers, and finally through a deformable convolutional large kernel attention mechanism layer, the feature extraction of the disturbed region is enhanced.
[0026] Furthermore, the specific process of step two is as follows:
[0027] Feature extraction module downsampled sub-images Feature extraction is performed to obtain feature maps at scale l2. Size is Expressed as a formula:
[0028]
[0029] Feature extraction module downsampled sub-images Feature extraction is performed to obtain feature maps at scale l1. Size is Expressed as a formula:
[0030]
[0031] Furthermore, in step three, the feature map... A double upsampling operation was performed, and the feature map was compared with the feature map. The image is stitched together, passed through a Conv layer, and finally upsampled twice to obtain the denoised output image y. i ; expressed as a formula:
[0032]
[0033] In the formula, C represents the splicing operation, ↑2 represents the doubling upsampling operation, and y i The image is denoised and has dimensions c×h×w.
[0034] Furthermore, the pyramid-shaped deformable convolutional large kernel attention mechanism denoising model includes a downsampling layer, a feature extraction layer, and an upsampling layer;
[0035] The downsampling layer is used to perform two double downsampling operations on the original spatial target image under stray light interference conditions to obtain a downsampled sub-image;
[0036] The feature extraction layer is implemented through a feature extraction module. and feature extraction module composition;
[0037] The feature extraction module Downsampled sub-images After extraction using a double convolutional activation layer, the feature map is extracted.
[0038] The feature extraction module Downsampled sub-images After processing through n convolutional activation layers, followed by m convolutional normalization activation layers, and finally a deformable convolutional large kernel attention mechanism layer, the feature map is extracted.
[0039] The upsampling layer will extract feature maps. After performing a double upsampling operation and comparing with the feature map The images are stitched together and then upsampled twice before being output as denoised images.
[0040] Furthermore, the dual convolutional activation layer is composed of convolutional units, modified linear units, and convolutional units; the convolutional activation layer is composed of convolutional units and modified linear units; and the convolutional normalization activation layer is composed of convolutional units, normalization units, and modified linear units.
[0041] The beneficial effects of this invention are as follows: The noise suppression method described in this invention constructs a novel pyramid-shaped deformable convolutional large kernel attention (D-LKA-Attention) denoising model. By combining a pyramid structure with the deformable convolutional large kernel attention mechanism, the ability to suppress stray light noise in spatial target images is improved. Using spatial target images with stray light interference as input to the model, a pyramid structure is constructed to expand the receptive field. A deeper feature pyramid is built through multiple downsampling and upsampling operations, enabling the model to capture stray light features at different scales. Furthermore, the addition of the deformable convolutional large kernel attention mechanism focuses more on edge features, thereby better removing stray light interference.
[0042] In the method of this invention, a novel attention mechanism beyond self-attention is used. A large convolutional kernel is employed to fully understand the volume context, and the flexible kernel shape of deformable convolution is used to enhance the definition of object boundaries. Therefore, the introduction of this mechanism aims to make the model more focused on edge features and improve the ability to suppress stray light noise near the edges, thereby more effectively recovering space targets and background stars from stray light.
[0043] In this invention, a pyramid-shaped deformable convolutional large kernel attention mechanism denoising model is trained using a noisy image of a space target. Test results show that it can achieve a peak signal-to-noise ratio (PSNR) of up to 32.540 and a structural similarity (SSIM) of 0.938. This indicates that the model can still maintain high-quality images after suppressing stray light. After recovering the space target and background stars, the space target is astronomically located. Test results show that the target location accuracy is better than 5 arcseconds. This indicates that the model not only recovers the space target and background stars, but also effectively preserves their grayscale values, shapes, and positional information. Attached Figure Description
[0044] Figure 1 This is a block diagram illustrating the principle of a deep learning-based method for suppressing stray light noise in large field-of-view images, as described in this invention.
[0045] Figure 2 The diagram shows the principle of the feature extraction module; where (a) is the feature extraction module. The schematic diagram is shown in (b), which is the feature extraction module. (c) is the schematic diagram of the deformable convolution big kernel attention mechanism.
[0046] Figure 3 The images shown are renderings of the spatial target obtained using the method described in this invention; where (a) is an image with stray light, (b) is an image after model denoising and restoration, and (c) is an image without stray light.
[0047] Figure 4 The image shows the effect of background stars obtained by applying the method described in this invention; where (a) is an image with stray light, (b) is an image after model denoising and restoration, and (c) is an image without stray light.
[0048] Figure 5 This is a comparison image of the sequence before and after processing. Detailed Implementation
[0049] Combination Figures 1 to 5This implementation describes a deep learning-based method for suppressing stray light noise in large field-of-view images. This method constructs a novel pyramid-shaped deformable convolutional large kernel attention mechanism denoising model. By combining the pyramid structure with the deformable convolutional large kernel attention mechanism, the suppression capability of stray light noise in spatial target images is improved. Using a spatial target image with stray light interference as the model input, a pyramid structure is constructed to expand the receptive field. A deeper feature pyramid is built through multiple downsampling and upsampling operations, enabling the model to capture stray light features at different scales. Furthermore, the addition of a deformable convolutional large kernel attention mechanism further focuses on edge features, thereby better removing stray light interference. The model consists of downsampling operations, feature extraction, and upsampling operations. Its overall framework is as follows: Figure 1 As shown. The specific process of this method is as follows:
[0050] I. Downsampling operation;
[0051] Space target image under conditions of stray light interference at scale l0 The input is subjected to two consecutive downsampling operations in a pyramid-shaped deformable convolution kernel attention mechanism denoising model.
[0052] The first 2x downsampling was performed, resulting in four downsampled sub-images at scale l1. Size is The calculation formula is:
[0053]
[0054] Where c represents the number of channels in the image, h represents the height of the image, w represents the width of the image, ↓2 represents a 2x downsampling operation, and there are 4 downsampled sub-images. The size is reduced to the original spatial target image under conditions of stray light interference. i 1 / 4 of it.
[0055] A second round of downsampling was performed, resulting in 16 downsampled sub-images at the l2 scale. Size is The calculation formula is:
[0056]
[0057] Where c represents the number of channels in the image, h represents the height of the image, w represents the width of the image, and ↓2 indicates a 2x downsampling operation, resulting in 16 downsampled sub-images. The size is reduced to the original spatial target image under conditions of stray light interference. i 1 / 16 of.
[0058] II. Feature Extraction;
[0059] Using feature extraction module downsampled sub-images Feature extraction is performed to obtain features at the l2 scale. Size is Feature extraction module It is a ConvReluConv structure (i.e., a dual convolutional activation layer composed of a Conv+Relu+Conv structure, wherein the dual convolutional activation layer is implemented by convolutional units, rectified linear units, and convolutional units); such as Figure 2 As shown in (a) in the figure. The calculation formula is:
[0060]
[0061] Using feature extraction module downsampled sub-images Feature extraction is performed to obtain features at the l1 scale. Size is The calculation formula is:
[0062]
[0063] The feature extraction module Structure such as Figure 2 As shown in (b) of the diagram, (1) firstly... (1) Processed by n=1 layers of ConvReLU structure (i.e., convolutional activation layer composed of Conv+ReLU structure, which is implemented by convolutional units and rectified linear units). (2) Then processed by m=10 layers of ConvBNReLU structure (i.e., convolutional normalized activation layer composed of Conv+BN+ReLU structure, which is implemented by convolutional units, normalized units and rectified linear units). (3) Finally processed by deformable convolutional large kernel attention mechanism layer to enhance feature extraction of the disturbed region, thereby better removing the interference. Its structure is as follows: Figure 2 As shown in (c), this mechanism is a novel attention mechanism that goes beyond self-attention. It uses a large convolutional kernel to fully understand the volume context and utilizes the flexible kernel shape of deformable convolution to enhance the definition of object boundaries. Therefore, the introduction of this mechanism aims to make the model more focused on edge features and improve the ability to suppress stray light noise near the edge, thereby more effectively recovering space targets and background stars from stray light.
[0064] III. Upsampling operation;
[0065] For the feature extraction module Introduce a double upsampling operation, and with The data is concatenated, passed through a Conv layer, and finally subjected to a double upsampling operation to obtain the output result y.i The calculation formula is:
[0066]
[0067] Where C represents the concatenation operation, ↑2 represents the doubling upsampling operation, and y i The dimensions of the denoised image are c×h×w.
[0068] Combination Figure 3 and Figure 5 This embodiment describes training a pyramid-shaped deformable convolutional large kernel attention mechanism denoising model using a noisy spatial target image. Test results show that it can achieve a peak signal-to-noise ratio (PSNR) as high as 32.540 and a structural similarity (SSIM) of 0.938. Figure 3 and Figure 4 As shown. This indicates that the model can still maintain high-quality images after suppressing stray light. After recovering the space target and background stars, astronomical positioning of the space target was performed. Test results show that the target positioning accuracy is better than 5 arcseconds. Figure 5 As shown in Table 1, the external symbol accuracy indicates that the model not only recovered the space targets and background stars, but also effectively preserved their grayscale values, shapes, and positional information. Table 1 shows the location information of the space targets.
[0069] Table 1
[0070]
[0071]
[0072] The above results demonstrate that the proposed method for constructing a pyramid-shaped deformable convolutional large kernel attention mechanism denoising model not only effectively removes stray light noise but also ensures the measurement accuracy of large field-of-view small aperture telescopes.
[0073] In this embodiment, the pyramid-shaped deformable convolutional large kernel attention mechanism denoising model includes a downsampling layer, a feature extraction layer, and an upsampling layer;
[0074] The downsampling layer is used to perform two double downsampling operations on the original spatial target image under stray light interference conditions to obtain a downsampled sub-image;
[0075] The feature extraction layer is implemented through a feature extraction module. and feature extraction module composition;
[0076] The feature extraction module Downsampled sub-images After extraction using a double convolutional activation layer, the feature map is extracted.
[0077] The feature extraction module Downsampled sub-images After processing through n convolutional activation layers, followed by m convolutional normalization activation layers, and finally a deformable convolutional large kernel attention mechanism layer, the feature map is extracted.
[0078] The upsampling layer will extract feature maps. After performing a double upsampling operation and comparing with the feature map The images are stitched together and then upsampled twice before being output as denoised images.
[0079] In this embodiment, the dual convolutional activation layer consists of convolutional units, modified linear units, and convolutional units; the convolutional activation layer consists of convolutional units and modified linear units; and the convolutional normalized activation layer consists of convolutional units, normalized units, and modified linear units.
[0080] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0081] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.
Claims
1. A method for suppressing stray light noise in large field-of-view images based on deep learning, characterized by: This method achieves denoising by constructing a pyramid-shaped deformable convolutional large kernel attention mechanism model. The specific steps are as follows: Step 1: Downsampling operation; The original spatial target image under stray light interference is input into the upsampling layer of the pyramid-shaped deformable convolutional large kernel attention mechanism denoising model for two consecutive downsampling operations to obtain the downsampled sub-image. ; The specific process of performing two downsampling operations in step one is as follows: Step 11: First double downsampling, for scale Space target image under the original conditions of stray light interference Perform a 2x downsampling to obtain the scale Four downsampled sub-images The size is This can be expressed as a formula: ； In the formula, c is the number of channels in the image, h is the height of the image, and w is the width of the image. This indicates a 2x downsampling operation with four downsampled sub-images. The size is reduced to the original spatial target image under conditions of stray light interference. 1 / 4; Step 22: Second downsampling, for scale Four downsampled sub-images All samples were downsampled by a factor of two to obtain the scale. 16 downsampled sub-images The size is This can be expressed as a formula: ； In the formula, there are 16 downsampled sub-images. The size is reduced to the original spatial target image under conditions of stray light interference. 1 / 16; Step 2: Feature extraction; The downsampled sub-image obtained in step one Perform feature extraction to obtain scale Feature map below ; for downsampled sub-images Perform feature extraction to obtain scale Feature map below ; Step 3: Upsampling operation; For feature maps A double upsampling operation was performed, and the feature map was compared with the data. The images are stitched together, convolutional, and then upsampled twice to obtain the denoised output image. .
2. The method for suppressing stray light noise in large field-of-view images based on deep learning according to claim 1, characterized in that: In step two, a feature extraction module is used. and feature extraction module Implement downsampling of sub-images and downsampled sub-images Feature extraction; The feature extraction module It consists of dual convolutional activation layers; The feature extraction module Downsampled sub-images After processing by n convolutional activation layers, followed by m convolutional normalization activation layers, and finally by deformable convolutional large kernel attention mechanism layers, the feature extraction of the disturbed region is enhanced.
3. The method for suppressing stray light noise in large field-of-view images based on deep learning according to claim 2, characterized in that: The specific process of step two is as follows: Feature extraction module downsampled sub-images Perform feature extraction to obtain scale Feature map below The size is ; Expressed as a formula: ； Feature extraction module downsampled sub-images Perform feature extraction to obtain scale Feature map below The size is , Expressed as a formula: 。 4. The method for suppressing stray light noise in large field-of-view images based on deep learning according to claim 1, characterized in that: In step three, the feature map is processed. A double upsampling operation was performed, and the feature map was compared with the data. The images are stitched together, passed through a Conv layer, and finally upsampled twice to obtain the denoised output image. ; expressed as a formula: ； In the formula, C represents the splicing operation. This indicates a double upsampling operation. The image after denoising has a size of [size missing]. .
5. The method for suppressing stray light noise in large field-of-view images based on deep learning according to claim 1, characterized in that: The pyramid-shaped deformable convolutional large kernel attention mechanism denoising model includes a downsampling layer, a feature extraction layer, and an upsampling layer; The downsampling layer is used to perform two double downsampling operations on the original spatial target image under stray light interference conditions to obtain a downsampled sub-image; The feature extraction layer is implemented through a feature extraction module. and feature extraction module composition; The feature extraction module Downsampled sub-images After extraction using a double convolutional activation layer, the feature map is extracted. ; The feature extraction module Downsampled sub-images After processing through n convolutional activation layers, followed by m convolutional normalization activation layers, and finally a deformable convolutional large kernel attention mechanism layer, the feature map is extracted. ; The upsampling layer will extract feature maps. After performing a double upsampling operation and comparing with the feature map The images are stitched together and then upsampled twice before being output as denoised images.
6. The method for suppressing stray light noise in large field-of-view images based on deep learning according to claim 5, characterized in that: The dual convolutional activation layer consists of convolutional units, modified linear units, and convolutional units; the convolutional activation layer consists of convolutional units and modified linear units; the convolutional normalized activation layer consists of convolutional units, normalized units, and modified linear units.