Dynamic scatter imaging method based on gaussian mixture large kernel convolution and feature fusion
By employing Gaussian mixture large kernel convolution and feature fusion, the problems of insufficient receptive field coverage and inadequate multi-scale feature fusion were solved, achieving high-quality imaging in dynamic scattering media.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING JIAOTONG UNIV
- Filing Date
- 2026-02-26
- Publication Date
- 2026-06-19
Smart Images
Figure CN122244189A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of optical imaging technology, specifically to a dynamic scattering medium imaging method based on Gaussian mixture large kernel convolution and multi-scale feature fusion. Background Technology
[0002] When light propagates in dynamic scattering media (such as fog, clouds, and smoke), randomly distributed scattering particles severely disrupt its propagation path. As the thickness of the medium or the scattering intensity increases, the spatial information of light tends to become chaotic, which greatly limits the depth and quality of optical imaging, posing a significant challenge to the application of traditional imaging technologies in fields such as remote sensing, biomedical diagnostics, military, and underwater exploration.
[0003] In the early stages of research, scattered photons were considered noise in the imaging process. Mainstream methods focused on extracting ballistic photons that remained unaffected by scattering and propagated in a straight line, as well as serpentine photons that underwent slight forward scattering and had slightly tortuous paths. To this end, researchers developed various "gating" techniques, such as time gates and coherence gates, to filter out scattered light. However, ballistic photons attenuate exponentially with increasing distance during propagation, which greatly limits the depth and application range of ballistic photon imaging. Subsequent studies found that even scattered photons that have undergone multiple scatterings still retain important information about the target. Therefore, a series of new methods for optical imaging using scattered light have been proposed, mainly including wavefront shaping, transfer matrix measurement, speckle deconvolution, and speckle autocorrelation imaging.
[0004] With the continuous growth of computing resources and data scale, deep learning, with its powerful data-driven learning paradigm, has provided a promising new approach to overcome the bottlenecks of traditional scattering imaging techniques. Researchers have successively proposed a variety of sophisticated network architectures. Li et al. proposed a one-to-many CNN framework, which achieved generalized recovery of hidden targets in the same medium that had never been seen before through training with various frosted glasses; Sun et al. designed a dual-network structure that integrates a classification network and a generative adversarial network, achieving adaptive recovery of scattering conditions for fat emulsion solutions of different concentrations; in 2023, Tang et al. proposed a hybrid framework DeepSCI that integrates memory effects and deep learning to achieve scalable speckle correlation imaging (SCI). This method showed strong robustness under different scattering media and domain shift conditions, but its performance... While these advancements come at the cost of computational efficiency, and the imaging field of view is limited by the memory effect, Hu et al. constructed a two-layer dynamic scattering model using frosted glass and polystyrene microsphere suspensions and proposed a Pix2pix network scheme based on the PSNR (Peak Signal-to-Noise Ratio) loss function. In the same year, Zhang et al. simulated the characteristics of dynamic scattering media using experimental methods such as water-immersed onion skins, proposing a high-throughput imaging method based on speckle deblurring. In 2025, Farea et al. proposed the ScatResUNet model, which effectively alleviated the gradient vanishing problem by integrating residual blocks into the U-Net architecture, enabling better extraction of complex spatial features. These studies have driven the development of scattering medium imaging technology, but their architectural design is limited by the local receptive field of small convolutional kernels, and their application scenarios are mainly for macroscopically homogeneous or at least quasi-static scattering media during acquisition, making it difficult to adapt to the needs of strongly dynamic scattering scenarios.
[0005] In recent years, the Transformer architecture based on a "non-local" attention mechanism has gradually emerged in the field of computer vision. Unlike traditional convolutional operations, which are limited by local receptive fields, Transformers can effectively model long-range dependencies through global or large-window attention mechanisms, thereby obtaining a larger receptive field. The SACNN model designed by Wang et al., by fusing CNN and self-attention mechanisms, can achieve high-quality reconstruction of sparse targets in unseen similar scattering media, significantly outperforming traditional CNNs on multiple evaluation metrics and demonstrating stronger generalization and stability. The OPT (Optronic Transformer) architecture constructed by Huang et al., utilizing a spatial light modulator (SLM) and an optical microlens array shifting system, implements a self-attention mechanism in the optical domain, which can efficiently capture global features of speckle patterns. Although these methods have the advantage of a large global receptive field, they are still mainly aimed at macroscopically homogeneous or at least quasi-static scattering media during acquisition, and their applicability under strongly time-varying scattering conditions is still limited. It is worth noting that related research shows that the self-attention mechanism module may not be the core factor for the excellent performance of VisionTransformers (ViTs).
[0006] Inspired by this, some research has begun to explore the introduction of large convolutional kernel structures into CNNs to achieve global receptive capabilities similar to Transformers. SLaK uses a 51×51 convolutional kernel; the large kernel convolutional dehazing module (LKDBlock) designed by Luo et al. significantly improves the dehazing effect while effectively expanding the receptive field and capturing long-range dependencies, and its performance significantly surpasses the Dehamer model based on Transformer; UniRepLKNet uses a 13×13 large kernel convolution by default, and research has confirmed that expanding the convolutional kernel to 31×31 or even 57×57 has significant performance advantages and efficiency feasibility. However, existing research on networks containing large convolutional kernels mainly focuses on visual tasks such as natural image restoration, and its application in the field of scattering imaging is still rarely explored, especially in complex dynamic scattering media. How to use large convolutional kernel structures to improve global modeling capabilities and reconstruction accuracy still lacks systematic research.
[0007] Therefore, although deep learning has made significant progress in the field of scattering imaging, existing technologies still have problems such as insufficient receptive field coverage, inadequate multi-scale feature fusion, and limited detail recovery capabilities when facing complex dynamic scattering media, making it difficult to achieve high-quality clear imaging. There is still considerable room for optimization in related technologies. Summary of the Invention
[0008] To address the aforementioned technical problems, this application discloses a dynamic scattering imaging method based on Gaussian mixture large kernel convolution and feature fusion, specifically including:
[0009] Acquire speckle images corresponding to a dynamic scattering medium, wherein the speckle images include original-scale speckle images and reduced-resolution speckle images;
[0010] The original scaled speckle image and the reduced-resolution speckle image are input into a preset GMLK-MSRNet network. The encoder of the GMLK-MSRNet network extracts multi-scale features. The encoder integrates a Gaussian Mixture Large Kernel Convolution (GMLKC) module and a residual connection structure. The Gaussian Mixture Large Kernel Convolution (GMLKC) module adaptively adjusts the spatial response range of the convolution weights by mixing Gaussian distribution functions with different variances.
[0011] The decoder of the GMLK-MSRNet network, combined with cross-scale skip connections, enables multi-scale feature fusion to generate high-resolution feature representations.
[0012] The GMLK-MSRNet network parameters are optimized based on a composite loss function in the spatial and frequency domains. This composite loss function includes spatial domain loss. Loss and Frequency Domain Based on Fast Fourier Transform loss;
[0013] The optimized GMLK-MSRNet network is used to output clear imaging results of the dynamic scattering medium.
[0014] Preferably, the convolution kernel of the Gaussian mixture large kernel convolution module is based on the isotropic Gaussian mixture model and is constructed by weighted combination of multiple Gaussian components. The discretization and normalization calculation formulas of the convolution kernel are as follows: ,in, Let be the set of two-dimensional coordinate grids for the convolution kernel. The grid coordinates are defined in a center-aligned manner, and the formula is: , The kernel size; For the first The mixture weights of the Gaussian components are given by the formula: ; For the first The standard deviation of each Gaussian component; To prevent stable terms with a denominator of zero; and These correspond to the horizontal and vertical positions of the convolution kernel, respectively. Due to the isotropic nature of the Gaussian distribution... .
[0015] Preferably, the standard deviation of the Gaussian component With mixed weights All are learnable parameters, among which the standard deviation The training process is ensured by a nonlinear mapping mechanism to guarantee positive values and effective gradient propagation. The specific implementation process is as follows:
[0016] Use the inverse function of the Softplus function to initialize the hyperparameters. Initialize using the following formula:
[0017]
[0018] Using the Softplus function By performing a nonlinear mapping, the final standard deviation is obtained:
[0019]
[0020] in, , , A stable term to prevent numerical underflow; mixed weights The contribution ratio of each Gaussian component to the convolution kernel is adjusted by adaptively updating the kernel during the training process.
[0021] Preferably, the convolution kernel adopts an odd-sized design, and the translational covariance of the convolution operation is achieved through a symmetrical padding strategy. The padding size formula is as follows: , The kernel size is set to ensure that the output feature map after convolution is the same size as the input feature map. The Gaussian mixture large kernel convolution module is configured with at least two different sizes of convolution kernels, which are adapted to the original scale speckle image and the scaled-down speckle image respectively. The size of the convolution kernel is in a preset ratio with the size of the corresponding input image, so as to cover global feature association under the premise of lightweight modeling.
[0022] Preferably, the feature extraction process of the encoder follows a multi-scale progressive principle, specifically including:
[0023] The input raw scale speckle image is subjected to feature mapping through a 3×3 convolution to obtain a feature map with dimension . The initial feature representation, where For the number of channels, Spatial resolution;
[0024] By performing downsampling operations with 3×3 convolutions with a stride of 2, the spatial resolution is reduced while the number of channels is increased, generating multi-scale feature layers.
[0025] The reduced-resolution speckle image is input into an independent convolutional block. Spatial feature extraction and channel feature recombination are achieved through alternating stacking of 3×3 convolution and 1×1 convolution. The output features are fused with the downsampled features of the corresponding level in the channel dimension, and then integrated by 3×3 convolution to maintain a stable number of channels, forming enhanced multi-scale features.
[0026] Preferably, the residual connection structure in the encoder consists of multiple residual blocks, each containing two cascaded 3×3 convolutional layers, with a GELU nonlinear activation function embedded between the two convolutional layers. The residual block propagates the input features directly to the output through a shortcut branch, and performs element-wise superposition with the features processed by the convolutional layers to alleviate the gradient vanishing problem in deep network training.
[0027] Preferably, the feature recovery process of the decoder and the feature extraction process of the encoder have a symmetrical structure, specifically including:
[0028] Upsampling is performed by transposed convolution to gradually enlarge the spatial dimension of the feature map and reduce the number of channels. The kernel size and stride parameters of the transposed convolution are adapted to the feature recovery requirements to ensure a smooth transition of feature scale.
[0029] Establish cross-scale skip connections to stitch together the features of each level of the decoder with the enhanced multi-scale features of the encoder at the corresponding scale, providing underlying information support for image detail restoration;
[0030] The concatenated features are compressed through 1×1 convolution to reduce computational overhead and enhance feature fusion effect;
[0031] Finally, a residual image is generated through 3×3 convolution. The residual image is then superimposed pixel-wise with the output of the original image processed by Gaussian mixture large kernel convolution to obtain a high-resolution feature representation.
[0032] Preferably, the spatial domain L1 loss is used to constrain the consistency between the predicted image and the real image in the pixel space, and the calculation formula is as follows: ,in, Represents predicted images at different scales. This represents the true image at the corresponding scale. element-wise The norm, by minimizing this loss, makes the predicted image approximate the real image at the pixel level.
[0033] Preferably, the frequency domain L1 loss captures the spectral features of the image through Fast Fourier Transform, compensating for the shortcomings of spatial domain loss in modeling high-frequency details. The calculation formula is as follows: ,in, For Fast Fourier Transform operations, and These represent the real and imaginary parts after the transformation, respectively. Denotes the element-wise L1 norm. This represents a joint feature formed by stacking the real and imaginary parts along a new dimension; and Corresponding to predicted and real images at different scales, this loss is minimized to enhance the ability to restore image texture information and preserve high-frequency details.
[0034] Preferably, the composite loss function is constructed by weighted combination of spatial domain L1 loss and frequency domain L1 loss, taking into account both the optimization of the overall image structure and local details. The calculation formula is as follows:
[0035]
[0036] in, This represents the total number of samples in a single training session. The loss balancing weights are used to adjust the contribution ratio of spatial domain loss to frequency domain loss, so that the network can pay attention to both low-frequency overall structure and high-frequency texture details during training, thereby improving the overall performance of image quality.
[0037] Compared with the prior art, the technical solution of this application has the following technical effects:
[0038] This invention achieves adaptive adjustment of the spatial response range of convolution weights through an innovative Gaussian mixture large-kernel convolution module. Based on the isotropic Gaussian mixture model, this module constructs the convolution kernel through a weighted combination of multiple Gaussian components, overcoming the local receptive field limitation of traditional fixed small convolution kernels. Its odd-size design, coupled with a symmetrical padding strategy, ensures translational covariance of the convolution operation, enabling efficient capture of global structural information in speckle images without significantly increasing the number of parameters, laying a core foundation for imaging tasks in dynamic scattering media.
[0039] The residual connection structure integrated in the encoder of this invention effectively solves the gradient vanishing problem in deep network training. The structure consists of multiple residual blocks connected in series. Each residual block directly propagates the input features through a shortcut branch, and then performs element-wise superposition with the features processed by the convolutional layer. Combined with a multi-scale progressive feature extraction strategy, the encoder can perform hierarchical feature mapping and downsampling on speckle images at both the original scale and reduced resolution, generating enhanced multi-scale features and significantly improving the model's ability to represent scattering information at different scales.
[0040] The decoder of this invention adopts a symmetrical architecture design with the encoder. It performs upsampling of feature maps through transposed convolution, gradually restoring the spatial resolution of the image. The establishment of cross-scale skip connections enables efficient stitching of features at each level of the decoder with corresponding scale features of the encoder, providing sufficient low-level information support for image detail restoration. The stitched features are compressed by a 1×1 convolution channel, which reduces computational overhead while enhancing the feature fusion effect. Finally, a high-resolution and clear imaging result is generated by pixel-level superposition of the residual image and the features of the original image.
[0041] The construction of the spatial and frequency domain composite loss function in this invention achieves dual constraints and optimizations on imaging quality. (Spatial domain...) The loss function ensures the consistency between the predicted image and the real image at the pixel level, and in the frequency domain... The loss function captures spectral features through Fast Fourier Transform, compensating for the spatial domain loss's inadequacy in modeling high-frequency details. This weighted combination allows the network to consider both low-frequency overall structure and high-frequency texture details during training, significantly enhancing its ability to recover image texture information and effectively improving the overall quality of imaging in dynamic scattering media.
[0042] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.
[0043] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description
[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.
[0045] Based on the description of the figures and their corresponding technical content in the document, the titles of the figures are as follows:
[0046] Figure 1 : Core flowchart of GMLK-MSRNet network based on Gaussian mixture large kernel convolution and multi-scale feature fusion;
[0047] Figure 2 : Schematic diagram of the overall architecture of GMLK-MSRNet network and the structure details of each functional module;
[0048] Figure 3 A dynamic scattering medium speckle image acquisition system based on a fog chamber;
[0049] Figure 4 : Trend chart of GMLK-MSRNet network training and validation set loss values as a function of training rounds;
[0050] Figure 5 : 127×127 Gaussian mixed large kernel convolution kernel and Parameter training process change curve;
[0051] Figure 6 Comparison of speckle image of dynamic scattering medium, GMLK-MSRNet reconstructed image and original target image;
[0052] Figure 7 Histograms showing the distribution of PSNR, SSIM, and PCC in images reconstructed by the GMLK-MSRNet network. Detailed Implementation
[0053] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.
[0054] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.
[0055] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.
[0056] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.
[0057] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.
[0058] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.
[0059] Example 1
[0060] This embodiment mainly describes a dynamic scattering medium imaging method based on Gaussian mixture large kernel convolution and multi-scale feature fusion, such as... Figure 1 As shown, it specifically includes:
[0061] Acquire speckle images corresponding to a dynamic scattering medium, wherein the speckle images include original-scale speckle images and reduced-resolution speckle images;
[0062] The original scaled speckle image and the reduced-resolution speckle image are input into a preset GMLK-MSRNet network. The encoder of the GMLK-MSRNet network extracts multi-scale features. The encoder integrates a Gaussian Mixture Large Kernel Convolution (GMLKC) module and a residual connection structure. The Gaussian Mixture Large Kernel Convolution (GMLKC) module adaptively adjusts the spatial response range of the convolution weights by mixing Gaussian distribution functions with different variances.
[0063] The decoder of the GMLK-MSRNet network, combined with cross-scale skip connections, enables multi-scale feature fusion to generate high-resolution feature representations.
[0064] The GMLK-MSRNet network parameters are optimized based on a composite loss function in the spatial and frequency domains. This composite loss function includes spatial domain loss. Loss and Frequency Domain Based on Fast Fourier Transform loss;
[0065] The optimized GMLK-MSRNet network is used to output clear imaging results of the dynamic scattering medium.
[0066] Furthermore, a speckle image corresponding to the dynamic scattering medium is acquired. This speckle image includes both a raw-scale speckle image and a reduced-resolution speckle image. The raw-scale speckle image has a resolution set to 64×64 and is acquired using a dynamic scattering medium speckle image acquisition system. This system includes a laser generator, a beam expander, a digital micromirror device (DMD), a fog chamber, a 360-degree rotating fan, a charge-coupled device (CCD), and a terminal PC. The laser generator emits a 532nm laser beam, which is expanded by the beam expander before illuminating the DMD. The digital micromirror device (DMD) has a pixel count of 1920×1080 and a pixel pitch of 7.56µm. The DMD is used to load and project a raw target image selected from the MNIST dataset. The raw target image is projected into a fog chamber filled with water mist. A 360-degree rotating fan in the fog chamber is used to continuously change the distribution of water mist to simulate a real dynamic scattering environment. The speckle image after passing through the dynamic scattering medium is acquired by a charge-coupled device (CCD). The acquired speckle image is transmitted to a terminal PC via a data transmission line for storage and serves as input data for subsequent network models.
[0067] Furthermore, the original scaled speckle image and the reduced-resolution speckle image are input into a pre-defined GMLK-MSRNet network. The encoder of the GMLK-MSRNet network extracts multi-scale features. The encoder integrates a Gaussian Mixture Large Kernel Convolutional (GMLKC) module with a residual connection structure. The GMLKC module adaptively adjusts the spatial response range of the convolution weights by mixing Gaussian distribution functions with different variances. The GMLK-MSRNet network structure is as follows: Figure 2 As shown in (a), the overall architecture adopts a U-shaped network framework, integrating a Gaussian mixture large kernel convolution module and a residual connection structure. A multi-input multi-output strategy is used for speckle image reconstruction. The Gaussian mixture large kernel convolution module is configured with two different sizes of odd-numbered convolution kernels: 127×127 and 63×63. The 127×127 kernel is adapted to the original 64×64 speckle image, while the 63×63 kernel is adapted to the reduced-resolution 32×32 speckle image. The kernel sizes are twice the size of the corresponding input image, thus covering the global feature associations of the speckle image while maintaining lightweight modeling. Figure 2 As shown in (d), the convolution kernel can adaptively adjust the spatial response range of the convolution weights by mixing Gaussian distribution functions with different variances, thereby effectively expanding the receptive field of feature extraction.
[0068] Furthermore, the Gaussian mixture large kernel convolution module is specifically as follows:
[0069] The convolution kernel of the Gaussian mixture large kernel convolution module is based on the isotropic Gaussian mixture model and is constructed by weighted combination of multiple Gaussian components. The probability density function of a single isotropic Gaussian component is:
[0070]
[0071] in, Let be the index of the Gaussian component, with a value of . , The total number of Gaussian components. For the first The standard deviation of each Gaussian component is used to control the degree of diffusion of that Gaussian component in the spatial domain;
[0072] The weighted combination of multiple Gaussian components yields the basic distribution of the convolution kernel as follows: in, For the first The mixed weights of Gaussian components satisfy the constraints. This ensures that the total weight of each Gaussian component is 1. By adjusting the value of the mixed weights, the contribution of different Gaussian components to the convolution kernel can be controlled.
[0073] To map a continuous Gaussian mixture distribution to a discrete convolution kernel space, a two-dimensional coordinate grid set for the convolution kernel is defined. Grid coordinates are defined using center alignment, and the formula is: ,in, The value is the kernel size, and it should be an odd number to ensure that the kernel has a clear center position. and These correspond to the horizontal and vertical positions of the convolution kernel, respectively. Due to the isotropic nature of the Gaussian distribution... This coordinate grid covers all pixel locations of the convolution kernel;
[0074] Based on coordinate grid set The formulas for discretizing and normalizing the convolution kernel are as follows: in, To prevent stable terms with a denominator of zero, and to avoid numerical overflow during calculation; For coordinate grid set For any coordinate point within the range, the discrete convolution kernel weights calculated using this formula are normalized to ensure the rationality and effectiveness of the weight values.
[0075] The convolutional kernel employs an odd-sized design and utilizes a symmetrical padding strategy to achieve translational covariance in the convolution operation. The padding size formula is as follows: , The kernel size is [size]. Indicates to Rounding down ensures that the size of the output feature map after convolution is exactly the same as the size of the input feature map, avoiding the shrinkage of the feature map size due to convolution. The Gaussian mixture large kernel convolution module is configured with two types of convolution kernels, 127×127 and 63×63, both of which follow this odd-number size design and symmetrical padding strategy. They are adapted to the original scale speckle image and the scaled-down speckle image, respectively. The size of the convolution kernel is proportional to the size of the corresponding input image, so as to achieve effective capture of global features of speckle image without significantly increasing the number of network parameters.
[0076] The standard deviation of the Gaussian component With mixed weights All parameters are learnable and can adaptively adjust their values based on the input speckle image features during network training; the hybrid weights The algorithm adaptively updates itself during training to adjust the contribution ratio of each Gaussian component to the convolution kernel, while always satisfying the constraints. The standard deviation A nonlinear mapping mechanism is used to ensure positive values and effective gradient propagation during training. Specifically, the inverse of the Softplus function is used to map the initial hyperparameters. Initialize using the following formula: ,in Ensure that the initialized parameter values are within a reasonable range; use the Softplus function to... By performing a nonlinear mapping, the final standard deviation is obtained, as shown in the formula. ,in , To prevent numerical underflow of stable terms, this nonlinear mapping mechanism ensures the standard deviation. It is always positive, which ensures that the gradient can be effectively propagated during backpropagation, thereby improving the stability and convergence speed of network training.
[0077] Furthermore, the feature extraction process of the encoder follows a multi-scale progressive principle, specifically including: performing feature mapping on the input original 64×64 speckle image through a 3×3 convolution, with the padding size of the 3×3 convolution set to 1 to ensure that the size of the output feature map remains unchanged at 64×64 after feature mapping, resulting in a dimension of... The initial feature representation, where For the number of channels, For spatial resolution, and These correspond to the height and width of the feature map, respectively. Downsampling is performed using a 3×3 convolution with a stride of 2. The stride of 2 reduces the spatial resolution of the feature map to half its original value after each downsampling, while simultaneously doubling the number of channels. This generates a multi-scale feature hierarchy, containing feature maps at three scales: the original scale, half the original scale's dimensions, and one-quarter of the original scale's dimensions. The reduced-resolution 32×32 speckle image is then input into an independent convolutional block, which is as follows: Figure 2 As shown in (c), spatial feature extraction and channel feature recombination are achieved by alternating stacking of 3×3 convolution and 1×1 convolution. The 3×3 convolution is used to extract the spatial local features of the speckle image, and the 1×1 convolution is used to transform and reorganize the extracted features in the channel dimension. The output features of this independent convolution block are fused with the downsampled features of the corresponding level in the channel dimension. The fused features are then integrated by 3×3 convolution to maintain the stability of the number of channels, forming enhanced multi-scale features. The enhanced multi-scale features contain feature information of both the original scale speckle image and the resolution-reduced speckle image, which improves the model's ability to represent speckle information at different scales.
[0078] The residual connection structure in the encoder consists of multiple residual blocks, such as Figure 2 As shown in (b), each residual block contains two cascaded 3×3 convolutional layers, with a GELU nonlinear activation function embedded between the two convolutional layers. The GELU nonlinear activation function can effectively improve the nonlinear expression capability of the network and alleviate the gradient vanishing problem. The residual block directly propagates the input features to the output end through the shortcut branch, and performs element-wise superposition with the features processed by the convolutional layer. The element-wise superposition operation can fuse the input features with the features extracted by the convolutional layer, thereby alleviating the gradient vanishing problem in deep network training, enabling the network to maintain stable training in a deeper structure, and improving the effectiveness and accuracy of feature extraction.
[0079] Furthermore, the decoder of the GMLK-MSRNet network, combined with cross-scale skip connections, achieves multi-scale feature fusion to generate high-resolution feature representations; the feature recovery process of the decoder and the feature extraction process of the encoder have a symmetrical structure, such as... Figure 2As shown in (a), the decoder also contains three feature processing layers at three scales, corresponding to the original scale, half the length and width of the original scale, and one-quarter the length and width of the original scale enhanced multi-scale features generated by the encoder, respectively. Specifically, it includes: performing upsampling operations through transposed convolution, where the kernel size of the transposed convolution is set to 4 and the stride is set to 2. This parameter setting enables the spatial dimension of the feature map to be magnified to twice its original size after each upsampling, while reducing the number of channels of the feature map to half its original size. This gradually magnifies the spatial dimension of the feature map and reduces the number of channels, ensuring a smooth transition of feature scale and gradually restoring the original spatial resolution of the speckle image; establishing cross-scale skip connections, concatenating the features of each level of the decoder with the enhanced multi-scale features of the encoder at the corresponding scale, with the concatenation operation performed in the channel dimension. This method can fuse the low-level detail features extracted by the encoder with the high-level semantic features generated by the decoder, providing sufficient low-level information support for image detail restoration. The concatenated features are compressed by 1×1 convolution, reducing the number of channels to half of the original, thereby reducing computational overhead and enhancing the feature fusion effect, resulting in better representational capabilities of the fused features. Finally, a residual image is generated by 3×3 convolution, and the residual image is pixel-level superimposed with the output of the original image processed by Gaussian mixture kernel convolution. The pixel-level superposition operation can fuse the detail information of the residual image with the global feature information extracted by Gaussian mixture kernel convolution, resulting in a high-resolution feature representation. The size of this high-resolution feature representation is consistent with the size of the original input speckle image, laying the foundation for the subsequent generation of a clear speckle reconstruction image.
[0080] Furthermore, a composite loss function is used to optimize the network parameters. The GMLK-MSRNet network parameters are optimized based on a composite loss function in both the spatial and frequency domains. This composite loss function includes spatial domain... Loss and Frequency Domain Based on Fast Fourier Transform loss;
[0081] The spatial domain The loss is used to constrain the consistency between the predicted image and the real image in the pixel space, and the calculation formula is as follows:
[0082] ,in, This represents the predicted image at different scales, including the original scale, half the length and width of the original scale, and one-quarter the length and width of the original scale. This represents the true image at the corresponding scale. element-wise The norm is the sum of the absolute values of the differences between the pixel values of the predicted image and the real image. By minimizing this loss function, the predicted image can approximate the real image at the pixel level, ensuring the overall structural accuracy of the reconstructed image.
[0083] The frequency domain The loss function captures the spectral features of the image through Fast Fourier Transform, compensating for the shortcomings of spatial domain loss in modeling high-frequency details. The calculation formula is as follows: ,in, For Fast Fourier Transform operations, and These represent the real and imaginary parts after the transformation, respectively. This represents a joint feature formed by stacking the real and imaginary parts along a new dimension; and These correspond to predicted and real images at different scales. element-wise Norms, by constraining the difference in spectral features between the predicted and real images, can enhance the ability to restore image texture information and preserve high-frequency details, thus giving the reconstructed image richer detail features.
[0084] The composite loss function is obtained through a weighted combination space domain. Loss and frequency domain The loss calculation, which takes into account both the overall image structure and local details, is as follows:
[0085]
[0086] in, This represents the total number of samples in a single training session. The loss balancing weight is set to 0.1, which is used to adjust the contribution ratio of spatial domain loss and frequency domain loss. This allows the network to focus on both low-frequency overall structure and high-frequency texture details during training. By minimizing this composite loss function, the parameters of the GMLK-MSRNet network are optimized and updated, improving the network's speckle image reconstruction capability and ultimately generating high-quality, clear imaging results.
[0087] The optimized GMLK-MSRNet network outputs clear imaging results in dynamic scattering media. The GMLK-MSRNet network optimized by the composite loss function is used as the final speckle image reconstruction model. The original scale speckle image is input into the optimized network model. After multi-scale feature extraction by the encoder and multi-scale feature fusion by the decoder, the network outputs a reconstructed image with the same size as the original target image. This reconstructed image is a clear imaging result under dynamic scattering media, which can effectively restore the details of the target image obscured by the dynamic scattering medium and achieve high-quality optical imaging in dynamic scattering medium environment.
[0088] This implementation details how to adaptively adjust the response range of convolution weights using a Gaussian mixture large kernel convolution module, combined with a multi-scale feature fusion mechanism, to effectively expand the receptive field and enhance feature representation capabilities. The spatial and frequency domain composite loss function takes into account both the overall image structure and high-frequency details, significantly improving the clarity and fidelity of imaging under dynamic scattering media.
[0089] Based on Example 1, this example details the verification of a dynamic scattering imaging method based on Gaussian mixture large kernel convolution and feature fusion. A high-precision dynamic scattering medium speckle image acquisition system was built, and a complete model training, verification, and testing process was designed. Imaging quality was quantitatively evaluated using multi-dimensional indicators. The experimental optical path is as follows: Figure 3 As shown, the system consists of five parts: a laser emission module, an image modulation module, a dynamic scattering simulation module, an image acquisition module, and a data storage module. The laser emission module uses a 532 nm solid-state laser (ADR-1805, Changchun Laser Photonics Technology Co., Ltd.). The laser beam is uniformly expanded by a beam expander and then projected onto a digital micromirror device (DMD, DLPF6500). This DMD has 1920×1080 pixels and a pixel pitch of 7.56 µm, enabling precise loading and reflection of the original target image selected from the MNIST dataset. The dynamic scattering simulation module is a custom-sized fog chamber. Inside the fog chamber, a fog generator continuously releases water mist, creating a stable scattering medium environment. A 360-degree rotating fan is installed inside the fog chamber; the continuous rotation of the fan disturbs the water mist distribution, thus simulating the dynamic scattering effect in a real scene. The image acquisition module uses a charge-coupled device (CCD, SP5000i). This CCD camera can capture speckle images after passing through the dynamic scattering medium. The data storage module is a high-performance terminal PC. The CCD camera transmits image data and control signals via two data transmission lines, enabling real-time storage and backup of speckle images. Based on the above experimental setup, this study collected 7033 pairs of matching data between speckle images and original target images. These were randomly divided into training, validation, and test sets in a 6:2:2 ratio. The training set contained 4219 image pairs, while the validation and test sets each contained 1407 image pairs, providing sufficient data support for model training and performance evaluation.
[0090] The model training is based on the PyTorch deep learning framework, using the Adam optimizer to iteratively update the network parameters. The initial learning rate is set to 0.001, and a cosine annealing strategy is used for learning rate decay. Training is performed using batches of 64 samples, with a preset training epoch duration of 50 epochs. The loss trends of the training and validation sets are shown below. Figure 4 As shown, Figure 4The graph is divided into two sub-graphs: the left half shows the training set loss curve, and the right half shows the validation set loss curve. The horizontal axis represents the number of training epochs, and the vertical axis represents the loss value. It is clear from the graph that as the number of training epochs increases, the training set loss almost continuously decreases, with the fastest decrease in the first 5 epochs, followed by a gradual slowdown and stabilization. The validation set loss also shows a decreasing trend in the first 5 epochs, reaching a local minimum of 0.1858 in the 11th epoch. Afterward, the validation set loss does not decrease for 10 consecutive epochs, reaching the preset early stopping tolerance number (patience=10). To avoid overfitting, training is terminated at this point, and the network parameters corresponding to the 11th epoch are taken as the optimal model parameters.
[0091] To explore the adaptive adjustment capability of the Gaussian mixture large kernel convolution module, this study monitored the key learning parameters of a 127×127 convolution kernel in real time, and their changing trends are as follows: Figure 5 As shown. Figure 5 It is divided into two subplots, with the upper part being the mixed weights. The curve shows that as the number of training rounds increases, the mixed weights of each Gaussian component exhibit a pattern of initial small fluctuations followed by stabilization. This indicates that the network can adaptively adjust the contribution ratio of different Gaussian components to the convolution kernel during training. Figure 5 The lower half represents the standard deviation. The numerical values and rate of change curves of the Gaussian components show that the standard deviation values of each Gaussian component change significantly in the early stages of training, with a high absolute value of the rate of change. As training progresses, the standard deviation values gradually stabilize, and the rate of change approaches 0. The dynamic changes of the above parameters intuitively reveal the reshaping process of the internal structure of the convolution kernel, proving that the module can adaptively adjust the spatial response range of the convolution weights according to the features of the input speckle image, providing strong support for the effective capture of global structural information.
[0092] The partial speckle image reconstruction results in the test set are as follows: Figure 6 As shown, the graph contains three rows of subgraphs, where Figure 6 The first line is a speckle image of the dynamic scattering medium. The image shows obvious blurring and noise, and the target details are completely obscured by the scattering effect. Figure 6 The second row is the target image reconstructed based on the GMLK-MSRNet network proposed in this study. Figure 6 The third row corresponds to the original target image. By comparison, it can be found that the reconstructed image is highly consistent with the original target image in terms of overall structure and detailed texture. Even the blurred edge areas in the speckle image can be clearly restored by the reconstruction result, which fully demonstrates the effectiveness of the proposed method in dynamic scattering medium imaging tasks.
[0093] To objectively and quantitatively evaluate the imaging quality of the model, this study selected three classic evaluation metrics: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Pearson Correlation Coefficient (PCC). These metrics were calculated for 1407 pairs of reconstructed images and original target images in the test set, and their statistical distribution is shown below. Figure 7 As shown, Figure 7 The graph contains three sub-graphs, corresponding to the distribution histograms of PSNR, SSIM, and PCC, respectively. The horizontal axis of each sub-graph represents the value of the corresponding evaluation metric, and the vertical axis represents the number of images falling within that value range. The color intensity indicates the concentration of data points; lighter colors (yellow-green) indicate a higher frequency of occurrence within that value range, while darker colors (dark purple / blue) indicate a lower frequency. The graphs show that the mean PSNR is 29.2010, with values mainly concentrated in the 29-33 range. The mean SSIM is 0.9202, with most values distributed between 0.86 and 0.95, and some values even exceeding 0.95. The mean PCC is 0.9026, with a relatively concentrated distribution, mainly in the 0.9-1.0 range. The high mean values and concentrated distribution ranges of the three evaluation metrics fully demonstrate that the proposed method possesses excellent imaging performance and good stability, enabling high-quality target image reconstruction in dynamic scattering media environments.
[0094] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter changes made to these embodiments within the spirit and principles of the present invention, without departing from the principles and spirit of the present invention, through conventional substitutions or to achieve the same function, fall within the scope of protection of the present invention.
Claims
1. A dynamic scattering imaging method based on Gaussian mixture large kernel convolution and feature fusion, characterized in that, include: Acquire speckle images corresponding to a dynamic scattering medium, wherein the speckle images include original-scale speckle images and reduced-resolution speckle images; The original scaled speckle image and the reduced-resolution speckle image are input into a preset GMLK-MSRNet network. The encoder of the GMLK-MSRNet network extracts multi-scale features. The encoder integrates a Gaussian Mixture Large Kernel Convolution (GMLKC) module and a residual connection structure. The Gaussian Mixture Large Kernel Convolution (GMLKC) module adaptively adjusts the spatial response range of the convolution weights by mixing Gaussian distribution functions with different variances. The decoder of the GMLK-MSRNet network, combined with cross-scale skip connections, enables multi-scale feature fusion to generate high-resolution feature representations. The GMLK-MSRNet network parameters are optimized based on a composite loss function in the spatial and frequency domains. This composite loss function includes spatial domain loss. Loss and Frequency Domain Based on Fast Fourier Transform loss; The optimized GMLK-MSRNet network is used to output clear imaging results of the dynamic scattering medium.
2. The dynamic scattering imaging method based on Gaussian mixture large kernel convolution and feature fusion according to claim 1, characterized in that, The convolution kernel of the Gaussian mixture large kernel convolution module is based on the isotropic Gaussian mixture model and is constructed by weighted combination of multiple Gaussian components. The discretization and normalization calculation formulas of the convolution kernel are as follows: ,in, Let be the set of two-dimensional coordinate grids for the convolution kernel. The grid coordinates are defined in a center-aligned manner, and the formula is: , The kernel size; For the first The mixture weights of the Gaussian components are given by the formula: ; For the first The standard deviation of each Gaussian component; To prevent stable terms with a denominator of zero; and These correspond to the horizontal and vertical positions of the convolution kernel, respectively. Due to the isotropic nature of the Gaussian distribution... .
3. The dynamic scattering imaging method based on Gaussian mixture large kernel convolution and feature fusion according to claim 2, characterized in that, The standard deviation of the Gaussian component With mixed weights All are learnable parameters, among which the standard deviation The training process is ensured by a nonlinear mapping mechanism to guarantee positive values and effective gradient propagation. The specific implementation process is as follows: Use the inverse function of the Softplus function to initialize the hyperparameters. Initialize using the following formula: Using the Softplus function By performing a nonlinear mapping, the final standard deviation is obtained: in, , , A stable term to prevent numerical underflow; mixed weights The contribution ratio of each Gaussian component to the convolution kernel is adjusted by adaptively updating the kernel during the training process.
4. The dynamic scattering imaging method based on Gaussian mixture large kernel convolution and feature fusion according to claim 2, characterized in that, The convolutional kernel employs an odd-sized design and utilizes a symmetrical padding strategy to achieve translational covariance in the convolution operation. The padding size formula is as follows: , The kernel size is set to ensure that the output feature map after convolution is the same size as the input feature map. The Gaussian mixture large kernel convolution module is configured with at least two different sizes of convolution kernels, which are adapted to the original scale speckle image and the scaled-down speckle image respectively. The size of the convolution kernel is in a preset ratio with the size of the corresponding input image, so as to cover global feature association under the premise of lightweight modeling.
5. The dynamic scattering medium imaging method based on Gaussian mixture large kernel convolution and multi-scale feature fusion according to claim 1, characterized in that, The feature extraction process of the encoder follows a multi-scale progressive principle, specifically including: The input raw scale speckle image is processed by... Convolution performs feature mapping, resulting in a dimension of The initial feature representation, where For the number of channels, Spatial resolution; By using a step size of 2 Convolution performs downsampling operations, which reduces spatial resolution while increasing the number of channels, generating multi-scale feature layers; The downscaled speckle image is input into independent convolutional blocks, and then... Convolution and 1 Alternating stacking of convolutions enables spatial feature extraction and channel feature recombination. The output features are fused with the downsampled features of the corresponding layer along the channel dimension, and then... Convolutional integration is used to maintain a stable number of channels, resulting in enhanced multi-scale features.
6. The dynamic scattering imaging method based on Gaussian mixture large kernel convolution and feature fusion according to claim 5, characterized in that, The residual connection structure in the encoder consists of multiple residual blocks. Each residual block contains two cascaded 3×3 convolutional layers, with a GELU nonlinear activation function embedded between the two convolutional layers. The residual block propagates the input features directly to the output through a shortcut branch, and performs element-wise superposition with the features processed by the convolutional layers to alleviate the gradient vanishing problem in deep network training.
7. The dynamic scattering imaging method based on Gaussian mixture large kernel convolution and feature fusion according to claim 1, characterized in that, The feature recovery process of the decoder has a symmetrical structure with the feature extraction process of the encoder, specifically including: Upsampling is performed by transposed convolution to gradually enlarge the spatial dimension of the feature map and reduce the number of channels. The kernel size and stride parameters of the transposed convolution are adapted to the feature recovery requirements to ensure a smooth transition of feature scale. Establish cross-scale skip connections to stitch together the features of each level of the decoder with the enhanced multi-scale features of the encoder at the corresponding scale, providing underlying information support for image detail restoration; The spliced features are obtained through 1 Convolution performs channel compression, reducing computational overhead and enhancing feature fusion performance; Finally passed Convolution generates a residual image, which is then pixel-wise superimposed with the output of the original image processed by Gaussian mixture large kernel convolution to obtain a high-resolution feature representation.
8. The dynamic scattering imaging method based on Gaussian mixture large kernel convolution and feature fusion according to claim 1, characterized in that, The spatial domain The loss is used to constrain the consistency between the predicted image and the real image in the pixel space, and the calculation formula is as follows: ,in, Represents predicted images at different scales. This represents the true image at the corresponding scale. element-wise The norm, by minimizing this loss, makes the predicted image approximate the real image at the pixel level.
9. The scattering medium imaging method based on Gaussian mixture large kernel convolution and feature fusion according to claim 1, characterized in that, The frequency domain The loss function captures the spectral features of the image through Fast Fourier Transform, compensating for the shortcomings of spatial domain loss in modeling high-frequency details. The calculation formula is as follows: ,in, For Fast Fourier Transform operation, and These represent the real and imaginary parts after the transformation, respectively. element-wise Norm, This represents a joint feature formed by stacking the real and imaginary parts along a new dimension; and Corresponding to predicted and real images at different scales, this loss is minimized to enhance the ability to restore image texture information and preserve high-frequency details.
10. The dynamic scattering imaging method based on Gaussian mixture large kernel convolution and feature fusion according to any one of claims 1, 8, and 9, characterized in that, The composite loss function is obtained through a weighted combination space domain. Loss and frequency domain The loss calculation, which takes into account both the overall image structure and local details, is as follows: in, This represents the total number of samples in a single training session. The loss balancing weights are used to adjust the contribution ratio of spatial domain loss and frequency domain loss, so that the network can pay attention to both low-frequency overall structure and high-frequency texture details during training, thereby improving the overall performance of image quality.