A deep learning-based laser speckle image enhancement method and device

Abstract

The application relates to a laser speckle image enhancement method and device based on deep learning, which simultaneously utilizes an original image and a second image subjected to logarithmic bilateral filtering, and extracts high and low frequency components thereof, thereby providing multi-source information for subsequent feature aggregation, and being capable of more comprehensively capturing image information and overcoming the limitation of a single image source; then, a basic model pre-trained based on laser speckle training samples is introduced to extract speckle features from the first image, and the generation of high-frequency features and low-frequency features is effectively guided through domain knowledge representing laser speckles, so that high-frequency guided features and low-frequency guided features are obtained; the knowledge guiding mechanism enables the model to more effectively distinguish effective textures and noises in the enhancement process, avoids blind noise reduction, and maximally retains and restores key texture details while suppressing speckle noise, thereby providing a more reliable and accurate image basis for subsequent medical diagnosis and quantitative analysis.

Description

Technical Field

[0001] This application relates to the field of laser speckle image enhancement technology, and in particular to a laser speckle image enhancement method and apparatus based on deep learning. Background Technology

[0002] Laser speckle imaging technology has been widely used in the biomedical field, enabling the monitoring of blood flow distribution and microvascular morphology in living tissues such as skin, cerebral cortex, and retina. This provides crucial visual and quantitative evidence for the early diagnosis of vascular diseases and intraoperative tissue perfusion assessment. Laser speckle images are formed by the interference of light through laser irradiation of rough biological tissue surfaces. However, during acquisition, they are susceptible to factors such as tissue scattering characteristics, optical system noise, and ambient light interference, inevitably resulting in multiplicative speckle noise. Furthermore, the overall image contrast is low, and effective texture details are highly coupled with noise, severely obscuring tissue physiological information. This has become a core bottleneck restricting the precise clinical application of the technology. Therefore, enhancement processing of laser speckle images has become a key research focus in this field.

[0003] Currently, laser speckle image enhancement methods are mainly divided into two categories: traditional image processing methods and deep learning-based methods. Traditional methods often transform multiplicative noise into additive noise through logarithmic transformation, and then combine bilateral filtering, Gaussian difference, wavelet decomposition, etc. to achieve noise reduction and detail enhancement. These methods are simple in principle and have low computational cost, but they are difficult to balance the suppression of speckle noise and the preservation of texture details, which can easily lead to blurred image edges and texture distortion. Deep learning-based methods rely on the feature learning capabilities of deep neural networks to learn the feature patterns of speckle images through training on a large number of samples, which significantly improves the enhancement effect. The mainstream solutions mostly adopt architectures such as U-Net and attention networks, or introduce multi-scale feature fusion strategies to optimize feature extraction effects, which has become the mainstream of research at this stage.

[0004] Existing laser speckle image enhancement techniques based on deep learning are mostly end-to-end feature learning, which does not fully explore the texture characteristics of speckle images. This can easily lead to problems such as excessive noise reduction resulting in loss of details or incomplete noise reduction leaving residual noise, making it difficult to meet the high-precision analysis needs of medical clinical practice. Summary of the Invention

[0005] The main objective of this disclosure is to propose a deep learning-based image enhancement method and apparatus for laser speckle, which can achieve refined enhancement of laser speckle images by decomposing and aggregating high and low frequencies of multi-source images and combining effective guidance from domain knowledge.

[0006] A first aspect of this application provides a deep learning-based image enhancement method for laser speckle, the method comprising: In response to the speckle image enhancement command, determine the first image. Second image The first image The first image is the original laser speckle image, and the second image is... It is the first image The image obtained after performing logarithmic bilateral filtering; Extract the first image respectively First high frequency component and the first low-frequency component and extracting the second image The second high frequency component Second low frequency component ; Aggregate the first high-frequency component and the second high-frequency component The corresponding features yield high-frequency features. and aggregating the first low-frequency component and the second low-frequency component The corresponding features yield low-frequency features. ; The base model, pre-trained based on laser speckle training samples, is derived from the first image. Extracting speckle features And based on the speckle characteristics For the high-frequency features Knowledge guidance was conducted to obtain high-frequency guidance characteristics. and based on the speckle characteristics Regarding the low-frequency features Knowledge guidance was conducted to obtain low-frequency guidance characteristics. ; Based on the high-frequency guiding features and the low-frequency guiding features The enhanced laser speckle image is reconstructed. .

[0007] This application provides a deep learning-based image enhancement method for laser speckle, which has at least the following beneficial effects: This embodiment considers the complementarity of speckle texture. By performing modal transformation on the original laser speckle image, the first image is sequentially logarithmically transformed and bilaterally filtered to obtain a noise-reduced texture-enhanced modal laser speckle image. The second image can compensate for the defects of the original image (such as high noise and blurred texture) while retaining the core speckle texture information, thus providing support for subsequent aggregation of high and low frequency features. The high and low frequency components of the two modal images are extracted respectively, providing multi-source information for subsequent feature aggregation. The introduction of this multi-source feature can capture image information more comprehensively and overcome the limitations of a single image source. Moreover, by introducing a basic model pre-trained based on laser speckle training samples, speckle features are extracted from the first image. The speckle features represent the domain knowledge of laser speckle and can effectively guide the generation of high-frequency and low-frequency features, resulting in high-frequency guided features and low-frequency guided features. This knowledge-guided mechanism enables the model to more intelligently distinguish between effective texture and noise during the enhancement process, avoiding blind noise reduction. Thus, while suppressing speckle noise, key texture details such as microvessels are preserved and restored to the maximum extent.

[0008] This embodiment achieves refined enhancement of laser speckle images by decomposing and aggregating high and low frequencies of multi-source images and combining effective guidance from domain knowledge. The method not only overcomes the problems of poor adaptability and insufficient detail preservation of traditional methods, but also makes up for the shortcomings of existing deep learning methods in insufficient speckle texture mining. The enhanced image obtained by the final reconstruction provides a more reliable and accurate image basis for subsequent medical diagnosis and quantitative analysis.

[0009] A second aspect of this application provides a deep learning-based laser speckle image enhancement device, the device comprising: The instruction response module is used to determine the first image in response to a speckle image enhancement instruction. Second image The first image The first image is the original laser speckle image, and the second image is... It is the first image The image obtained after performing logarithmic bilateral filtering; The high- and low-frequency component extraction module is used to extract the first image respectively. First high frequency component and the first low-frequency component and extracting the second image The second high frequency component Second low frequency component ; The high- and low-frequency feature extraction module is used to aggregate the first high-frequency component. and the second high-frequency component The corresponding features yield high-frequency features. and aggregating the first low-frequency component and the second low-frequency component The corresponding features yield low-frequency features. ; The feature knowledge guidance module is used to pre-train the base model based on laser speckle training samples from the first image. Extracting speckle features And based on the speckle characteristics For the high-frequency features Knowledge guidance was conducted to obtain high-frequency guidance characteristics. and based on the speckle characteristics Regarding the low-frequency features Knowledge guidance was conducted to obtain low-frequency guidance characteristics. ; The speckle image reconstruction module is used to reconstruct the image based on the high-frequency guiding features. and the low-frequency guiding features The enhanced laser speckle image is reconstructed. .

[0010] A third aspect of this application provides an electronic device including at least one controller and a memory for communicatively connecting to the controller; the memory stores instructions executable by the at least one controller, the instructions being executed by the at least one controller to cause the at least one controller to perform a deep learning-based laser speckle image enhancement method as described in the first aspect of this application.

[0011] A fourth aspect of this application provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform a deep learning-based laser speckle image enhancement method as described in the first aspect of this application.

[0012] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1This is a schematic flowchart of a laser speckle image enhancement method based on deep learning provided in an embodiment of this application; Figure 2 This is a schematic diagram of the main body model provided in the embodiments of this application; Figure 3 This is a schematic diagram illustrating the process of forming low-frequency guidance features according to an embodiment of this application; Figure 4 This is a schematic diagram of a laser speckle image enhancement device based on deep learning provided in an embodiment of this application; Figure 5 This is a schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0016] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0017] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed or function in a specific orientation, and therefore should not be construed as a limitation of this application.

[0018] like Figure 1 One embodiment of this application provides a deep learning-based image enhancement method for laser speckle, the method comprising: Step S110: In response to the speckle image enhancement command, determine the first image. Second image The first image The first image is the original laser speckle image; the second image is... It is the first image The image obtained after performing logarithmic bilateral filtering; Step S120: Extract the first image respectively First high frequency component and the first low-frequency component and extracting the second image The second high frequency component Second low frequency component ; Step S130: Aggregate the first high-frequency component Second high frequency component The corresponding features yield high-frequency features. and aggregate the first low-frequency component Second low frequency component The corresponding features yield low-frequency features. ; Step S140: The base model is pre-trained based on the laser speckle training samples from the first image. Extracting speckle features And based on the speckle characteristics For high frequency features Knowledge guidance was conducted to obtain high-frequency guidance characteristics. And based on speckle characteristics For low-frequency features Knowledge guidance was conducted to obtain low-frequency guidance characteristics. ; Step S150, based on high-frequency guidance features and low-frequency guidance characteristics The enhanced laser speckle image was reconstructed. .

[0019] The core of this application includes: dual-image high and low frequency extraction → feature aggregation → knowledge-guided → adaptive fusion. Specifically, it revolves around the core problem of multiplicative noise and texture-noise coupling in laser speckle images. First, unlike ordinary laser images, the multiplicative noise of the speckle is transformed into additive noise, laying the foundation for subsequent bilateral filtering denoising to enhance the image. This also allows the high-frequency components (pure texture) of the second image and the high-frequency components (texture + noise) of the first image to form precise complementarity. Then, by extracting their high and low frequency components separately, multi-source information is provided for subsequent feature aggregation, enabling a more comprehensive capture of image information and overcoming the limitations of a single image source. Finally, by introducing a basic model pre-trained based on laser speckle training samples, speckle features are extracted from the first image. The speckle feature represents domain knowledge of laser speckle and can effectively guide the generation of high-frequency and low-frequency features, resulting in high-frequency guided features and low-frequency guided features. This enables the model to more intelligently distinguish effective texture and noise during the enhancement process, suppressing speckle noise while preserving and restoring key texture details such as microvessels to the maximum extent. Finally, the high-frequency guided features and low-frequency guided features are adaptively fused, ensuring that high-frequency information can be fully utilized for enhancement in areas with rich image details, while low-frequency information is relied more on to suppress noise in smooth areas. This avoids the problem of detail loss or noise amplification that may be caused by traditional fixed-weight fusion, ultimately providing a more reliable and accurate image basis for subsequent medical diagnosis and quantitative analysis.

[0020] This method first responds to the speckle image enhancement command and determines the first image. Second image The first image The first image is the original laser speckle image; the second image is... It is the first image The image obtained after logarithmic bilateral filtering. Specifically, a user interface can provide a button or menu option. When the user clicks the button or selects an option, the system receives a speckle image enhancement instruction. Subsequently, the system can load the original laser speckle image from a preset storage path as the first image. For the second image The system can call a log-bilateral filtering function from an image processing library to process the first image. As input, the second image is generated after processing. .

[0021] As another implementation, the system can be configured to automatically monitor specific folders. When a new laser speckle image file is placed in the folder, the system automatically recognizes the event as a speckle image enhancement command, and the new file is then identified as the first image. Next, the system can use a custom logarithmic bilateral filtering algorithm to process the first image. Perform pixel-by-pixel processing to generate a second image. .

[0022] In step S120, the method extracts the first image respectively. First high frequency component and the first low-frequency component and extracting the second image The second high frequency component Second low frequency component High-frequency components can be extracted using frequency domain analysis methods such as Fourier transform or wavelet transform. For example, the image can be converted to the frequency domain, and high-frequency information can be separated by designing appropriate filters. Then, the high-frequency components can be obtained through inverse transform. Low-frequency components can be obtained by subtracting the high-frequency components from the original image, or by extracting them directly from the frequency domain using a low-pass filter.

[0023] In step S130, the method aggregates the first high-frequency component. Second high frequency component The corresponding features yield high-frequency features. and aggregate the first low-frequency component Second low frequency component The corresponding features yield low-frequency features. High-frequency components and Feature aggregation can be achieved through pixel-level weighted averaging. For example, preset weights can be assigned to two high-frequency components, and then they can be added pixel by pixel. Similarly, low-frequency components... and Feature aggregation can also employ a similar weighted averaging method, or by directly concatenating two components as a multi-channel feature map. Alternatively, a small convolutional neural network module can be designed. This module receives two high-frequency components as input, performs feature fusion through several convolutional layers and activation functions, and finally outputs high-frequency features. For low-frequency characteristics The aggregation can also employ a similar neural network structure, receiving two low-frequency components as input and outputting low-frequency features after processing. .

[0024] In step S140, the base model pre-trained based on the laser speckle training samples is obtained from the first image. Extracting speckle features And based on the speckle characteristics For high frequency features Knowledge guidance was conducted to obtain high-frequency guidance characteristics. And based on speckle characteristics For low-frequency features Knowledge guidance was conducted to obtain low-frequency guidance characteristics. The base model can be a self-supervised encoder network that has learned a general representation of the speckle images from a large number of laser speckle images. The model receives the first image. As input, output speckle features As an alternative approach, the base model can be a convolutional neural network pre-trained on a speckle image classification task, where the features output from the intermediate layers are used as speckle features. Knowledge guidance can be designed as a gating mechanism, where speckle features... These weights are used to generate gating weights, which are then applied to high-frequency features. This allows for the selective enhancement or suppression of certain components, thereby obtaining high-frequency guiding characteristics. Of course, low-frequency guidance characteristics The acquisition can also be achieved through a similar gating mechanism.

[0025] In step S150, the method is based on high-frequency guiding features. and low-frequency guidance characteristics The enhanced laser speckle image was reconstructed. .

[0026] High-frequency guidance characteristics and low-frequency guidance characteristics The images can be fused directly using an addition operation to obtain a fused feature map. This fused feature map can then be input into a deconvolutional network or an upsampling layer sequence to gradually restore the spatial resolution and details of the image, ultimately outputting an enhanced laser speckle image. As another implementation method, high-frequency guiding features can be used. and low-frequency guidance characteristics The images are fed into two independent decoder networks, each responsible for reconstructing a portion of the image information from its corresponding guiding features. The outputs of these two decoders are then fused, for example, by weighted averaging or concatenation, and then passed through an additional convolutional layer to generate the final enhanced image. .

[0027] Here is an example: Suppose a doctor needs to enhance a patient's retinal laser speckle image to more clearly observe the blood flow distribution in the microvessels, aiding in the diagnosis of eye diseases. However, the original laser speckle image typically suffers from severe speckle noise, low contrast, and obscured microvascular details, making accurate analysis difficult.

[0028] First, when a user issues a speckle image enhancement command through the medical imaging system, the system responds. The system loads a raw retinal laser speckle image from the image database and designates it as the first image. To provide multi-source information, the system simultaneously performs logarithmic bilateral filtering on the first image to generate a second image. The second image smooths noise to some extent and preserves some edge information, but may suffer from detail loss or over-smoothing.

[0029] Next, the system extracts high and low frequency components from the first and second images respectively. For the first image, by applying a spatial domain filter or frequency domain transform, its first high frequency component (containing the rich details and noise of the original image) and first low frequency component (containing the overall structure and brightness of the original image) can be separated. Similarly, for the second image, its second high frequency component and second low frequency component are also extracted. This step aims to decompose the image information into different levels so that it can be processed in a targeted manner later.

[0030] Subsequently, the system performs feature aggregation, fusing the features corresponding to the first high-frequency component and the second high-frequency component to obtain high-frequency features. The high-frequency features combine the rich details of the original image with the denoising advantages of the logarithmic bilateral filter image. At the same time, the features corresponding to the first low-frequency component and the second low-frequency component are fused to obtain low-frequency features. The low-frequency features combine the overall structural information of the two images. Through this aggregation, a more comprehensive and robust high- and low-frequency representation can be obtained.

[0031] Building upon this foundation, the system introduces a knowledge-guided mechanism. A base model (intermediate layer) pre-trained on a large number of laser speckle training samples is used to extract speckle features from the first image. These speckle features encode the unique texture and noise distribution patterns of the laser speckle image. Then, the speckle features are used to guide the aggregated high-frequency features, generating high-frequency guided features. This guidance allows the high-frequency features to better suppress speckle noise while preserving effective details. Similarly, the speckle features also guide the low-frequency features, generating low-frequency guided features, thus making the structural information of the low-frequency features clearer and reducing noise interference. Through knowledge guidance, the model can more intelligently distinguish between effective texture and noise in the image, avoiding problems of over-denoising or incomplete denoising.

[0032] Finally, based on the obtained high-frequency and low-frequency guiding features, the system uses an image reconstruction module to fuse and upsample these guided features, ultimately reconstructing an enhanced laser speckle image. Compared with the original image, the enhanced image significantly suppresses speckle noise, clearly restores the edges and texture details of microvessels, and improves overall contrast, providing reliable visual evidence for doctors to conduct precise observation of vascular morphology and assessment of blood flow distribution.

[0033] Based on the above examples, in traditional laser speckle image enhancement methods, the separation of high- and low-frequency features often relies on fixed, manually designed filters, such as Gaussian filters or Laplacian operators. This method has poor adaptability when processing speckle images with different noise intensities or tissue types, easily leading to blurred image edges or texture distortion. In contrast, this embodiment utilizes both the original image and a second image after logarithmic bilateral filtering, extracting their high- and low-frequency components separately. This provides multi-source information for subsequent feature aggregation. The introduction of this multi-source feature allows the system to capture image information more comprehensively, overcoming the limitations of a single image source.

[0034] Moreover, while existing deep learning methods have improved the enhancement effect to some extent, they are mostly end-to-end feature learning methods that do not fully explore the unique texture characteristics of laser speckle images. This often leads to excessive noise reduction that results in the loss of important details, or incomplete noise reduction that leaves residual noise. This embodiment introduces a basic model pre-trained based on laser speckle training samples to extract speckle features from the first image. The speckle features represent the domain knowledge of laser speckle and can effectively guide the generation of high-frequency and low-frequency features, resulting in high-frequency guided features and low-frequency guided features. This knowledge-guided mechanism enables the model to more intelligently distinguish between effective texture and noise during the enhancement process, avoiding blind noise reduction. Thus, while suppressing speckle noise, it preserves and restores key texture details such as microvessels to the maximum extent.

[0035] Therefore, this embodiment achieves refined enhancement of laser speckle images by decomposing and aggregating high and low frequencies of multi-source images and effectively guiding them with domain knowledge. This method not only overcomes the problems of poor adaptability and insufficient detail preservation in traditional methods, but also compensates for the shortcomings of existing deep learning methods in fully mining speckle textures. The resulting enhanced image provides a more reliable and accurate image foundation for subsequent medical diagnosis and quantitative analysis.

[0036] In some embodiments of this application, step S140, based on speckle characteristics... For high frequency features Knowledge guidance was conducted to obtain high-frequency guidance characteristics. ,include: Step S1411, speckle features The input is fed into the first convolutional layer, and the output features of the first convolutional layer are combined with the speckle features. Residual connection yields the first intermediate feature. ; Step S1412, the first intermediate feature The values ​​are mapped to Q, K, and V, and then input into the self-attention layer to obtain the second intermediate feature. ; Step S1413, the second intermediate feature The input is fed into the second convolutional layer to obtain the third intermediate feature output by the second convolutional layer. ; Step S1414, high-frequency features The input is fed into the third convolutional layer, and the output features of the third convolutional layer are combined with the high-frequency features. Residual connection yields the fourth intermediate feature. ; Step S1415, the fourth intermediate feature The values ​​are mapped to Q, K, and V, and then input into the self-attention layer to obtain the fifth intermediate feature. ; Step S1416, the fifth intermediate feature The input is fed into the fourth convolutional layer to obtain the sixth intermediate feature output by the fourth convolutional layer. ; Step S1417, the third intermediate feature and the sixth intermediate feature Adding them together yields the high-frequency guiding characteristics. ; Step S140 based on speckle characteristics For low-frequency features Knowledge guidance was conducted to obtain low-frequency guidance characteristics. ,include: Step S1421, low-frequency features The input is fed into the fifth convolutional layer, and the output features of the fifth convolutional layer are combined with the low-frequency features. Residual connection yields the seventh intermediate feature. ; Step S1422, the seventh intermediate feature The values ​​are mapped to Q, K, and V, and then input into the self-attention layer to obtain the eighth intermediate feature. ; Step S1423, the eighth intermediate feature The input is fed into the sixth convolutional layer to obtain the ninth intermediate feature output by the sixth convolutional layer. ; Step S1424, the third intermediate feature and the ninth intermediate feature Adding them together yields the low-frequency guiding characteristics. .

[0037] Convolutional layers are the core components of deep learning networks used for feature extraction. By sliding convolutional kernels (or filters) across the input data, they learn local patterns and spatial hierarchical representations. Their role is to capture features such as edges and textures in images, and more abstract, higher-level semantic information can be extracted by stacking multiple convolutional layers. Residual connections can help alleviate the vanishing gradient problem in deep network training, allowing the network to be trained deeper, while promoting the effective propagation of information and the reuse of features.

[0038] Mapping features to Q, K, and V values ​​is a key step in self-attention mechanisms. Input features are transformed into three different representations—Query (Q), Key (K), and Value (V)—through linear transformations (typically 1x1 convolutional layers or fully connected layers). This mapping aims to provide the attention mechanism with different perspectives for calculating the relationships between features, enabling the model to more flexibly capture complex dependencies between features. For example, three independent 1x1 convolutional layers can be used to convert input features into Q, K, and V tensors respectively, with the number of channels in each tensor designed based on the number and dimensionality of the attention heads.

[0039] Self-attention layers can compute the correlations between different positions in the input sequence, thereby capturing long-distance dependencies. They obtain attention weights by computing the dot product of Q and K, and then apply these weights to V to generate a weighted sum output. This allows the model to dynamically adjust the representation of features based on the importance of the input features themselves, thereby enhancing the focus on key information. For example, a multi-head self-attention mechanism can be used, where the input features are divided into multiple "heads," each of which independently computes attention, and then the results are concatenated to capture richer semantic information and improve the model's expressive power.

[0040] This embodiment introduces a refined knowledge-guided mechanism to enhance speckle characteristics. It can effectively guide high-frequency characteristics and low frequency characteristics Generate more meaningful high-frequency guiding features Low-frequency guidance characteristics Specifically, for high-frequency guidance features To obtain the speckle features, firstly... The data is fed into the first convolutional layer for initial feature extraction and then connected to the original speckle features via residual connections. Fusion yields the first intermediate feature. This step aims to enhance the speckle characteristics. The expressive power, while preserving its original information, then, the first intermediate feature The values ​​are mapped to Q, K, and V and then fed into a self-attention layer to capture speckle features. The complex internal dependencies generate a second intermediate feature. Second intermediate feature After processing through a second convolutional layer, the third intermediate feature is obtained. The features can be viewed as speckled knowledge representation refined through deep learning and self-attention mechanisms.

[0041] High frequency characteristics It is also independently input into the third convolutional layer and connected to the original high-frequency features via residual connections. Fusion yields the fourth intermediate feature. To enhance its feature representation and preserve the original high-frequency information, the fourth intermediate feature They are also mapped to Q, K, and V values ​​and input into the self-attention layer to capture high-frequency features. Internal dependencies generate the fifth intermediate feature. Fifth intermediate feature After processing through the fourth convolutional layer, the sixth intermediate feature is obtained. Ultimately, by analyzing the speckle characteristics The third intermediate feature extracted from With high frequency characteristics The sixth intermediate feature extracted from the middle By adding them together, we obtain the high-frequency guiding characteristics. This additive operation allows speckle knowledge to be directly applied to high-frequency features, enabling refined guidance.

[0042] For low-frequency guiding characteristics Acquisition of low-frequency features It is input into the fifth convolutional layer and connected to the original low-frequency features via residual connections. Fusion yields the seventh intermediate feature. The seventh intermediate feature These values ​​are mapped to Q, K, and V values ​​and fed into a self-attention layer to capture low-frequency features. Internal dependencies generate the eighth intermediate feature. Eighth intermediate feature After processing through the sixth convolutional layer, the ninth intermediate feature is obtained. Finally, we will discuss the speckle characteristics. The third intermediate feature extracted from With low-frequency characteristics The ninth intermediate feature extracted from the middle By adding them together, we obtain the low-frequency guiding characteristics. .

[0043] Through the above mechanism, the knowledge of speckle features is deeply encoded and used to guide the generation of high-frequency and low-frequency features. The self-attention mechanism enables the model to dynamically focus on the parts of speckle features most relevant to high-frequency and low-frequency features, thereby achieving more accurate knowledge transfer and feature fusion. This method not only solves the problem of how to effectively fuse speckle feature knowledge, but also ensures the depth and precision of knowledge guidance through the combination of multi-layer convolution and self-attention mechanism, so that high-frequency and low-frequency guided features can more accurately reflect the real content of the image and provide high-quality input for subsequent image reconstruction.

[0044] The mechanism provided in this embodiment utilizes a multi-layer convolutional network and a self-attention layer to deeply encode and effectively fuse the speckle features extracted from the first image into high-frequency and low-frequency features. The self-attention mechanism enables the model to dynamically identify and emphasize the parts of the speckle features most relevant to the high-frequency and low-frequency components, thereby achieving more accurate knowledge transfer. This deep and adaptive guidance method can effectively solve the problem of insufficient speckle knowledge fusion or information loss in traditional methods, ensuring that high-frequency and low-frequency guided features can more accurately reflect the true content of the image, thus providing high-quality, high-fidelity feature input for the subsequent reconstructed and enhanced laser speckle image, significantly improving the quality of image enhancement and detail preservation.

[0045] In some embodiments of this application, the aggregation of the first low-frequency component in step S130 Second low frequency component The corresponding features yield low-frequency features. ,include: Step S1311: Aggregate the first low-frequency component based on the attention mechanism. Second low frequency component The eleventh intermediate feature is obtained. ; Step S1312, eleventh intermediate feature Perform a 2x downsampling to obtain the twelfth intermediate feature. The twelfth intermediate feature Perform a 2x downsampling to obtain the thirteenth intermediate feature. The thirteenth intermediate feature Perform a 2x downsampling to obtain the fourteenth intermediate feature. The fourteenth intermediate feature Perform a 2x downsampling to obtain the fifteenth intermediate feature. ; Step S1313, the fifteenth intermediate feature The input is fed into multiple stacked convolutional blocks to obtain the sixteenth intermediate feature from the output of the multiple stacked convolutional blocks. ; Step S1314, the sixteenth intermediate feature Perform upsampling by 2x to obtain the seventeenth intermediate feature. According to the seventeenth intermediate feature Intermediate features of the fifteenth The eighteenth intermediate feature is obtained. Among them, the eighteenth intermediate feature The acquisition process includes: ;in It's pixel attention. It is channel attention. , All are learnable weights; Step S1315, the eighteenth intermediate feature and the seventeenth intermediate feature The nineteenth intermediate feature is obtained by performing upsampling by 2x and fusing the features after upsampling. According to the nineteenth intermediate feature With the fourteenth intermediate feature The twentieth intermediate feature was obtained. Among them, the twentieth intermediate feature The acquisition process includes: ; Step S1316, the twentieth intermediate feature and the nineteenth intermediate feature The twenty-first intermediate feature is obtained by performing upsampling by a factor of 2 and then fusing the features after upsampling. According to the twenty-first intermediate feature With the thirteenth intermediate feature The twenty-second intermediate feature was obtained. Among them, the twenty-second intermediate feature The acquisition process includes: ; Step S1317, the twenty-second intermediate feature Twenty-first intermediate feature The low-frequency features are obtained by performing upsampling by 2x and fusing the features after upsampling. .

[0046] Among them, the first low-frequency component is aggregated based on the attention mechanism. Second low frequency component The eleventh intermediate feature is obtained. The attention mechanism, in particular, adaptively adjusts the weights of low-frequency components from different sources based on their importance when fusing them. This allows for more effective extraction and integration of useful structural information while suppressing noise and redundancy. The attention mechanism can employ channel attention, which learns the weights of each channel to emphasize the importance of different feature channels; or spatial attention, which learns the weights of each spatial location to highlight key regions in the image.

[0047] The eleventh intermediate feature Perform a 2x downsampling to obtain the twelfth intermediate feature. The downsampling process is repeated multiple times until the fifteenth intermediate feature is obtained. Downsampling aims to gradually reduce the size of feature maps, thereby expanding the model's receptive field and enabling it to capture a wider range of contextual information and more abstract semantic features in the image. Simultaneously, downsampling helps reduce the dimensionality of feature maps, decreasing the complexity of subsequent computations. Downsampling can be achieved through max pooling layers, which select the maximum value within a local region as the output; or through average pooling layers, which calculate the average value within a local region as the output; or through convolutional layers with a stride of 2, performing size reduction simultaneously with feature extraction.

[0048] The fifteenth intermediate feature The input is fed into multiple stacked convolutional blocks to obtain the sixteenth intermediate feature from the output of the multiple stacked convolutional blocks. Stacked convolutional blocks are used to further abstract and learn representations of deep features after multiple downsampling. Each convolutional block typically contains one or more convolutional layers, activation functions, and normalization layers. Through multiple nonlinear transformations, more discriminative low-frequency semantic information can be extracted from low-dimensional features. These convolutional blocks can use a combination of standard convolutional layers and ReLU activation functions, or a residual connection structure to deepen the network and alleviate the gradient vanishing problem.

[0049] The sixteenth intermediate feature Perform upsampling by 2x to obtain the seventeenth intermediate feature. Upsampling aims to gradually restore the spatial resolution of the feature map in order to fuse deep semantic information with shallow detail information, thereby generating high-resolution output features. Upsampling can be achieved through interpolation methods such as bilinear interpolation or nearest neighbor interpolation; or through transposed convolution (also known as deconvolution) layers, which restore the size of the feature map through learning.

[0050] According to the seventeenth intermediate feature Intermediate features of the fifteenth The eighteenth intermediate feature is obtained. Among them, the eighteenth intermediate feature The acquisition process includes: This step combines pixel attention (PA) and channel attention (CA) mechanisms and introduces learnable weights. , This approach refines the fusion of features at different scales. Pixel attention (PA) enables the model to focus on the importance of each pixel in the feature map, while channel attention (CA) focuses on the importance of each feature channel. In this way, the model can adaptively adjust the feature contributions from different levels, enhancing key information and suppressing irrelevant information. This results in more accurate detail recovery and semantic integration during feature fusion, and learnable weights. , The model is allowed to automatically optimize the contribution ratio of different feature branches based on the data during training.

[0051] This embodiment constructs a multi-scale feature aggregation network to efficiently and robustly fuse the first and second low-frequency components to generate high-quality low-frequency features. First, the original first low-frequency component and the second low-frequency component (after logarithmic bilateral filtering) are initially aggregated using an attention mechanism to obtain the eleventh intermediate feature. This step utilizes the adaptive weighting capability of the attention mechanism to intelligently filter and integrate low-frequency information from different sources at an early stage, ensuring the feature quality of subsequent processing. Subsequently, the eleventh intermediate feature undergoes a series of 2x downsampling operations to gradually generate the twelfth to fifteenth intermediate features. This multi-level downsampling process effectively expands the receptive field, enabling the network to capture global contextual information and more abstract low-frequency structures of the image, while reducing the feature dimensionality, laying the foundation for deep feature learning. The network establishes a foundation. Based on the lowest-resolution fifteenth intermediate feature, deep feature extraction is performed using multiple stacked convolutional blocks to obtain the sixteenth intermediate feature. These convolutional blocks learn more discriminative semantic representations from the compressed features. Next, the network enters an upsampling stage, upsampling the sixteenth intermediate feature by a factor of 2 to obtain the seventeenth intermediate feature. In this stage, to compensate for any details lost during downsampling and to fully utilize features at different scales, a feature fusion mechanism based on pixel attention and channel attention is introduced. Specifically, the seventeenth intermediate feature is fused with the corresponding fifteenth intermediate feature through a weighted summation, where the weights... , It is learnable, and the fusion process combines pixel attention and channel attention, enabling the fused eighteenth intermediate feature to more accurately preserve important structural details and suppress noise. Subsequently, the multi-level upsampling and attention fusion process is repeated, that is, the eighteenth and seventeenth intermediate features are fused to obtain the nineteenth intermediate feature, which is then fused with the fourteenth intermediate feature to obtain the twentieth intermediate feature, and the twentieth and nineteenth intermediate features are fused to obtain the twenty-first intermediate feature, which is then fused with the thirteenth intermediate feature to obtain the twenty-second intermediate feature. This fusion strategy of progressive upsampling combined with attention mechanism ensures that while restoring image resolution, it can finely integrate multi-scale features and effectively utilize low-frequency information at different levels. Finally, by upsampling the twenty-second and twenty-first intermediate features by 2 times and then fusing the features after upsampling, the final low-frequency feature is obtained.

[0052] Through the aforementioned hierarchical aggregation and attention-guided fusion mechanism, this embodiment can overcome the limitations of simple aggregation methods in processing complex low-frequency information. It can not only extract complementary low-frequency information from the original image and the preprocessed image, but also select and enhance features that are crucial to the image structure through multi-scale processing and attention mechanisms, while effectively suppressing noise, thereby generating a purer and clearer low-frequency feature. This high-quality low-frequency feature can provide a more accurate structural basis for the reconstruction of the enhanced laser speckle image in the subsequent knowledge-guided process, significantly improving the overall effect of image enhancement.

[0053] In some embodiments of this application, the aggregation of the first high-frequency component in step S130 Second high frequency component The corresponding features yield high-frequency features. ,include: Step S1321: Aggregate the first high-frequency component based on the attention mechanism. Second high frequency component The tenth intermediate feature is obtained. ; Step S1322, based on U-net from the tenth intermediate feature Extracting high-frequency features .

[0054] Among them, the first high-frequency component is aggregated based on the attention mechanism. Second high frequency component The tenth intermediate feature is obtained. This refers to the weighted fusion of two high-frequency components from different sources using an attention mechanism. Attention mechanisms are techniques that simulate the selective attention of the human visual system, allowing the model to assign different weights to different parts of information, thereby highlighting important information and suppressing unimportant information. When aggregating high-frequency components, channel attention mechanisms can be used, which learn the importance of each channel and assign different weights to different channels to enhance or suppress features of specific channels. Spatial attention mechanisms can also be used, which learn the importance of each spatial location on the feature map and assign different weights to different locations to highlight key regions in the image. Furthermore, hybrid attention mechanisms can be used, combining channel attention and spatial attention to more comprehensively capture the importance of features. In this way, the importance of the first high-frequency component can be dynamically weighed. Second high frequency component The contribution of the aggregated tenth intermediate feature makes It can more effectively preserve useful high-frequency details and suppress noise.

[0055] This embodiment first utilizes an attention mechanism to target the first high-frequency component. Second high frequency component Aggregation is performed, and their contributions to the final high-frequency features are dynamically weighed to generate the tenth intermediate feature. This aggregation method avoids the information redundancy or noise amplification problems that may result from simple superposition, enabling the system to more effectively focus on the truly valuable high-frequency details in the image. Based on this, the tenth intermediate feature... The input is fed into the U-net network for deep processing. U-net, with its unique encoder-decoder structure and skip connections, can capture feature information at multiple scales and effectively preserve spatial details during reconstruction. The encoder progressively extracts abstract semantic features, while the decoder utilizes these features and fine-grained information from skip connections to progressively recover the spatial resolution and details of high-frequency features, ensuring the obtained high-frequency features are fully preserved. It not only contains complementary information from different sources, but also undergoes thorough denoising and detail enhancement, thus providing high-quality input for subsequent knowledge-guided and image reconstruction.

[0056] In this embodiment, when aggregating the first and second high-frequency components, an attention mechanism is used to dynamically balance the contributions of different high-frequency components. This allows the system to more intelligently focus on important details in the image while suppressing noise interference, thus obtaining a more representative tenth intermediate feature. Based on this, high-frequency features are extracted from the tenth intermediate feature using a U-net network. The encoder-decoder structure and skip connections of U-net effectively capture multi-scale contextual information and fine spatial details, further refining and denoising the high-frequency information. This combination of the attention mechanism and the aggregation and extraction method of U-net ensures that the obtained high-frequency features retain rich texture details while effectively suppressing speckle noise, providing high-quality input for subsequent knowledge guidance and image reconstruction, and significantly improving the enhancement effect and visual quality of laser speckle images.

[0057] In some embodiments of this application, step S150 is based on high-frequency guiding features. and low-frequency guidance characteristics The enhanced laser speckle image was reconstructed. ,include: Step S1511, calculate the first image Local variance ;in, The first image; ; ; in, To integrate high-frequency guidance features and low-frequency guidance characteristics The resulting fusion features; Step S1512: Reconstruct the enhanced laser speckle image by fusing the features using a 1x1 convolutional layer. .

[0058] Among them, the first image is calculated. Local variance Aimed at quantifying the first image The local variance of an image is used to determine the texture complexity and information content of each local region. High-variance regions typically indicate image edges, texture details, or noise, while low-variance regions represent smooth backgrounds. By obtaining the local variance, regional weight information can be provided for subsequent feature fusion, enabling the fusion process to adaptively focus on the characteristics of different regions of the image. Local variance can be implemented in various ways, such as using a sliding window (e.g., a 3x3 or 5x5 square window) in the first image. The algorithm iterates through the image and calculates the variance of the grayscale value for each pixel within the window. Alternatively, it can perform Gaussian smoothing on the image, calculate the square of the difference between the original image and the smoothed image, and then perform local averaging to obtain an approximate value of the local variance.

[0059] formula A weighted fusion strategy is defined to integrate high-frequency guiding features. and low-frequency guidance characteristics Combined, to generate fusion features By introducing variable weights The fusion method can dynamically adjust the contributions of high-frequency and low-frequency features based on the local characteristics of the image, thereby achieving more refined image enhancement. In deep learning models, this is typically achieved through element-wise multiplication and addition operations, where... It is a weight map that matches the image size. Alternatively, it can be achieved through a specially designed fusion layer that receives high-frequency guiding features. Low-frequency guidance characteristics and weight As input, and output fused features .

[0060] formula Define fusion weights The calculation method. By calculating the local variance. Perform normalization (divide by maximum variance) By applying the sigmoid activation function, a weight value between 0 and 1 can be obtained. This weight value reflects the texture complexity of a local region of the image, allowing high-variance regions (usually regions rich in detail) to achieve higher variance. The weights are calculated to retain more high-frequency information during fusion, while low-variance regions rely more on low-frequency information. The calculation of weights can begin by calculating the local variance. Then find its maximum value. Perform pixel-by-pixel division and finally input the result into the sigmoid function. Alternatively, a small neural network module can be designed to receive the local variance. As input, output the weight map after normalization and sigmoid activation. .

[0061] This embodiment can adaptively fuse high-frequency and low-frequency guiding features based on the local texture characteristics of the laser speckle image. This dynamic weighted fusion strategy ensures that high-frequency information can be fully utilized for enhancement in areas with rich image details, while low-frequency information is relied upon more in smooth areas to suppress noise. This avoids the problem of detail loss or noise amplification that may be caused by traditional fixed-weight fusion. Therefore, this embodiment significantly improves the visual quality and information readability of the laser speckle image while maintaining the integrity of the image structure, making the enhanced image clearer, more detailed, and more conducive to subsequent analysis and processing.

[0062] In some embodiments of this application, step S140 involves the first image... Extracting speckle features ,include: Step S1431: Train the teacher model based on the laser speckle training samples so that the teacher model learns the speckle texture of the laser speckle training samples; Step S1432: Distill the feature knowledge of the teacher model into the student model, so that the student model can learn the distribution pattern and edge features of speckle texture; Step S1432, based on the student model from the first image Extracting speckle features .

[0063] The teacher model is a computational model capable of learning complex patterns from large amounts of data. Its main function is to comprehensively and deeply understand the intrinsic texture characteristics of laser speckle images. The teacher model can be a deep convolutional neural network (CNN), such as ResNet, DenseNet, or U-Net, or it can be a model based on the Transformer architecture. The laser speckle training samples are the dataset used to train the teacher model. They can contain a large number of original laser speckle images, or pairs of original laser speckle images and their corresponding ideal speckle-free images. By learning from these samples, the teacher model can capture the statistical characteristics, spatial distribution, and texture information related to the image content of the speckle. The process of learning speckle texture can be achieved through supervised learning, such as minimizing the difference between the teacher model output and the real texture; or through unsupervised learning, such as learning a low-dimensional representation of speckle images through an autoencoder, or learning to generate realistic speckle textures through a generative adversarial network (GAN).

[0064] Knowledge distillation is a model compression and acceleration technique that aims to transfer knowledge learned by a large, complex teacher model to a small, efficient student model. Knowledge distillation can be implemented in several ways: soft label distillation, where the student model mimics the output probability distribution of the teacher model; feature distillation, where the student model mimics the feature representations of the intermediate layers of the teacher model; or relation distillation, where the student model mimics the relationships between different layers or samples in the teacher model. The student model, trained through knowledge distillation, typically has fewer parameters and lower computational complexity while maintaining performance similar to the teacher model. The student model can be a lightweight convolutional neural network, such as MobileNet, ShuffleNet, or EfficientNet, or a compact network structure customized to meet specific application needs. Through knowledge distillation, the student model can effectively learn the distribution patterns and edge features of speckle textures, i.e., the statistical distribution characteristics of speckles in images and the speckle representation at object boundaries in images.

[0065] This embodiment overcomes the challenge of directly extracting high-quality features from complex and noisy laser speckle images. By comprehensively learning speckle texture through a teacher model and efficiently transferring this knowledge to a student model using knowledge distillation, the extracted speckle features are ensured to have higher accuracy and representativeness. This enables the subsequent knowledge-guided process to more effectively utilize speckle features to guide the generation of high-frequency and low-frequency features, thereby significantly improving the quality of the enhanced laser speckle image, making image details clearer, and suppressing speckle noise more effectively.

[0066] In some embodiments of this application, step S120 involves extracting the second image. The second high frequency component Second low frequency component include: Step S1211: Extract the second image based on a 3×3 convolution kernel and the Sobel operator. The second high frequency component ; Step S1212, based on the second image Subtract the second high frequency component The second low-frequency component was obtained. ; Extracting the first image in step S120 First high frequency component and the first low-frequency component include: Step S1221: Extract the first image based on a 3×3 convolution kernel and the Sobel operator. First high frequency component ; Step S1222, based on the first image Subtract the first high-frequency component The first low-frequency component was obtained. .

[0067] The 3x3 convolution kernel is a filter used in image processing to extract features. The Sobel operator is a commonly used edge detection operator that highlights edge information in an image by calculating the gradient approximation of image pixels. It typically consists of a pair of 3x3 convolution kernels, used to detect edges in the horizontal and vertical directions, respectively. The operator can effectively identify regions in an image with drastic brightness changes; these regions typically correspond to the high-frequency components of the image. Extracting the second image. The second high frequency component The aim is to extract the second image after logarithmic bilateral filtering. The high-frequency information, namely the edges, textures, and other details of the image, is extracted. By applying specific filters, these high-frequency features can be effectively highlighted, providing fine detail information for subsequent feature aggregation.

[0068] Second image Subtract the second high frequency component The second low-frequency component was obtained. The operation can convert the second image The relatively smooth, slowly changing regions in the image are separated out. These regions represent the overall structure and brightness information of the image. Besides this subtraction operation, this can also be achieved by directly modifying the second image. Low-pass filtering (e.g., Gaussian blur) is used to obtain the second low-frequency component. Alternatively, wavelet transform and other frequency domain decomposition methods can be used to separate low-frequency components and extract the first image. First high frequency component The aim is to extract from the original laser speckle image Extracting its high-frequency information from the original image, since the original image It may contain more noise and speckle texture, therefore accurate extraction of its high-frequency components is crucial for subsequent speckle feature analysis. (First image) Subtract the first high-frequency component The first low-frequency component was obtained. The low-frequency components represent the original image. The overall brightness distribution and macroscopic structure of the first image provide basic information for subsequent feature aggregation. Besides this subtraction operation, it is also possible to directly perform feature aggregation on the first image. Perform low-pass filtering (e.g., mean filtering) to obtain the first low-frequency component. Alternatively, frequency domain analysis methods such as Fourier transform can be used to separate low-frequency components.

[0069] This embodiment employs a 3x3 convolution kernel and the Sobel operator to extract high-frequency components, combined with image subtraction to obtain low-frequency components. This enables precise high- and low-frequency separation of the original laser speckle image and the image after logarithmic bilateral filtering. This method effectively captures details such as edges and textures in the image while separating smooth background information, avoiding the loss of details or noise confounding problems that may occur with traditional single filtering methods. Because the high- and low-frequency components are separated more accurately, the subsequent feature aggregation and knowledge-guided processes can obtain cleaner and more representative inputs, enabling the deep learning model to learn and utilize these components more effectively for image enhancement. This significantly improves the quality of the enhanced laser speckle image, making the image details clearer, effectively suppressing speckle noise, and significantly improving the overall visual effect, providing high-quality image data for subsequent image analysis and applications.

[0070] like Figures 2 to 3 One embodiment of this application provides a deep learning-based image enhancement method for laser speckle, comprising the following steps: Step S910: In response to the speckle image enhancement command, determine the first image and the second image; Upon receiving a user-triggered speckle image enhancement command, the original laser speckle image is acquired as the first image. The first image is a single-channel grayscale image acquired in the biomedical field, containing speckle textures and multiplicative noise related to tissue blood flow. Regarding the first image... The second image is obtained by sequentially performing logarithmic transformation and bilateral filtering. (Belongs to a different modality than the original image); the logarithmic transformation formula is: , avoid ; Convert multiplicative noise into additive noise; The formula for bilateral filtering is: ； The bilateral filter parameters are set to filter window 9, color standard deviation 75, and spatial standard deviation 75, which preserves image edge and texture details while reducing noise.

[0071] Step S920: Extract the first and second images respectively. High and low frequency components; Step S9210: Extract the first high-frequency component of the first image. and the first low-frequency component ; A high-frequency extractor is constructed by combining a 3×3 convolution kernel with the Sobel edge detection operator. The first image is input into the high-frequency extractor to extract high-frequency information such as edges and textures of the image to obtain the first high-frequency component. The first low-frequency component is obtained by subtracting the first high-frequency component from the first image. The first low-frequency component contains low-frequency information such as the overall brightness of the image and the smooth background.

[0072] Step S9220: Extract the second high-frequency component of the second image. Second low frequency component Similar to step S9210.

[0073] Step S930: Aggregate high and low frequency component features to obtain high frequency features. and low frequency characteristics ; Step S9310: Aggregate high-frequency component features to obtain high-frequency features. ; Based on a hybrid attention mechanism combining channel attention and spatial attention, feature aggregation is performed on the first and second high-frequency components, and feature weights are adaptively assigned to the two high-frequency components to obtain the tenth intermediate feature; the tenth intermediate feature is then... The input is fed into a lightweight network, and through the encoding, decoding and skip connection structure, the high-frequency features of speckle texture are extracted in depth.

[0074] Step S9320: Aggregate low-frequency component features to obtain low-frequency features. ; Based on the attention mechanism, features of the first low-frequency component and the second low-frequency component are aggregated to obtain the eleventh intermediate feature. ; Perform 2x downsampling sequentially to obtain the twelfth to fifteenth intermediate features. Gradually compress the feature dimensions and extract global low-frequency features; The input is fed into a feature extraction layer consisting of three stacked convolutional blocks. Each convolutional block contains a 3×3 convolution, a BN layer, and a ReLU activation function to obtain the sixteenth intermediate feature. ; For the sixteenth intermediate feature 2x upsampling Combining pixel attention (PA) and channel attention (CA), the formula is obtained. ; ; in and The weights are learnable and are initially set to 0.5, which are then adaptively adjusted during training. Will and After 2x upsampling, the samples were fused together. Similarly, through get ; right and Repeat the upsampling, fusion, and attention-weighted operations to obtain and ′; Finally, and The low-frequency features were obtained by upsampling by 2 times and fusing the features respectively. .

[0075] Step S940: Extract prior features of the speckle domain and provide knowledge guidance for high- and low-frequency features; Step S9410: Extract speckle features ; Teacher and student models were constructed. The teacher model adopted the Swin Transformer architecture, and the student model adopted the lightweight ResNet18 architecture, replacing the first convolutional layer with a 1-channel input to adapt to laser speckle images. Training samples of laser speckle with different tissues and noise intensities were collected to train the teacher model for speckle texture reconstruction, enabling the teacher model to learn the high-order feature rules of speckle texture. Feature distillation was used to distill the feature knowledge of the intermediate layers of the teacher model into the student model, enabling the student model to learn the distribution rules and edge features of speckle texture. The first image was then used to... The input is fed into a trained student model with fixed parameters, and the output of the model's avgpool layer is extracted as speckle features. And adjust it to the same channel dimension as the high and low frequency features by 1×1 convolution.

[0076] Step S9420: High-frequency features Knowledge guidance ; Will The input is fed into the first convolutional layer (3×3 convolution), and the output features are then compared with... Perform residual connection to obtain To avoid feature loss; The feature matrices Q, K, and V are mapped through a linear layer and then input into a self-attention layer to learn the global dependencies of the speckle features, thus obtaining... ;Will The input is fed into the second convolutional layer (3×3 convolution) to obtain... ; Will The input is fed into the third convolutional layer (3×3 convolution), and the output features are... Residual connection obtained Similarly, Mapped to Q, K, V and input to the self-attention layer, we obtain ;Will The input is fed into the fourth convolutional layer (3×3 convolution) to obtain... ;Will and By adding elements one by one, we obtain the high-frequency guiding characteristics.

[0077] Step S9430: Low-frequency characteristics Knowledge guidance ; Will The input is fed into the fifth convolutional layer (3×3 convolution), and the output features are... Residual connection obtained ;Will Mapped to Q, K, V and input to the self-attention layer, we obtain ;Will The input is fed into the sixth convolutional layer (3×3 convolution), resulting in... ; and f3′ and Element-by-element addition yields low-frequency guiding characteristics. .

[0078] Step S950: Fuse high- and low-frequency guiding features to reconstruct the enhanced laser speckle image. ; Calculate the local variance of the first image A 3×3 sliding window is used to calculate the local variance of each pixel, reflecting the texture density of different regions of the image; this is achieved through the formula... Calculate adaptive fusion weights , The value range is (0,1), and the texture is dense in the region. Approaching 1, smooth region Approaching 0; According to the formula right and Weighted fusion is performed to obtain fusion features. ; to integrate features The image is input into a 1×1 convolutional layer, and the feature channel is adjusted to 1 channel to obtain the enhanced laser speckle image, thus completing the entire image enhancement process.

[0079] This embodiment has at least the following beneficial effects: The preprocessed image is obtained by performing logarithmic bilateral filtering on the original laser speckle image. The high and low frequency components of the two images are extracted respectively. The complete texture features of the original image and the noise-reduced clean features of the preprocessed image are fully combined to achieve the complementarity of multi-source features, effectively solving the problem of insufficient feature information of a single image and laying a good foundation for subsequent feature enhancement. Different aggregation strategies are designed for high-frequency and low-frequency features. High-frequency features are extracted deeply using U-Net, while low-frequency features are accurately aggregated by combining multiple rounds of downsampling, convolutional block processing, and upsampling with an attention mechanism, thereby improving the fusion efficiency and feature representation ability of multi-source features. We introduce speckle domain prior features based on knowledge distillation, and through the combined processing of residual connections, self-attention layers and convolutional layers, we achieve efficient fusion and knowledge guidance of domain knowledge and high and low frequency enhancement features, so that feature learning is more in line with the texture pattern of laser speckle, and effectively improves the targeting and effectiveness of features. An adaptive weight fusion strategy based on the local variance of the original image is designed to differentially weight high-frequency regions with dense textures and smooth low-frequency regions, thereby achieving accurate fusion of high and low frequency guided features. Finally, the image is reconstructed through a 1×1 convolutional layer, which effectively suppresses speckle noise, preserves texture details, and improves the contrast and clarity of the enhanced image, meeting the needs of the biomedical field for high-precision analysis of laser speckle images.

[0080] Figure 4 One embodiment of this application provides an apparatus comprising: The instruction response module 1001 is used to determine the first image in response to a speckle image enhancement instruction. Second image The first image The first image is the original laser speckle image; the second image is... It is the first image The image obtained after performing logarithmic bilateral filtering; The high and low frequency component extraction module 1002 is used to extract the first image respectively. First high frequency component and the first low-frequency component and extracting the second image The second high frequency component Second low frequency component ; The high and low frequency feature extraction module 1003 is used to aggregate the first high frequency component. Second high frequency component The corresponding features yield high-frequency features. and aggregate the first low-frequency component Second low frequency component The corresponding features yield low-frequency features. ; Feature knowledge guidance module 1004 is used to pre-train the base model based on laser speckle training samples from the first image. Extracting speckle features And based on the speckle characteristics For high frequency features Knowledge guidance was conducted to obtain high-frequency guidance characteristics. And based on speckle characteristics For low-frequency features Knowledge guidance was conducted to obtain low-frequency guidance characteristics. ; The speckle image reconstruction module 1005 is used for high-frequency guided features. and low-frequency guidance characteristics The enhanced laser speckle image was reconstructed. .

[0081] It should be noted that the deep learning-based laser speckle image enhancement device provided in this embodiment is based on the same inventive concept as the deep learning-based laser speckle image enhancement method described above. Therefore, the content of the deep learning-based laser speckle image enhancement method described above is also applicable to the content of the deep learning-based laser speckle image enhancement device in this embodiment, and will not be repeated here.

[0082] like Figure 5 One embodiment of this application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned deep learning-based laser speckle image enhancement method. The electronic device includes: At least one battery; At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the deep learning-based laser speckle image enhancement method described above in this disclosure.

[0083] Electronic devices can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0084] The electronic devices according to embodiments of this application will now be described in detail.

[0085] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to perform an image enhancement method for laser speckle based on deep learning according to an embodiment of this disclosure.

[0086] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0087] This disclosure also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the detection method of the pressurized water reactor containment pressure control system described above.

[0088] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0089] The embodiments described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided by this disclosure. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by this disclosure are also applicable to similar technical problems.

[0090] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this disclosure, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0091] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0092] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0093] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0094] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0095] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0096] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0097] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0098] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

Claims

1. A laser speckle image enhancement method based on deep learning, characterized in that, The method includes: In response to the speckle image enhancement command, determine the first image. Second image The first image The first image is the original laser speckle image, and the second image is... It is the first image The image obtained after performing logarithmic bilateral filtering; Extract the first image respectively First high frequency component and the first low-frequency component and extracting the second image The second high frequency component Second low frequency component ; Aggregate the first high-frequency component and the second high-frequency component The corresponding features yield high-frequency features. and aggregating the first low-frequency component and the second low-frequency component The corresponding features yield low-frequency features. ; The base model, pre-trained based on laser speckle training samples, is derived from the first image. Extracting speckle features And based on the speckle characteristics For the high frequency features Knowledge guidance was conducted to obtain high-frequency guidance characteristics. and based on the speckle characteristics Regarding the low-frequency features Knowledge guidance was conducted to obtain low-frequency guidance characteristics. ; Based on the high-frequency guiding features and the low-frequency guiding features The enhanced laser speckle image is reconstructed. .

2. The image enhancement method for laser speckle based on deep learning according to claim 1, characterized in that, According to the speckle feature For the high frequency features Knowledge guidance was conducted to obtain high-frequency guidance characteristics. ,include: The speckle feature The input is fed into the first convolutional layer, and the output features of the first convolutional layer are combined with the speckle features. Residual connection yields the first intermediate feature. ; The first intermediate feature The values ​​are mapped to Q, K, and V, and then input into the self-attention layer to obtain the second intermediate feature. ; The second intermediate feature The input is fed into the second convolutional layer to obtain the third intermediate feature output by the second convolutional layer. ; The high-frequency features The input is fed into the third convolutional layer, and the output features of the third convolutional layer are combined with the high-frequency features. Residual connection yields the fourth intermediate feature. ; The fourth intermediate feature The values ​​are mapped to Q, K, and V, and then input into the self-attention layer to obtain the fifth intermediate feature. ; The fifth intermediate feature The input is fed into the fourth convolutional layer to obtain the sixth intermediate feature output by the fourth convolutional layer. ; The third intermediate feature and the sixth intermediate feature Adding them together yields the high-frequency guiding characteristics. ; According to the speckle feature Regarding the low-frequency features Knowledge guidance was conducted to obtain low-frequency guidance characteristics. ,include: The low-frequency features The input is fed into the fifth convolutional layer, and the output features of the fifth convolutional layer are combined with the low-frequency features. Residual connection yields the seventh intermediate feature. ; The seventh intermediate feature The values ​​are mapped to Q, K, and V, and then input into the self-attention layer to obtain the eighth intermediate feature. ; The eighth intermediate feature The input is fed into the sixth convolutional layer to obtain the ninth intermediate feature output by the sixth convolutional layer. ; The third intermediate feature and the ninth intermediate feature Adding them together yields the low-frequency guiding characteristics. .

3. The image enhancement method for laser speckle based on deep learning according to claim 2, characterized in that, The aggregation of the first low-frequency component and the second low-frequency component The corresponding features yield low-frequency features. ,include: The first low-frequency component is aggregated based on an attention mechanism. and the second low-frequency component The eleventh intermediate feature is obtained. ; The eleventh intermediate feature Perform a 2x downsampling to obtain the twelfth intermediate feature. The twelfth intermediate feature Perform a 2x downsampling to obtain the thirteenth intermediate feature. The thirteenth intermediate feature Perform a 2x downsampling to obtain the fourteenth intermediate feature. The fourteenth intermediate feature Perform a 2x downsampling to obtain the fifteenth intermediate feature. ; The fifteenth intermediate feature The input is fed into multiple stacked convolutional blocks to obtain the sixteenth intermediate feature output by the multiple stacked convolutional blocks. ; The sixteenth intermediate feature Perform upsampling by 2x to obtain the seventeenth intermediate feature. According to the seventeenth intermediate feature With the fifteenth intermediate feature The eighteenth intermediate feature is obtained. The eighteenth intermediate feature The acquisition process includes: ;in It's pixel attention. It is channel attention. , All are learnable weights; The eighteenth intermediate feature and the seventeenth intermediate feature The nineteenth intermediate feature is obtained by performing upsampling by 2x on each feature and then fusing the features after upsampling. According to the nineteenth intermediate feature With the fourteenth intermediate feature The twentieth intermediate feature was obtained. The twentieth intermediate feature The acquisition process includes: ; The twentieth intermediate feature and the nineteenth intermediate feature The twenty-first intermediate feature is obtained by performing upsampling by a factor of 2 and fusing the features after upsampling. According to the twenty-first intermediate feature With the thirteenth intermediate feature The twenty-second intermediate feature was obtained. The twenty-second intermediate feature The acquisition process includes: ; The second twelfth intermediate feature and the twenty-first intermediate feature The low-frequency features are obtained by performing upsampling by 2x and fusing the features after upsampling. .

4. The image enhancement method for laser speckle based on deep learning according to claim 2, characterized in that, The aggregation of the first high-frequency component and the second high-frequency component The corresponding features yield high-frequency features. ,include: Aggregating the first high-frequency component based on an attention mechanism and the second high-frequency component The tenth intermediate feature is obtained. ; Based on U-net from the tenth intermediate feature Extract the high-frequency features .

5. The image enhancement method for laser speckle based on deep learning according to claim 1, characterized in that, The high-frequency guiding feature and the low-frequency guiding features The enhanced laser speckle image is reconstructed. ,include: Calculate the first image Local variance ;in, The first image; ； ； in, To integrate the high-frequency guiding features and the low-frequency guiding features The resulting fusion features; The enhanced laser speckle image is obtained by reconstructing the fused features using a 1x1 convolutional layer. .

6. The image enhancement method for laser speckle based on deep learning according to claim 1, characterized in that, The first image Extracting speckle features ,include: The teacher model is trained based on laser speckle training samples so that the teacher model learns the speckle texture of the laser speckle training samples; The feature knowledge of the teacher model is distilled into the student model, enabling the student model to learn the distribution pattern and edge features of the speckle texture; According to the student model from the first image Extracting speckle features .

7. The image enhancement method for laser speckle based on deep learning according to claim 1, characterized in that, Extracting the second image The second high frequency component Second low frequency component include: Extracting the second image based on a 3×3 convolution kernel and the Sobel operator. The second high frequency component ; Based on the second image Subtract the second high-frequency component The second low-frequency component was obtained. ; Extract the first image First high frequency component and the first low-frequency component include: Extracting the first image based on a 3×3 convolution kernel and the Sobel operator. First high frequency component ; Based on the first image Subtract the first high-frequency component The first low-frequency component was obtained. .

8. An image enhancement device for laser speckle based on deep learning, characterized in that, The device includes: The instruction response module is used to determine the first image in response to a speckle image enhancement instruction. Second image The first image The first image is the original laser speckle image, and the second image is... It is the first image The image obtained after performing logarithmic bilateral filtering; The high- and low-frequency component extraction module is used to extract the first image respectively. First high frequency component and the first low-frequency component and extracting the second image The second high frequency component Second low frequency component ; The high- and low-frequency feature extraction module is used to aggregate the first high-frequency component. and the second high-frequency component The corresponding features yield high-frequency features. and aggregating the first low-frequency component and the second low-frequency component The corresponding features yield low-frequency features. ; The feature knowledge guidance module is used to pre-train the base model based on laser speckle training samples from the first image. Extracting speckle features And based on the speckle characteristics For the high frequency features Knowledge guidance was conducted to obtain high-frequency guidance characteristics. and based on the speckle characteristics Regarding the low-frequency features Knowledge guidance was conducted to obtain low-frequency guidance characteristics. ; The speckle image reconstruction module is used to reconstruct the image based on the high-frequency guiding features. and the low-frequency guiding features The enhanced laser speckle image is reconstructed. .

9. An electronic device, characterized in that, It includes at least one controller and a memory for communicatively connecting with the controller; the memory stores instructions executable by the at least one controller, which, when executed by the at least one controller, causes the at least one controller to perform a deep learning-based laser speckle image enhancement method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions for causing a computer to perform a deep learning-based laser speckle image enhancement method as described in any one of claims 1 to 7.

Images