An entropy-guided real-time laser speckle imaging method, device and system

Abstract

This application provides an entropy-guided real-time laser speckle imaging method, apparatus, and system, relating to the field of image processing technology. The method involves calculating the contrast of a laser speckle image of a target biological sample to obtain a first contrast image; downsampling the first contrast image at n downsampling rates to obtain n second contrast images; extracting the local entropy of each region in both the first and second contrast images; identifying regions containing blood vessels in both the first and second contrast images based on the extracted local entropy; and performing noise reduction on both the first and second contrast images to obtain multiple denoised images. These denoised images are then fused to obtain a third contrast image. By applying this application, real-time laser speckle imaging can be achieved with low sampling frame rates while maintaining image quality comparable to that with high sampling frame rates.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an entropy-guided real-time laser speckle imaging method, apparatus and system. Background Technology

[0002] Laser speckle imaging is an important non-contact, non-destructive imaging technique widely used in medical imaging, biological experiments, and industrial inspection. Its principle utilizes the coherent properties of laser light to illuminate the surface of biological tissue. Under the influence of moving scatterers (such as blood cells), time-varying speckle images are generated. The speckle contrast in local areas of the image reflects blood flow velocity and microstructure. However, laser speckle imaging images are often affected by various noises, including coherent noise, random noise introduced by the optical system, and environmental background noise. These noises can cause blurring of the speckle image and random fluctuations in the speckle pattern, resulting in poor image quality and affecting the accuracy and reliability of speckle analysis. Furthermore, when using time-contrast algorithms, multiple frames of laser speckle images are required to calculate a single frame of contrast image; for example, 10 frames of laser speckle images are needed to obtain a single high-quality contrast image. Currently, the frame rate of economical 4K cameras is about 60fps. If a high frame rate is used for speckle imaging, that is, more frames are used to calculate the contrast, such as 10 frames, the overall imaging frame rate will drop to 5~6fps, which is difficult to meet the real-time requirements.

[0003] To meet the requirements of real-time imaging, it is often necessary to reduce image resolution or the number of sampling frames, but this can compromise image quality. To improve image quality, spatial filtering or frequency domain filtering is commonly used to denoise laser speckle images. However, while suppressing noise, image details are often blurred, leading to the loss of spatial structure and speckle information. Furthermore, statistical models such as nonlocal means and block matching are frequently used for speckle image denoising, but these algorithms are computationally complex, require complex parameter tuning, and struggle to handle large noise levels. Although deep learning methods based on convolutional neural networks are also commonly used in image denoising, they typically lack the integration of physical information, resulting in limited denoising effectiveness. Moreover, spatial filtering, nonlocal means, and block matching algorithms exhibit poor real-time performance at high resolutions (e.g., 3840×2160); deep learning methods, when processing high-resolution images, require image segmentation, which also impacts real-time performance.

[0004] Therefore, in laser speckle image imaging, how to ensure real-time imaging of laser speckle images while achieving image quality similar to that under high sampling frame rates is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide an entropy-guided real-time laser speckle imaging method, apparatus, and system to ensure real-time imaging of laser speckle images while achieving image quality similar to that of high sampling frame rates, even with low sampling frame rates. The specific technical solution is as follows:

[0006] A first aspect of this application provides a real-time laser speckle imaging method based on entropy guidance, the method comprising:

[0007] The contrast of the laser speckle image of the target biological sample is calculated to obtain the first contrast image;

[0008] The first contrast image is downsampled according to n downsampling rates to obtain n second contrast images, where n≥1;

[0009] Local entropy is extracted from each region in the first contrast image and each of the second contrast images;

[0010] Based on the extracted local entropy, regions containing blood vessels are identified in the first contrast image and each of the second contrast images, respectively.

[0011] The first contrast image and each of the second contrast images are subjected to noise reduction processing to obtain multiple noise-reduced images. In the noise reduction processing, the first noise reduction method is used for the area containing blood vessels, and the second noise reduction method is used for other areas. The first noise reduction method has a lower degree of noise suppression than the second noise reduction method.

[0012] By fusing the denoised images, a third contrast image is obtained.

[0013] A second aspect of this application provides an entropy-guided real-time laser speckle imaging device, the device comprising:

[0014] The contrast calculation module is used to calculate the contrast of the laser speckle image of the target biological sample to obtain the first contrast image;

[0015] The downsampling module is used to downsample the first contrast image according to n downsampling rates to obtain n second contrast images, where n≥1;

[0016] The first extraction module is used to extract the local entropy of each region in the first contrast image and each of the second contrast images, respectively.

[0017] The region determination module is used to determine the regions containing blood vessels in the first contrast image and each of the second contrast images based on the extracted local entropy.

[0018] The noise reduction processing module is used to perform noise reduction processing on the first contrast image and each of the second contrast images respectively to obtain multiple noise-reduced images. In the noise reduction processing, a first noise reduction method is used for the area containing blood vessels, and a second noise reduction method is used for other areas. The first noise reduction method has a lower degree of noise suppression than the second noise reduction method.

[0019] An image fusion module is used to fuse the denoised images to obtain a third contrast image.

[0020] A third aspect of this application provides an entropy-guided real-time laser speckle imaging system, the system including an image acquisition device and a processor;

[0021] The image acquisition device is used to capture laser speckle images;

[0022] The processor is used to implement the steps of the entropy-guided real-time laser speckle imaging method described in the first aspect above.

[0023] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the entropy-guided real-time laser speckle imaging method described in the first aspect above.

[0024] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the entropy-guided real-time laser speckle imaging methods described above.

[0025] Beneficial effects of the embodiments in this application:

[0026] This application provides an entropy-guided real-time laser speckle imaging method, apparatus, and system. It calculates the contrast of laser speckle images of target biological samples and downsamples the contrast images. The resulting multi-scale images capture the features of blood vessels at different resolutions, compensating for inaccurate contrast calculations caused by low sampling frame rates. That is, even if the sampling frame rate is reduced to meet real-time imaging requirements, the solution in this application will not suffer from inaccurate contrast calculations due to low sampling frame rates. Furthermore, inaccurate contrast calculations will not affect image quality. Moreover, by downsampling the contrast images, the amount of image data is reduced, thus reducing the computational load for subsequent local entropy extraction and noise reduction. By simultaneously performing local entropy extraction and noise reduction on each downsampled image, the overall processing time is shortened, further ensuring real-time imaging. By extracting the local entropy of each region in the image at each scale, the contrast value of the region where the blood vessel is located is relatively large due to the blood flow movement, and the texture is usually more complex than other regions. Therefore, the local entropy value of this region will also be higher. Thus, the region where the blood vessel is located and other regions can be accurately identified by the local entropy of each region. Furthermore, although noise in a single image may mask the blood vessel signal at low sampling frame rates, multi-scale local entropy calculation can improve the robustness of blood vessel detection and has a certain tolerance to noise. By applying a first denoising method to areas containing blood vessels and a second denoising method to other areas, the first denoising method suppresses noise less effectively than the second. This allows for the preservation of details within the blood vessel area while suppressing noise in other areas. After fusing the denoised images, the denoising performance of the laser speckle image is improved while reducing detail loss and artifact generation. This achieves image quality similar to traditional high-sampling-frame-count methods at low sampling frame rates, thus ensuring real-time imaging of laser speckle images while maintaining image quality similar to that at high sampling frame rates.

[0027] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0029] Figure 1a This is a schematic diagram of the time contrast algorithm.

[0030] Figure 1b This is a schematic diagram of the spatial contrast algorithm.

[0031] Figure 1c This is a schematic diagram of the spatiotemporal joint contrast algorithm.

[0032] Figure 2 A first schematic diagram of an entropy-guided real-time laser speckle imaging method provided in an embodiment of this application;

[0033] Figure 3 A first flowchart of an entropy-guided real-time laser speckle imaging method provided in an embodiment of this application;

[0034] Figure 4 A second schematic diagram of the entropy-guided real-time laser speckle imaging method provided in the embodiments of this application;

[0035] Figure 5 A second flowchart of the entropy-guided real-time laser speckle imaging method provided in the embodiments of this application;

[0036] Figure 6 A third flowchart of the entropy-guided real-time laser speckle imaging method provided in the embodiments of this application;

[0037] Figure 7a This is a schematic diagram showing the signal-to-noise ratio comparison provided in the embodiments of this application;

[0038] Figure 7b This is a schematic diagram showing the peak signal-to-noise ratio comparison provided in the embodiments of this application;

[0039] Figure 7c This is a schematic diagram showing the structural similarity comparison of embodiments of this application;

[0040] Figure 7d This is a schematic diagram of the images after denoising the original image using different methods;

[0041] Figure 8 This is a comparison chart showing the average running time when using various noise reduction methods to denoise images;

[0042] Figure 9 A schematic diagram of the structure of the entropy-guided real-time laser speckle imaging device provided in the embodiments of this application;

[0043] Figure 10 A schematic diagram of the first structure of the real-time laser speckle imaging system based on entropy guidance provided in the embodiments of this application;

[0044] Figure 11 A schematic diagram of a second structure of a real-time laser speckle imaging system based on entropy guidance provided in an embodiment of this application;

[0045] Figure 12 The fourth flowchart of the entropy-guided real-time laser speckle imaging method provided in the embodiments of this application is shown. Detailed Implementation

[0046] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0047] Laser speckle contrast imaging obtains blood flow images by calculating the contrast of laser speckle images. Existing contrast algorithms include temporal contrast algorithms, spatial contrast algorithms, and spatiotemporal joint contrast algorithms. The principles of each algorithm are as follows: Figure 1a , Figure 1b , Figure 1c As shown in the figure above, W x and W y These are horizontal and vertical windows, respectively. t The time window and longitudinal window are used. Because the spatial resolution of the temporal contrast algorithm is higher than that of the spatial contrast algorithm, most related technologies use the temporal contrast algorithm when calculating contrast. However, the temporal contrast algorithm requires multiple frames of laser speckle images to calculate a single frame of contrast image; for example, 30 frames of laser speckle images are needed to obtain a single high-quality contrast image. Since a 4K camera's frame rate is only 45fps, this high frame rate leads to poor real-time performance when using the temporal contrast algorithm. Using fewer frames of laser speckle images for contrast calculation can improve real-time performance, but image quality will decrease.

[0048] To ensure real-time imaging of laser speckle images with low sampling frame rates while achieving image quality similar to that with high sampling frame rates, a first aspect of this application provides an entropy-guided real-time laser speckle imaging method applied to an electronic device, which can be a server or a terminal device, both of which are reasonable. In practical applications, the terminal device can be a smartphone, tablet computer, desktop computer, etc.

[0049] See Figure 2 , Figure 2 A first schematic diagram of an entropy-guided real-time laser speckle imaging method provided in this application embodiment, the method comprising the following steps:

[0050] Step S101: Calculate the contrast of the laser speckle image of the target biological sample to obtain the first contrast image;

[0051] Step S102: Downsample the first contrast image according to n downsampling rates to obtain n second contrast images;

[0052] Where n is a positive integer greater than or equal to 1;

[0053] Step S103: Extract the local entropy of each region in the first contrast image and each second contrast image respectively;

[0054] Step S104: Based on the extracted local entropy, determine the regions where blood vessels exist in the first contrast image and each of the second contrast images;

[0055] Step S105: Denoise reduction processing is performed on the first contrast image and each of the second contrast images respectively to obtain multiple denoised images;

[0056] In the noise reduction process, the first noise reduction method is used for areas with blood vessels, and the second noise reduction method is used for other areas. The first noise reduction method has a lower degree of noise suppression than the second noise reduction method.

[0057] Step S106: Fuse the denoised images to obtain the third contrast image.

[0058] By employing the embodiments of this application, contrast can be calculated from the laser speckle image of the target biological sample, and the contrast image can be downsampled. The resulting multi-scale image can capture the features of blood vessels at different resolutions, compensating for the inaccuracy in contrast calculation caused by low sampling frame count. That is, even if the sampling frame count is reduced to meet the requirements of real-time imaging, the solution of this application will not suffer from the problem of inaccurate contrast calculation due to low sampling frame count. Furthermore, inaccurate contrast calculation will not affect image quality. Moreover, by downsampling the contrast image, the amount of image data is reduced, thereby reducing the computational load for subsequent local entropy extraction and noise reduction. By simultaneously performing local entropy extraction and noise reduction on each downsampled image, the overall processing time is shortened, further ensuring real-time imaging. By extracting the local entropy of each region in the image at each scale, the contrast value of the region where the blood vessel is located is relatively large due to the blood flow movement, and the texture is usually more complex than other regions. Therefore, the local entropy value of this region will also be higher. Thus, the region where the blood vessel is located and other regions can be accurately identified by the local entropy of each region. Furthermore, although noise in a single image may mask the blood vessel signal at low sampling frame rates, multi-scale local entropy calculation can improve the robustness of blood vessel detection and has a certain tolerance to noise. By applying a first denoising method to areas containing blood vessels and a second denoising method to other areas, the first denoising method suppresses noise less effectively than the second. This allows for the preservation of details within the blood vessel area while suppressing noise in other areas. After fusing the denoised images, the denoising performance of the laser speckle image is improved while reducing detail loss and artifact generation. This achieves image quality similar to traditional high-sampling-frame-count methods at low sampling frame rates, thus ensuring real-time imaging of laser speckle images while maintaining image quality similar to that at high sampling frame rates.

[0059] The following explains steps S101 to S106:

[0060] In step S101 above, the target biological sample can be biological tissue or a part of the animal being studied. It is understood that blood vessels are present in the target biological sample. The laser speckle image of the target biological sample is obtained by an image acquisition device capturing the sample when a highly coherent laser beam is applied to it.

[0061] Contrast calculation for laser speckle images refers to calculating the contrast of each pixel in the image. Specifically, for each pixel, the quotient of the standard deviation and the average value of the pixel grayscale values ​​within a preset region surrounding that pixel is used as the contrast of that pixel. The pixel value of each pixel in the first contrast image is the contrast of that pixel.

[0062] The preset area is set based on practical experience and needs, and its size can be adjusted according to the image resolution to avoid loss of image details.

[0063] In step S102 above, downsampling of the first contrast image can be achieved through average pooling, max pooling, or simple interpolation. The downsampling rate is determined based on requirements and experience; that is, n is determined based on requirements and experience. For example, assuming n is 3, the downsampling rates are 2, 4, and 8, meaning the size of the first contrast image is reduced by a factor of 2, 4, and 8, respectively, resulting in three second contrast images.

[0064] In step S103 above, local entropy refers to the information entropy of each local region, that is, the average information content of each pixel in each local region. It is used to represent the texture complexity or information content of a local region. The information content of each pixel can be calculated from its grayscale value. The higher the local entropy value, the richer the texture of the region, and the higher the probability of it corresponding to a vascular region; conversely, the lower the entropy value, the higher the probability that the region is a smooth region (such as uniform tissue). Local entropy can be represented by the Shannon entropy of each local region, or by the Tsallis entropy (an information entropy) of each layout region; both are acceptable.

[0065] Specifically, one could first calculate the frequency of each pixel's grayscale value appearing in the image, and then calculate the information content of each pixel using the following formula:

[0066]

[0067] Then, by calculating the average information content of each pixel in the local area, the information entropy of the local area can be obtained, and the local entropy of each local area can be determined.

[0068] It is understood that the size of a local region in the first contrast image and the size of a local region in the second contrast image can be the same or different. In one possible implementation, the local region in the second contrast image can be scaled down proportionally according to the downsampling rate compared to the size of the local region in the first contrast image, in order to maintain relative scale.

[0069] In step S104 above, based on the extracted local entropy, regions containing blood vessels are determined in both the first contrast image and each of the second contrast images. This can be determined by threshold comparison. Specifically, an entropy threshold can be preset, and regions with entropy values ​​higher than the threshold are identified as regions containing blood vessels, while regions with entropy values ​​lower than the threshold are identified as other regions, hereinafter referred to as background regions. The entropy threshold can be preset based on practical experience and requirements, or it can be determined statistically by analyzing the information entropy of each pixel in the image; this application does not limit this.

[0070] In another possible implementation, the presence of blood vessels can also be determined by a machine classification model. This machine classification model is based on deep learning and is pre-trained. The machine classification model can combine the local entropy of each local region with features such as shape to determine the presence of blood vessels.

[0071] To improve the efficiency of subsequent noise reduction, in one possible implementation, the regions containing blood vessels and the background regions can be determined using mask mapping. Specifically, after determining the regions containing blood vessels in the image in S104, a binary mask can be generated for each image (the first contrast image and each second contrast image) to segment the blood vessel region and the background region of each image. Then, the blood vessel region and the background region in each image are extracted using the mask. In one possible implementation, a probability map of the mask can be generated using a sigmoid function (an S-shaped curve function), and then a binary mask can be obtained through thresholding.

[0072] In step S105, denoising the first contrast image and each of the second contrast images means that the regions containing blood vessels in the first contrast image and each of the second contrast images are denoised using the first denoising method, and the regions without blood vessels in the first contrast image and each of the second contrast images are denoised using the second denoising method.

[0073] The first noise reduction method suppresses noise less than the second noise reduction method. This means that after denoising the same image using the first and second noise reduction methods, the signal-to-noise ratio (SNR) of the image denoised using the first noise reduction method is less than the SNR of the image denoised using the second noise reduction method.

[0074] The first noise reduction method can be Gaussian filtering, bilateral filtering, or wavelet thresholding, while the second noise reduction method can employ 3D block matching algorithms, median filtering, or nonlocal mean filtering. In one possible implementation, the first noise reduction method is smoothing and noise suppression, and the second noise reduction method is sharpening and enhancement. This approach can suppress noise in the vascular region of the image while avoiding blurring, preserving the details of the vascular region, and ensuring an improved signal-to-noise ratio in the background region of the image.

[0075] For example, for a first contrast image, if local region 1 in the image is a blood vessel region, then the first noise reduction method is applied to denoise local region 1; if local region 2 in the image is a background region, then the second noise reduction method is applied to denoise local region 2.

[0076] It is understandable that after denoising the blood vessel region and background region in the first contrast image and the blood vessel region and background region in each of the second contrast images, the number of denoised images obtained is n+1.

[0077] In step S106 above, a weighted average method can be used to fuse the denoised images. The weights of each denoised image can be determined based on its scale or its information entropy. In another possible implementation, the denoised images corresponding to each second contrast image can be upsampled to make them have the same scale, and then the denoised images can be fused using a weighted average method. The weights of each denoised image are determined through Bayesian optimization.

[0078] The entropy-guided real-time laser speckle imaging method of this application will be described in detail below with reference to the flowchart:

[0079] See Figure 3 , Figure 3 The first flowchart of the entropy-guided real-time laser speckle imaging method provided in this application embodiment is as follows: After inputting the original contrast image (i.e., the first contrast image in step S101), a multi-scale image is constructed to obtain contrast images of different scales, scale 1, scale 2, and scale 3 (corresponding to the aforementioned step S102). For the contrast images of different scales, feature extraction is performed on the image to extract the local entropy in the contrast image (corresponding to the aforementioned step S103). Based on the local entropy in the contrast image, the regions containing blood vessels and background regions in the contrast image are determined (corresponding to the aforementioned step S104). Then, the regions containing blood vessels and background regions in the contrast image are denoised in separate regions (corresponding to the aforementioned step S105) to obtain a denoised image. Finally, the denoised images obtained by denoising the contrast images of different scales are weighted and fused to output a third contrast image (corresponding to the aforementioned step S106).

[0080] To further improve the accuracy of the identified vascular regions, in one possible implementation, local entropy can be combined with other features to determine the vascular regions and background regions in each contrast image. These other features include one or more of gradient magnitude, physical constraints, local range, and structural tensor guidance.

[0081] Based on this, in one possible implementation, such as Figure 4 The diagram shown is a second schematic of the entropy-guided real-time laser speckle imaging method provided in this application embodiment. The method includes the following steps:

[0082] Step S101: Calculate the contrast of the laser speckle image of the target biological sample to obtain the first contrast image;

[0083] Step S102: Downsample the first contrast image according to n downsampling rates to obtain n second contrast images;

[0084] Step S103: Extract the local entropy of each region in the first contrast image and each second contrast image respectively;

[0085] Step S301: Extract other features of each region in the first contrast image and each second contrast image respectively;

[0086] Step S1041: Based on the extracted local entropy and other features, determine the regions where blood vessels exist in the first contrast image and each of the second contrast images, respectively.

[0087] Step S105: Denoise reduction processing is performed on the first contrast image and each of the second contrast images respectively to obtain multiple denoised images;

[0088] Step S106: Fuse the denoised images to obtain the third contrast image.

[0089] Steps S101-S103 and S105-S106 are described above and will not be repeated here. Step S1041 is... Figure 2 One possible implementation of step S104 is described below, with steps S301 and S1041 being explained in detail below.

[0090] In step S301, other features include one or more of gradient magnitude, physical constraints, local range, and structural tensor guidance. These other features are also extracted based on the gray values ​​of each pixel in each contrast image.

[0091] In this context, the physical constraint represents the velocity of blood flow. In one possible implementation, the physical constraint is the blood flow index, which is calculated using the following formula:

[0092]

[0093] in, Indicates blood flow index, For the exposure time, The contrast value is calculated. The blood flow index is directly proportional to the blood flow velocity, that is, the higher the blood flow index, the faster the blood flow velocity. Since blood flow only occurs within the vascular region, i.e., blood flow velocity exists, the region where the blood flow velocity is greater than a preset velocity threshold (or the region where the blood flow index is greater than a preset index threshold) can be defined as the vascular region, and the region where the blood flow velocity is not greater than a preset velocity threshold (or the region where the blood flow index is not greater than a preset index threshold) can be defined as the background region.

[0094] Gradient magnitude represents the intensity and direction of grayscale value change at each pixel, reflecting the edge and texture information of the image. Locations with large gradient magnitudes are usually the boundaries of regions or areas with complex structures and rich textures, such as blood vessel regions. The gradient magnitude of each region can be determined by calculating the average, maximum, or standard deviation of the gradient magnitudes of all pixels within the region. For example, the average gradient magnitude is higher in blood vessel regions, while it is lower in background regions.

[0095] Local range is the difference between the maximum and minimum grayscale values ​​within a given region, used to describe texture roughness and local uniformity. Extracting the local range involves calculating this difference within each region. For vascular regions, the local range is typically larger, while for background regions, it is smaller.

[0096] In one possible implementation, the gradient magnitude can be replaced by structural tensor guidance, which is used to describe the local structural directionality of an image. This can be achieved by calculating the gradient of each region in the X and Y directions, constructing a structural tensor for each pixel to obtain a matrix, and then performing Gaussian smoothing and eigenvalue decomposition on each component of the matrix to obtain the structural tensor guidance for each region.

[0097] In step S1041 above, determining the regions containing blood vessels in the first contrast image and each of the second contrast images based on the extracted local entropy and other features means determining the blood vessel region and background region in the contrast image based on the local entropy and other features of each local region in the contrast image. Specifically, this can be determined by machine learning classification. For example, the local entropy and other features of a certain region in the contrast image are input into a machine classification model, and the region type of the region is output, i.e., whether the region is a blood vessel region or a background region.

[0098] In another possible implementation, the region containing blood vessels in the contrast image can also be determined by performing a first weighted summation on the local entropy and other features in each contrast image. For example, assuming that regions with a weighted summation value greater than 0.9 are identified as blood vessel regions, and the other features are physical constraints, with a preset weight of 0.7 for local entropy and 0.6 for physical constraints, and given that the local entropy in region 1 of contrast image 1 is 0.9 and the physical constraint is 0.8, the weighted summation result is 0.984, thus region 1 is identified as a blood vessel region.

[0099] It is understandable that when performing the first weighted summation of the local entropy in each contrast image and the local entropy and other features of each local region in the contrast image, the weights of the local entropy and other features can be determined through Bayesian optimization or through other means.

[0100] By using the embodiments of this application, the region type of each local region is determined by combining features such as local range, physical constraints, gradient magnitude or structural tensor guidance with local entropy, making the region type classification results more accurate.

[0101] In this case, the flowchart of the entropy-guided real-time laser speckle imaging method provided in the embodiments of this application is as follows: Figure 5 As shown. After inputting the original contrast image (i.e., the first contrast image in step S101), a multi-scale image is constructed to obtain contrast images at different scales: scale 1, scale 2, and scale 3 (corresponding to the aforementioned step S102). For the contrast images at different scales, feature extraction is performed on the images to extract the local entropy and other features in the contrast images (corresponding to the aforementioned steps S103 and S301). Based on the local entropy and other features in the contrast images, the regions containing blood vessels and the background regions in the contrast images are determined (corresponding to the aforementioned step S1041). Then, the regions containing blood vessels and the background regions in the contrast images are denoised separately (corresponding to the aforementioned step S105) to obtain denoised images. Finally, the denoised images obtained by denoising the contrast images at different scales are fused to output the third contrast image (corresponding to the aforementioned step S106).

[0102] In the above text, when the region containing blood vessels in a contrast image is determined by first weighted summation of the local entropy and other features in each contrast image, the blood vessel region and the background region in the image are determined directly based on the value of the weighted result.

[0103] In another possible implementation, for each contrast image, a second weighted summation can be performed based on the extracted local entropy and other features of the contrast image to obtain fusion features. Then, the vascular region and background region in the contrast image can be determined based on the fusion features. It is understood that when performing the second weighted summation on the local entropy in each contrast image and the local entropy and other features of each local region in the contrast image, the weights of the local entropy and other features can be determined by Bayesian optimization or by other means.

[0104] For example, assuming other features are local range, physical constraints, and gradient magnitude, the weights of local entropy, local range, physical constraints, and gradient magnitude of a certain region are determined as a, b, c, and d respectively according to Bayesian optimization. Then, the fusion feature A is calculated as a × local entropy + b × local range + c × physical constraints + d × gradient magnitude. Then, the region type of the region is determined based on the fusion feature A.

[0105] like Figure 6 The diagram shows the third flowchart of the real-time laser speckle imaging method based on entropy guidance provided in this application embodiment. After inputting the original contrast image (i.e., the first contrast image in step S101), a multi-scale image is constructed to obtain contrast images of different scales, scale 1, scale 2, and scale 3 (corresponding to the aforementioned step S102). For the contrast images of different scales, feature extraction is performed on the image, extracting the local entropy, gradient magnitude, physical constraints, and local range in the contrast image (corresponding to the aforementioned steps S103 and S301). Then, the local entropy, gradient magnitude, physical constraints, and local range in the contrast image are weighted and fused (e.g., weighted summation) to obtain fused features. The blood vessel region and background region are determined by mask mapping, and partitioned noise reduction is performed (corresponding to the aforementioned step S105) to obtain denoised images of different scales. Then, the denoised images of different scales are weighted and fused to output the third contrast image (corresponding to the aforementioned step S106).

[0106] It is understandable that, since the contrast images are determined based on the grayscale values ​​of each pixel in the laser speckle image, the first, second, and third contrast images are all monochrome images without color information. To improve the readability of the third contrast image, in one possible implementation, pseudo-color mapping can be performed on the third contrast image to obtain a color image for display to the user. Pseudo-color mapping refers to pseudo-color processing, the basic principle of which is to match each grayscale level of a black-and-white or monochrome image to a point in the color space, thereby mapping the monochrome image into a color image.

[0107] After applying the method of this application, the nonlocal mean denoising algorithm, the 3D block matching algorithm, the wavelet transform denoising algorithm, and the traditional time contrast algorithm to the same image, and obtaining vascular images corresponding to different algorithms, the signal-to-noise ratio, peak signal-to-noise ratio, and structural similarity between each vascular image and the standard image are as follows: Figure 7a , Figure 7b , Figure 7c As shown, by Figure 7a and Figure 7b As shown, after processing the laser speckle imaging image using the laser speckle imaging method provided in this application, the signal-to-noise ratio (SNR) and peak signal-to-noise ratio (PSNR) of the blood vessel image are higher than those obtained using other existing methods. Therefore, it can be considered that the denoising effect of the laser speckle imaging method of this application is better than that of the other four algorithms. Furthermore, combined with Figure 7c It is evident that the laser speckle imaging method described in this application yields images with the highest structural similarity to standard images, where the standard image is a clear vascular image used for comparison. For example, such as... Figure 7d As shown, Figure 7d These are schematic diagrams illustrating the images after denoising the original images using different methods. Figure 7d As can be seen, after denoising the original image using the entropy-guided real-time laser speckle imaging method provided in this application embodiment, the noise in the resulting image is less than the noise in the original image, and also less than the noise in the image obtained after denoising the original image using non-mean filtering, block matching filtering, or wavelet denoising methods. Therefore, the image quality obtained after denoising using the method of this application embodiment is higher than the image quality obtained after denoising using non-mean filtering, block matching filtering, or wavelet denoising methods, and even approaches the image quality of a noise-free image. It is understood that the noise-free image in this application refers to the image obtained through temporal contrast calculation under high sampling frame count conditions, and is not strictly speaking an image without noise. For example, the noise-free image in this application can be a contrast map obtained by using a temporal contrast algorithm on 100 frames of speckle images. In other words, imaging a laser speckle image using the method of this application embodiment can achieve image quality similar to that under high sampling frame count conditions.

[0108] Combination Figure 8 As shown, Figure 8 This is a comparison chart showing the average running time when using various noise reduction methods to denoise images. Figure 8As can be seen, the average running time of the entropy-guided real-time laser speckle imaging method provided in this application embodiment for image denoising is 31.7ms, while the average running times of the non-mean value filtering, wavelet denoising (i.e., wavelet denoising mentioned above), and block matching filtering methods for image denoising are 260.5ms, 1512.8ms, and 243863.9ms, respectively. It is evident that the method provided in this application embodiment can achieve real-time image denoising, ensuring that laser speckle images can be imaged in real time.

[0109] Corresponding to the first aspect mentioned above, a second aspect of the embodiments of this application provides an entropy-guided real-time laser speckle imaging device, such as... Figure 9 The diagram shown is a schematic representation of the entropy-guided real-time laser speckle imaging device provided in an embodiment of this application. The device includes:

[0110] The contrast calculation module 601 is used to calculate the contrast of the laser speckle image of the target biological sample to obtain the first contrast image.

[0111] The downsampling module 602 is used to downsample the first contrast image according to n downsampling rates to obtain n second contrast images, where n≥1;

[0112] The first extraction module 603 is used to extract the local entropy of each region in the first contrast image and each of the second contrast images respectively;

[0113] The region determination module 604 is used to determine the regions containing blood vessels in the first contrast image and each of the second contrast images based on the extracted local entropy.

[0114] The noise reduction processing module 605 is used to perform noise reduction processing on the first contrast image and each of the second contrast images respectively to obtain multiple noise-reduced images. In the noise reduction processing, a first noise reduction method is used for the area containing blood vessels, and a second noise reduction method is used for other areas. The degree of noise suppression by the first noise reduction method is less than that by the second noise reduction method.

[0115] Image fusion module 606 is used to fuse the denoised images to obtain a third contrast image.

[0116] By employing the embodiments of this application, contrast can be calculated from laser speckle images of the target biological sample, and the contrast image can be downsampled. The resulting multi-scale images can capture the features of blood vessels at different resolutions, compensating for the inaccuracy in contrast calculation caused by low sampling frame counts. That is, even if the sampling frame count is reduced to meet real-time imaging requirements, the problem of inaccurate contrast calculation at low sampling frame counts will not exist. Furthermore, by downsampling the contrast image, the amount of image data is reduced, thereby reducing the computational load for subsequent local entropy extraction and noise reduction. By simultaneously performing local entropy extraction and noise reduction on each downsampled image, the overall processing time is shortened, further ensuring real-time imaging. By extracting the local entropy of each region in the image at each scale, the contrast value of the region where the blood vessel is located is relatively large due to the blood flow movement, and the texture is usually more complex than other regions. Therefore, the local entropy value of this region will also be higher. Thus, the region where the blood vessel is located and other regions can be accurately identified by the local entropy of each region. Furthermore, although noise in a single image may mask the blood vessel signal at low sampling frame rates, multi-scale local entropy calculation can improve the robustness of blood vessel detection and has a certain tolerance to noise. By applying a first denoising method to areas containing blood vessels and a second denoising method to other areas, the first denoising method suppresses noise less effectively than the second. This allows for the preservation of details within the blood vessel area while suppressing noise in other areas. After fusing the denoised images, the denoising performance of the laser speckle image is improved while reducing detail loss and artifact generation. This achieves image quality similar to traditional high-sampling-frame-count methods at low sampling frame rates, thus ensuring real-time imaging of laser speckle images while maintaining image quality similar to that at high sampling frame rates.

[0117] In one possible implementation, the device further includes:

[0118] The second extraction module is used to extract the physical constraints of each region in the first contrast image and each of the second contrast images, respectively.

[0119] The region determination module is specifically used for:

[0120] Based on the extracted local entropy and physical constraints, regions containing blood vessels are identified in the first contrast image and each of the second contrast images.

[0121] In one possible implementation, the device further includes:

[0122] The third extraction module is used to extract the gradient magnitude of each region in the first contrast image and each of the second contrast images, respectively;

[0123] The region determination module is specifically used for:

[0124] Based on the extracted local entropy and gradient magnitude, regions containing blood vessels are identified in the first contrast image and each of the second contrast images.

[0125] In one possible implementation, the device further includes:

[0126] The fourth extraction module is used to extract the local range of each region in the first contrast image and each of the second contrast images, respectively;

[0127] The region determination module is specifically used for:

[0128] Based on the extracted local entropy and local range, regions containing blood vessels are identified in the first contrast image and each of the second contrast images.

[0129] In one possible implementation, the device further includes:

[0130] The fifth extraction module is used to extract the gradient magnitude, physical constraints, and local range of each region in the first contrast image and each of the second contrast images, respectively.

[0131] The region determination module is specifically used for:

[0132] Based on the extracted local entropy, gradient magnitude, physical constraints, and local range, regions containing blood vessels are identified in the first contrast image and each of the second contrast images.

[0133] In one possible implementation, the region determination module determines regions containing blood vessels in the first contrast image and each of the second contrast images based on extracted local entropy, gradient magnitude, physical constraints, and local range, including:

[0134] For the first contrast image or each of the second contrast images, the extracted local entropy, gradient magnitude, physical constraints and local range are weighted and summed to obtain the fused features;

[0135] Based on the fusion features, regions containing blood vessels are determined in the first contrast image or in each of the second contrast images.

[0136] In one possible implementation, the first noise reduction method is smoothing and noise suppression, and the second noise reduction method is sharpening and enhancement.

[0137] A third aspect of this application provides an entropy-guided real-time laser speckle imaging system, such as... Figure 10As shown, the system includes an image acquisition device 701 and a processor 702;

[0138] Image acquisition device 701 is used to capture laser speckle images;

[0139] The processor 702 is used to implement the steps of the entropy-guided real-time laser speckle imaging method described in the first aspect above.

[0140] By employing the embodiments of this application, contrast can be calculated from the laser speckle image of the target biological sample, and the contrast image can be downsampled. The resulting multi-scale image can capture the features of blood vessels at different resolutions, compensating for the inaccuracy in contrast calculation caused by low sampling frame count. That is, even if the sampling frame count is reduced to meet the requirements of real-time imaging, the solution of this application will not suffer from the problem of inaccurate contrast calculation due to low sampling frame count. Furthermore, inaccurate contrast calculation will not affect image quality. Moreover, by downsampling the contrast image, the amount of image data is reduced, thereby reducing the computational load for subsequent local entropy extraction and noise reduction. By simultaneously performing local entropy extraction and noise reduction on each downsampled image, the overall processing time is shortened, further ensuring real-time imaging. By extracting the local entropy of each region in the image at each scale, the contrast value of the region where the blood vessel is located is relatively large due to the blood flow movement, and the texture is usually more complex than other regions. Therefore, the local entropy value of this region will also be higher. Thus, the region where the blood vessel is located and other regions can be accurately identified by the local entropy of each region. Furthermore, although noise in a single image may mask the blood vessel signal at low sampling frame rates, multi-scale local entropy calculation can improve the robustness of blood vessel detection and has a certain tolerance to noise. By applying a first denoising method to areas containing blood vessels and a second denoising method to other areas, the first denoising method suppresses noise less effectively than the second. This allows for the preservation of details within the blood vessel area while suppressing noise in other areas. After fusing the denoised images, the denoising performance of the laser speckle image is improved while reducing detail loss and artifact generation. This achieves image quality similar to traditional high-sampling-frame-count methods at low sampling frame rates, thus ensuring real-time imaging of laser speckle images while maintaining image quality similar to that at high sampling frame rates.

[0141] The entropy-guided real-time laser speckle imaging method and system provided in this application will be described in detail below with reference to specific embodiments.

[0142] See Figure 11 , Figure 11This is a schematic diagram of a second structure of an entropy-guided real-time laser speckle imaging system provided in an embodiment of this application. The system includes an image acquisition device 701, a processor 702, a laser source module 703, a power supply module 704, a display module 705, and a phantom or animal experimental platform 706.

[0143] Combination Figure 12 , Figure 12 This is the fourth flowchart of the entropy-guided real-time laser speckle imaging method provided in the embodiments of this application. First, the laser source module 703 emits a highly coherent laser, which, after collimation, is reflected by a beam splitter onto the surface of the tissue to be tested on the phantom or animal experimental platform 706, triggering the image acquisition device 701 to acquire an image (i.e.,...). Figure 12 The laser is irradiated and the camera is triggered to acquire a laser speckle image at a frame rate of 50fps and transmit it to the power module 704. The power module 704 uploads the image to the processor 702, which performs region of interest cropping, speckle contrast calculation, entropy-guided noise reduction (i.e., the aforementioned steps S101 to S106), and pseudo-color mapping. After reconstructing the image, the image is returned to the power module 704. The power module 704 pushes the reconstructed image to the display module 705 for real-time display and storage for user interaction.

[0144] The following detailed description, using a specific embodiment as an example, illustrates the model and structure of each component in the entropy-guided real-time laser speckle imaging system provided in this application. It is understood that this embodiment is merely one possible implementation of the entropy-guided real-time laser speckle imaging system provided in this application, and does not limit the model and structure of each component in the laser speckle imaging system.

[0145] The image acquisition device 701 includes a high-sensitivity CMOS sensor, a filter holder, and a cooling fan. It is a rectangular box (100mm×80mm×60mm) with a filter slot mounted at the front end, located opposite the light source unit and fixed on a C-arm.

[0146] The processor 702 includes a GPU (Graphics Processing Unit) card, a heatsink, and a parallel computing module. It is shaped like a standard dual-slot graphics card (245mm × 120mm × 40mm), plugs into the host PCIe (High-Speed ​​Serial Bus) slot, and shares a chassis with the power module 704. The GPU is secured to the side of the chassis via rails.

[0147] The laser source module 703 includes a 785nm semiconductor laser diode and a beam collimating lens group. It has a cylindrical shell (20mm in diameter and 50mm in height), and the light-emitting end is equipped with a collimating lens tube. It is fixed to the center of the front end of the base and is bolted to the bracket of the image acquisition device 701 through a flange.

[0148] The power module 704 includes a data acquisition card, a power module, and an I / O (input / output) interface board. It is a rectangular aluminum shell (200mm×150mm×50mm) and is installed inside the cabinet. It is connected to the host motherboard through a PCIe card slot. The power module 704 is connected to the image acquisition device 701 via a USB 3.0 cable.

[0149] The display module 705 includes a high-definition display, a touch panel, a control keyboard and mouse, and is placed on the operating table. The display base is connected by a universal rotating bracket.

[0150] The phantom or animal experimental platform 706 is located below the optical path of the system.

[0151] The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0152] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described entropy-guided real-time laser speckle imaging methods.

[0153] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the entropy-guided real-time laser speckle imaging methods described in the above embodiments.

[0154] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0155] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0156] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and system embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0157] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method of real-time laser speckle imaging based on entropy guidance, characterized in that, The method includes: The contrast of the laser speckle image of the target biological sample is calculated to obtain the first contrast image; The first contrast image is downsampled according to n downsampling rates to obtain n second contrast images, where n≥1; Local entropy is extracted from each region in the first contrast image and each of the second contrast images; Based on the extracted local entropy, regions containing blood vessels are identified in the first contrast image and each of the second contrast images, respectively. The first contrast image and each of the second contrast images are denoised to obtain multiple denoised images. In the denoising process, a first denoising method is used for areas containing blood vessels, and a second denoising method is used for other areas. The first denoising method has a lower degree of noise suppression than the second denoising method. By fusing the denoised images, a third contrast image is obtained.

2. The method according to claim 1, characterized in that, The method further includes: Physical constraints are extracted from each region in the first contrast image and each of the second contrast images, respectively. The step of determining the regions containing blood vessels in the first contrast image and each of the second contrast images based on the extracted local entropy includes: Based on the extracted local entropy and physical constraints, regions containing blood vessels are identified in the first contrast image and each of the second contrast images.

3. The method according to claim 1, characterized in that, The method further includes: Extract the gradient magnitude of each region in the first contrast image and each of the second contrast images respectively; The step of determining the regions containing blood vessels in the first contrast image and each of the second contrast images based on the extracted local entropy includes: Based on the extracted local entropy and gradient magnitude, regions containing blood vessels are determined in the first contrast image and each of the second contrast images.

4. The method according to claim 1, characterized in that, The method further includes: The local range of each region in the first contrast image and each of the second contrast images is extracted respectively; The step of determining the regions containing blood vessels in the first contrast image and each of the second contrast images based on the extracted local entropy includes: Based on the extracted local entropy and local range, regions containing blood vessels are identified in the first contrast image and each of the second contrast images.

5. The method according to claim 1, characterized in that, The method further includes: Gradient magnitude, physical constraints, and local ranges of each region in the first contrast image and each of the second contrast images are extracted respectively; The step of determining the regions containing blood vessels in the first contrast image and each of the second contrast images based on the extracted local entropy includes: Based on the extracted local entropy, gradient magnitude, physical constraints, and local range, regions containing blood vessels are identified in the first contrast image and each of the second contrast images.

6. The method according to claim 5, characterized in that, The step of determining the regions containing blood vessels in the first contrast image and each of the second contrast images based on the extracted local entropy, gradient magnitude, physical constraints, and local range includes: For the first contrast image or each of the second contrast images, the extracted local entropy, gradient magnitude, physical constraints and local range are weighted and summed to obtain the fused features; Based on the fusion features, regions containing blood vessels are determined in the first contrast image or in each of the second contrast images.

7. The method according to claim 1, characterized in that, The first noise reduction method is smoothing and noise suppression, and the second noise reduction method is sharpening and enhancement.

8. A real-time laser speckle imaging device based on entropy guidance, characterized in that, The device includes: The contrast calculation module is used to calculate the contrast of the laser speckle image of the target biological sample to obtain the first contrast image; The downsampling module is used to downsample the first contrast image according to n downsampling rates to obtain n second contrast images, where n≥1; The first extraction module is used to extract the local entropy of each region in the first contrast image and each of the second contrast images, respectively. The region determination module is used to determine the regions containing blood vessels in the first contrast image and each of the second contrast images based on the extracted local entropy. The noise reduction processing module is used to perform noise reduction processing on the first contrast image and each of the second contrast images respectively to obtain multiple noise-reduced images. In the noise reduction processing, a first noise reduction method is used for the area containing blood vessels, and a second noise reduction method is used for other areas. The first noise reduction method has a lower degree of noise suppression than the second noise reduction method. An image fusion module is used to fuse the denoised images to obtain a third contrast image.

9. A real-time laser speckle imaging system based on entropy guidance, characterized in that, The system includes an image acquisition device and a processor; The image acquisition device is used to capture laser speckle images; The processor is configured to implement the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-7.

Images