A method for automatically determining whether to remove noise

Abstract

The present invention discloses a method for automatically judging whether to perform denoising, comprising the following steps: S1: scanning a sample using a light field microscopy system to obtain a light field microscopy image with 4-dimensional information; wherein the sample is a sample to be 3D reconstructed; S2: rearranging the light field microscopy image to obtain a multi-view rearranged image; S3: calculating a cleanliness score of the multi-view rearranged image; judging whether to perform denoising on the multi-view rearranged image based on the cleanliness score of the multi-view rearranged image. The present invention can automatically judge whether to perform denoising, which is not only efficient but also more objective in judgment results compared to manual judgment.

Description

Technical Field

[0001] The present invention relates to the technical field of light field microscope systems, and more particularly to a method for automatically determining whether to remove noise. Background Art

[0002] As an emerging three-dimensional imaging technology, light field microscopy can not only obtain high-resolution three-dimensional reconstructed images, but also achieve fast and non-destructive imaging. Therefore, it has broad application prospects in the biomedical field.

[0003] However, due to the complexity of the scanned samples and the limitations of the light field microscope system itself, the final reconstructed 3D image is often affected by various noises, such as scattering noise, background noise, and detector noise. Severe noise will not only reduce the quality of the 3D reconstructed image, but may also cause distortion of the 3D reconstructed image, thus affecting subsequent analysis and processing. Therefore, it is particularly important to quickly and accurately determine whether the light field microscope image obtained by scanning the sample with the light field microscope system needs to be denoised.

[0004] At present, whether the light field microscopy images obtained by scanning samples with a light field microscopy system need to be denoised depends entirely on the personal experience of the system user, which is not only inefficient but also highly subjective.

[0005] Therefore, how to provide a method for automatically determining whether to perform denoising, which is not only efficient but also provides a more objective determination result, is an urgent problem to be solved by those skilled in the art. Summary of the invention

[0006] In view of this, an object of the present invention is to provide a method for automatically determining whether to perform denoising, which is not only efficient but also provides a more objective determination result.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A method for automatically determining whether to remove noise comprises the following steps:

[0009] S1: Scanning a sample using a light field microscopy system to obtain a light field microscopy image with 4D information; wherein the sample is a sample to be 3D reconstructed;

[0010] S2: rearrange the light field microscopic image to obtain a multi-view rearranged image;

[0011] S3: Calculating the cleanliness score of the multi-view rearranged image;

[0012] Whether to perform denoising processing on the multi-view rearranged image is determined based on the cleanliness score of the multi-view rearranged image.

[0013] Preferably, S3 specifically includes:

[0014] S31: selecting a central perspective image, an image adjacent to the left side of the central perspective image, and an image adjacent to the right side of the central perspective image from the multi-perspective rearranged image using the microlens array parameters used by the scanning light field microscopy system;

[0015] S32: obtaining a cleanliness score of the multi-view rearranged image based on the central perspective image, an image adjacent to the left of the central perspective image, and an image adjacent to the right of the central perspective image;

[0016] S33: Compare the cleanliness score of the multi-view rearranged image with a preset threshold:

[0017] If the cleanliness score of the multi-view rearranged image is greater than or equal to the preset threshold, the multi-view rearranged image does not need to be denoised;

[0018] If the cleanliness score of the multi-view rearranged image is less than the preset threshold, executing S34;

[0019] S34: Divide the rearranged images of the three perspectives, namely, the central perspective image, the image adjacent to the left of the central perspective image, and the image adjacent to the right of the central perspective image, into n*n sub-images; wherein, , H represents the pixel height of the central perspective image, the image adjacent to the left of the central perspective image, and the image adjacent to the right of the central perspective image;

[0020] S35: Calculate the cleanliness scores of the three perspective images corresponding to each sub-image to obtain an image cleanliness score matrix;

[0021] S36: Compare the maximum value in the image cleanliness score matrix with the preset threshold value:

[0022] If the maximum value in the image cleanliness score matrix is ​​greater than or equal to the preset threshold, then

[0023] The multi-view rearranged image does not need to be denoised;

[0024] If the maximum value in the image cleanliness score matrix is ​​less than the preset threshold, then

[0025] The perspective rearranged image needs to be denoised.

[0026] Preferably, S32 specifically includes the following steps:

[0027] S321: Calculate a structural similarity measure value between the central viewing angle image and an image adjacent to the left side of the central viewing angle image;

[0028] Calculating a structural similarity measure value between the central perspective image and an image adjacent to the right side of the central perspective image;

[0029] S322: Calculate the average of the two structural similarity measurement values ​​in S321, and use the average as the cleanliness score of the multi-view rearranged image.

[0030] Preferably, in S35, the following steps are used to calculate the cleanliness scores of the three perspective images corresponding to the m-th sub-image;

[0031] S351: Calculate the structural similarity metric value of the mth sub-image in the central perspective image and the mth sub-image in the adjacent image on the left side of the central perspective image; where 1<=m<=n 2 ;

[0032] Calculating a structural similarity measure value between an m-th sub-image in the central perspective image and an m-th sub-image in an adjacent image on the right side of the central perspective image;

[0033] S352: Calculate the average of the two structural similarity measurement values ​​in S351, and use the average as the cleanliness score of the three perspective images corresponding to the m-th sub-image.

[0034] Preferably, the structural similarity metric values ​​in S321 and S351 are obtained using the following formula:

[0035] ;

[0036] Where x and y represent images for which structural similarity measures are to be calculated; Represents the structural similarity measure between image x and image y; Represents the brightness comparison function value of image x and image y, Represents the contrast comparison function value of image x and image y; Represents the structure comparison function value of image x and image y; α, β, and γ represent the relative importance of brightness, contrast, and structure respectively; α> 0; β> 0; γ> 0.

[0037] Preferably, , and The calculation formulas are:

[0038]

[0039] in, Represents the brightness comparison function value of image x and image y, Represents the contrast comparison function value of image x and image y; Represents the structural comparison function value of image x and image y; Represents the average value of all pixel values ​​in image x; Represents the average value of all pixel values ​​in image y; Represents the standard deviation of all pixel values ​​in image x; Represents the standard deviation of all pixel values ​​in image y; represents the structure-aware operator; , , is a constant.

[0040] Preferably, and The calculation formulas are:

[0041]

[0042] in, Represents the average value of all pixel values ​​in image x; represents the average value of all pixel values ​​in image y; N represents the total number of pixels in image x and the total number of pixels in image y; Represents the pixel value of the i-th pixel in image x; Represents the pixel value of the i-th pixel in image y.

[0043] Preferably, and The calculation formulas are:

[0044]

[0045] in, Represents the standard deviation of all pixel values ​​in image x; Represents the standard deviation of all pixel values ​​in image y; Represents the average value of all pixel values ​​in image x; represents the average value of all pixel values ​​in image y; N represents the total number of pixels in image x and the total number of pixels in image y; Represents the pixel value of the i-th pixel in image x; Represents the pixel value of the i-th pixel in image y.

[0046] Preferably, The calculation formula is:

[0047] ;

[0048] in, represents the structure-aware operator; Represents the average value of all pixel values ​​in image x; represents the average value of all pixel values ​​in image y; N represents the total number of pixels in image x and the total number of pixels in image y; Represents the pixel value of the i-th pixel in image x; Represents the pixel value of the i-th pixel in image y.

[0049] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a method for automatically determining whether to perform denoising, which can achieve the following beneficial technical effects:

[0050] 1) The present invention can automatically determine whether to perform denoising processing. Compared with the manual judgment method, it is not only efficient but also more objective;

[0051] 2) The present invention assists in realizing the fully automatic three-dimensional reconstruction process of the scanned sample by the light field microscopy system, without the need for manual judgment on whether to perform three-dimensional reconstruction after denoising as in the background art. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0053] Figure 1 A flow chart of a method for automatically determining whether to perform denoising is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] like Figure 1 As shown, the embodiment of the present invention discloses a method for automatically determining whether to remove noise, comprising the following steps:

[0056] S1: Scanning a sample using a light field microscopy system to obtain a light field microscopy image with 4D information; wherein the sample is a sample to be 3D reconstructed;

[0057] S2: rearranging the light field microscopic image to obtain a multi-view rearranged image with unknown noise level;

[0058] S3: Calculating the cleanliness score of the multi-view rearranged image;

[0059] Whether to perform denoising processing on the multi-view rearranged image is determined based on the cleanliness score of the multi-view rearranged image.

[0060] It can be understood that: the parallax of adjacent view images is small, the structure is similar, and the noise of each view image is random and independent;

[0061] It can be understood that determining whether the multi-view rearranged image needs to be denoised is equivalent to determining whether the light field microscopy image needs to be denoised; because the multi-view rearranged image and the light field microscopy image contain the same noise, but the image form presented on the computer is different.

[0062] Furthermore, S3 specifically includes:

[0063] S31: selecting a central perspective image, an image adjacent to the left side of the central perspective image, and an image adjacent to the right side of the central perspective image from the multi-perspective rearranged image using the microlens array parameters used by the scanning light field microscopy system;

[0064] S32: obtaining a cleanliness score of the multi-view rearranged image based on the central perspective image, an image adjacent to the left of the central perspective image, and an image adjacent to the right of the central perspective image;

[0065] Further, S32 specifically includes the following steps:

[0066] S321: Calculate a structural similarity measure value between the central viewing angle image and an image adjacent to the left side of the central viewing angle image;

[0067] Calculating a structural similarity measure value between the central perspective image and an image adjacent to the right side of the central perspective image;

[0068] Furthermore, the structural similarity metric in S321 is obtained using the following formula:

[0069]

[0070] Wherein, x and y represent images whose structural similarity metrics are to be calculated (specifically: when calculating the structural similarity metrics of the central perspective image and the image adjacent to the left of the central perspective image, image x represents the central perspective image, and image y represents the image adjacent to the left of the central perspective image; when calculating the structural similarity metrics of the central perspective image and the image adjacent to the right of the central perspective image, image x represents the central perspective image, and image y represents the image adjacent to the right of the central perspective image); Represents the structural similarity measure between image x and image y; Represents the brightness comparison function value of image x and image y, Represents the contrast comparison function value of image x and image y; Represents the structure comparison function value of image x and image y; α, β, and γ represent the relative importance of brightness, contrast, and structure respectively; α> 0; β> 0; γ> 0; represents the average value of all pixel values ​​in image x, which represents the brightness level of image x; represents the average value of all pixel values ​​in image y, which represents the brightness level of image y; represents the standard deviation of all pixel values ​​in image x, which represents the contrast level of image x; represents the standard deviation of all pixel values ​​in image y, which represents the contrast level of image y; represents the structure-aware operator; , , is a constant used to ensure stability when the denominator is 0; N represents the total number of pixels of image x and the total number of pixels of image y (the total number of pixels of image x is the same as the total number of pixels of image y); Represents the pixel value of the i-th pixel in image x; Represents the pixel value of the i-th pixel in image y.

[0071] S322: Calculate the average of the two structural similarity measurement values ​​in S321, and use the average as the cleanliness score of the multi-view rearranged image.

[0072] S33: Compare the cleanliness score of the multi-view rearranged image with a preset threshold:

[0073] If the cleanliness score of the multi-view rearranged image is greater than or equal to the preset threshold, the multi-view rearranged image does not need to be denoised;

[0074] If the cleanliness score of the multi-view rearranged image is less than the preset threshold, executing S34;

[0075] S34: Divide the rearranged images of the three perspectives, namely, the central perspective image, the image adjacent to the left of the central perspective image, and the image adjacent to the right of the central perspective image, into n*n sub-images; wherein, , H represents the pixel height of the central perspective image, the image adjacent to the left of the central perspective image, and the image adjacent to the right of the central perspective image;

[0076] S35: Calculate the cleanliness scores of the three perspective images corresponding to each sub-image to obtain an image cleanliness score matrix;

[0077] Furthermore, in S35, the following steps are used to calculate the cleanliness scores of the three perspective images corresponding to the m-th sub-image;

[0078] S351: Calculate the structural similarity metric value of the mth sub-image in the central perspective image and the mth sub-image in the adjacent image on the left side of the central perspective image; where 1<=m<=n 2 ;

[0079] Calculating a structural similarity measure value between an m-th sub-image in the central perspective image and an m-th sub-image in an adjacent image on the right side of the central perspective image;

[0080] Furthermore, the structural similarity metric in S351 is obtained using the following formula:

[0081]

[0082] Wherein, x and y represent images whose structural similarity metrics are to be calculated (specifically: when calculating the structural similarity metrics of the mth sub-image in the central perspective image and the mth sub-image in the adjacent image on the left side of the central perspective image, image x represents the mth sub-image in the central perspective image, and image y represents the mth sub-image in the adjacent image on the left side of the central perspective image; when calculating the structural similarity metrics of the mth sub-image in the central perspective image and the mth sub-image in the adjacent image on the right side of the central perspective image, image x represents the mth sub-image in the central perspective image, and image y represents the mth sub-image in the adjacent image on the right side of the central perspective image); Represents the structural similarity measure between image x and image y; Represents the brightness comparison function value of image x and image y, Represents the contrast comparison function value of image x and image y; Represents the structure comparison function value of image x and image y; α, β, and γ represent the relative importance of brightness, contrast, and structure respectively; α> 0; β> 0; γ> 0; represents the average value of all pixel values ​​in image x, which represents the brightness level of image x; represents the average value of all pixel values ​​in image y, which represents the brightness level of image y; represents the standard deviation of all pixel values ​​in image x, which represents the contrast level of image x; represents the standard deviation of all pixel values ​​in image y, which represents the contrast level of image y; represents the structure-aware operator; , , is a constant, which is used to ensure stability when the denominator is 0; N represents the total number of pixels of image x and the total number of pixels of image y (the total number of pixels of image x is the same as the total number of pixels of image y); Represents the pixel value of the i-th pixel in image x; Represents the pixel value of the i-th pixel in image y.

[0083] S352: Calculate the average of the two structural similarity measurement values ​​in S351, and use the average as the cleanliness score of the three perspective images corresponding to the m-th sub-image.

[0084] It is understandable that: as the parallax between adjacent perspectives increases with the increase of sample thickness, the structural similarity will also decrease. In order to prevent the influence of excessive parallax caused by excessive sample thickness, the present invention performs a secondary judgment when the cleanliness score of the multi-perspective rearranged image is less than the preset threshold.

[0085] S36: Compare the maximum value in the image cleanliness score matrix with the preset threshold value:

[0086] If the maximum value in the image cleanliness score matrix is ​​greater than or equal to the preset threshold, then

[0087] The multi-view rearranged image does not need to be denoised;

[0088] If the maximum value in the image cleanliness score matrix is ​​less than the preset threshold, then

[0089] The perspective rearranged image needs to be denoised.

[0090] It can be understood that: when the method of the present invention is used for three-dimensional reconstruction of samples, when it is automatically determined that the multi-view rearranged image does not need to be denoised, the multi-view rearranged image is directly three-dimensionally reconstructed; when it is automatically determined that the multi-view rearranged image needs to be denoised, the multi-view rearranged image is input into the denoising network for denoising before three-dimensional reconstruction; obviously, the present invention realizes the fully automatic three-dimensional reconstruction process of the scanned sample by the light field microscope system, without the need for manual judgment of whether to perform three-dimensional reconstruction after denoising as in the background technology.

[0091] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0092] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for automatically determining whether to remove noise, characterized in that: The following steps are involved: S1: Scanning a sample using a light field microscopy system to obtain a light field microscopy image with 4D information; wherein the sample is a sample to be 3D reconstructed; S2: rearrange the light field microscopic image to obtain a multi-view rearranged image; S3: Calculating the cleanliness score of the multi-view rearranged image; Determining whether to perform denoising processing on the multi-view rearranged image based on the cleanliness score of the multi-view rearranged image; S3 specifically includes: S31: selecting a central perspective image, an image adjacent to the left side of the central perspective image, and an image adjacent to the right side of the central perspective image from the multi-perspective rearranged image using the microlens array parameters used by the scanning light field microscopy system; S32: obtaining a cleanliness score of the multi-view rearranged image based on the central perspective image, an image adjacent to the left of the central perspective image, and an image adjacent to the right of the central perspective image; S33: Compare the cleanliness score of the multi-view rearranged image with a preset threshold: If the cleanliness score of the multi-view rearranged image is greater than or equal to the preset threshold, the multi-view rearranged image does not need to be denoised; If the cleanliness score of the multi-view rearranged image is less than the preset threshold, executing S34; S34: Divide the rearranged images of the three perspectives, namely, the central perspective image, the image adjacent to the left of the central perspective image, and the image adjacent to the right of the central perspective image, into n*n sub-images; wherein, H represents the pixel height of the central perspective image, the image adjacent to the left side of the central perspective image, and the image adjacent to the right side of the central perspective image; S35: Calculate the cleanliness scores of the three perspective images corresponding to each sub-image to obtain an image cleanliness score matrix; S36: Compare the maximum value in the image cleanliness score matrix with the preset threshold value: If the maximum value in the image cleanliness score matrix is ​​greater than or equal to the preset threshold, the multi-view rearranged image does not need to be denoised; If the maximum value in the image cleanliness score matrix is ​​less than the preset threshold, the multi-view rearranged image needs to be denoised.

2. The method for automatically determining whether to remove noise according to claim 1, characterized in that: S32 specifically includes the following steps: S321: Calculate a structural similarity measure value between the central viewing angle image and an image adjacent to the left side of the central viewing angle image; Calculating a structural similarity measure value between the central perspective image and an image adjacent to the right side of the central perspective image; S322: Calculate the average of the two structural similarity measurement values ​​in S321, and use the average as the cleanliness score of the multi-view rearranged image.

3. A method for automatically determining whether to remove noise according to claim 1 or 2, characterized in that: In S35, the following steps are used to calculate the cleanliness scores of the three perspective images corresponding to the m-th sub-image; S351: Calculate the structural similarity metric value of the mth sub-image in the central perspective image and the mth sub-image in the adjacent image on the left side of the central perspective image; wherein 1<=m<=n 2 ; Calculating a structural similarity measure value between an m-th sub-image in the central perspective image and an m-th sub-image in an adjacent image on the right side of the central perspective image; S352: Calculate the average of the two structural similarity measurement values ​​in S351, and use the average as the cleanliness score of the three perspective images corresponding to the m-th sub-image.

4. The method for automatically determining whether to remove noise according to claim 3, characterized in that: The structural similarity measure in S321 and S351 is obtained using the following formula: SSIM(x,y)=[l(x,y)] α ·[c(x,y)] β ·[s(x,y)] γ ; Where x and y represent images for which structural similarity measures are to be calculated; SSIM (x, y) represents the structural similarity measure between image x and image y; l (x, y) represents the brightness comparison function value of image x and image y, c(x, y) represents the contrast comparison function value of image x and image y; s(x, y) represents the brightness comparison function value of image x and image y. y The structure comparison function value; α, β and γ represent the relative importance of brightness, contrast and structure respectively; α>0; β>0; γ>0.

5. The method for automatically determining whether to remove noise according to claim 4, characterized in that: l The calculation formulas for (x, y), c(x, y) and s(x, y) are respectively; Where, l(x, y) represents the brightness comparison function value of image x and image y, c(x, y) represents the contrast comparison function value of image x and image y; s(x, y) represents the structure comparison function value of image x and image y; μ x Represents the average value of all pixels in image x; μ y Represents the average value of all pixel values ​​in image y; σ x Represents the standard deviation of all pixel values ​​in image x; σ y Represents the standard deviation of all pixel values ​​in image y; σ xy Represents the structure-aware operator; C1, C2, and C3 are constants.

6. The method for automatically determining whether to remove noise according to claim 5, characterized in that: μ x and μ y The calculation formulas are: Among them, μ x Represents the average value of all pixel values ​​in image x; μ y represents the average value of all pixel values ​​in image y; N represents the total number of pixels in image x and the total number of pixels in image y; x i Represents the pixel value of the i-th pixel in image x; y i Represents the pixel value of the i-th pixel in image y.

7. The method for automatically determining whether to remove noise according to claim 6, characterized in that: σ x and σ y The calculation formulas are: Among them, σ x Represents the standard deviation of all pixel values ​​in image x; σ y Represents the standard deviation of all pixel values ​​in image y; μ x Represents the average value of all pixel values ​​in image x; μ y Represents the average value of all pixel values ​​in image y; N represents the total number of pixels in image x and the total number of pixels in image y; xi represents the pixel value of the i-th pixel in image x; yi represents the pixel value of the i-th pixel in image y.

8. The method for automatically determining whether to remove noise according to claim 6, characterized in that: σ xy The calculation formula is: Among them, σ xy represents the structure-aware operator; μ x Represents the average value of all pixel values ​​in image x; μ y represents the average value of all pixel values ​​in image y; N represents the total number of pixels in image x and the total number of pixels in image y; x i Represents the pixel value of the i-th pixel in image x; y i Represents the pixel value of the i-th pixel in image y.

Images