A microscope real-time image optimization method and system

Abstract

The present invention discloses a microscope real-time image optimization method and system, which comprises: step 1, obtaining the noise point ratio coefficient of the original image under various imaging conditions, calculating the noise point ratio coefficient P of the microscope system system ; Step 2, deduct the original image with the P system Calculate the corresponding pixel points to obtain a first image; step 3, obtain the maximum grayscale value of the first image; step 4, set the pixel point value on the original image that is greater than the maximum grayscale value of the first image as the maximum grayscale value of the first image, and keep the other pixel point values ​​unchanged to obtain an optimized image. The present invention has the advantages of significant denoising effect, fast calculation speed, and high signal fidelity, and is very suitable for image optimization processing in real-time imaging of microscopes to improve imaging quality.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a microscope real-time image optimization method and system. Background Art

[0002] Microscopic imaging is one of the important technical means to study life sciences. Super-resolution microscopic imaging technology can reveal the mysteries of life at the cellular scale, and structured light super-resolution imaging technology can achieve long-term real-time observation of the movement and coordination of cells and organelles due to its low phototoxicity, high utilization rate of fluorescent photons, and high temporal resolution. In real-time microscopic imaging, the fluorescence signal of the object is transmitted through the optical system, then collected by a precision camera, and finally displayed on the terminal software through electrical signal transmission. The dynamic image of the cell is displayed in the terminal software. In the process of signal transmission, it is inevitable that due to the influence of noise, especially the electronic interference of the camera, when the final image is formed, there will be noise on the image display, which affects the quality of the final image display.

[0003] The commonly used denoising methods are time-consuming and have poor real-time performance. Therefore, developing a method for fast denoising of real-time microscope images is of great significance for real-time imaging, especially super-resolution real-time imaging.

[0004] There is currently a median filter algorithm that can be used to remove image noise by taking the median of the pixel size in the pixel area. The main methods of the median filter algorithm include: taking an n*n pixel matrix in the area centered on each pixel point in the image, sorting the pixel sizes in the matrix, and taking the median to replace the pixel value size of the center point of the matrix, thereby achieving the effect of removing image noise. The median filter algorithm has a good effect on impulse noise and can also eliminate salt and pepper noise, but it is less effective in eliminating Gaussian noise. In the actual imaging process of the microscope system, there is a certain probability that Gaussian noise points will appear, which affects the quality of the real-time microscope image, and the use of the median filter algorithm to remove noise is time-consuming. Therefore, the algorithm has certain limitations and is not suitable for real-time microscope systems. Summary of the invention

[0005] The object of the present invention is to provide a microscope real-time image optimization method and system, which can be used for rapid noise removal during microscope imaging.

[0006] To achieve the above object, the present invention provides a microscope real-time image optimization method, the method comprising:

[0007] Step 1: Obtain the noise ratio coefficient of the original image under various imaging conditions and calculate the noise ratio coefficient P of the microscope system. system ;

[0008] Step 2, obtain the original image after deducting the Psystem The maximum grayscale value G of the image after the relevant noise max ;

[0009] Step 3: remove the original image that is larger than G max The pixel value is set to G max , while the original image is lower than G max The pixel value remains unchanged to obtain the optimized image.

[0010] Furthermore, the multiple imaging conditions include one or more of imaging field of view, imaging exposure time and laser power.

[0011] Furthermore, the multiple imaging conditions include imaging field of view, imaging exposure time and laser power at the same time;

[0012] The P gystem Calculated by formula (1):

[0013] P system =αP roi +βP exposure +γP LaserPower +C (1)

[0014] Among them, P roi represents the noise ratio coefficient of the preset imaging field of view, and α represents the noise ratio of the preset imaging field of view to the P system The weight of the proportion; P exposure represents the noise ratio coefficient of the preset imaging exposure time, and β represents the effect of the preset imaging exposure time on the P system The weight of the proportion; P LaserPower represents the noise point ratio coefficient of the preset laser power, and γ represents the effect of the preset laser power on the P system The weight of the proportion of dark current shot noise is represented by C.

[0015] Furthermore, α, β, and γ all have values ​​in the range of (0-1).

[0016] Furthermore, the microscope real-time image optimization method also includes:

[0017] Step 4: transform the two-dimensional matrix I of the original image pixels into origin Convert to a one-dimensional array I linear , the I linear The pixels in the array are sorted by pixel value to obtain a one-dimensional array I sort :I linear =R(I origin ), I sort =S(I linear ), where R() and S() represent functions;

[0018] The step 3 is described by formula (2):

[0019] G max =G[I sort (row×col×(1-P system ))] (2)

[0020] Among them, G[] max The acquisition function G[] represents the maximum grayscale value of the image, which means obtaining the pixel value of the corresponding pixel point. max Indicates the maximum grayscale value of the image, I sort It represents a one-dimensional array after sorting the gray value matrix I of the original image from small to large, and row×col represents the resolution of the original image.

[0021] The present invention also provides a microscope real-time image optimization system, the system comprising:

[0022] The noise ratio coefficient calculation unit is used to obtain the noise ratio coefficient of the original image under various imaging conditions and calculate the noise ratio coefficient P of the microscope system. system ;

[0023] An image processing unit is used to obtain the original image after the P system The maximum grayscale value G of the image after the relevant noise max ;

[0024] The image optimization unit is used to optimize the original image max The pixel value is set to G max , while the original image is lower than G max The pixel value remains unchanged to obtain the optimized image.

[0025] Furthermore, the multiple imaging conditions include one or more of imaging field of view, imaging exposure time and laser power.

[0026] Furthermore, the multiple imaging conditions include imaging field of view, imaging exposure time and laser power at the same time;

[0027] The P system Calculated by formula (1):

[0028] P system =αP roi +βP exposure +γP LaserPower +C ( 1 )

[0029] Among them, P roirepresents the noise ratio coefficient of the preset imaging field of view, and α represents the noise ratio of the preset imaging field of view to the P system The weight of the proportion; P exposure represents the noise ratio coefficient of the preset imaging exposure time, and β represents the effect of the preset imaging exposure time on the P system The weight of the proportion; P LaserPower represents the noise point ratio coefficient of the preset laser power, and γ represents the effect of the preset laser power on the P system The weight of the proportion of dark current shot noise is represented by C.

[0030] Furthermore, α, β, and γ all have values ​​in the range of (0-1).

[0031] Furthermore, the microscope real-time image optimization system also includes:

[0032] An image preprocessing unit is used to convert the two-dimensional matrix I of the original image pixels into origin Convert to a one-dimensional array I linear , the I linear The pixels in the array are sorted by pixel value to obtain a one-dimensional array I sort :I linear =R(I origin ), I sort =S(I linear ), where R() and S() represent functions;

[0033] The image optimization unit is described by formula (2):

[0034] G max =G[I sort (row×col×(1-P system ))] (2)

[0035] Among them, G[] max The acquisition function G[] represents the maximum grayscale value of the image, which means obtaining the pixel value of the corresponding pixel point. max Indicates the maximum grayscale value of the image, I sort It represents a one-dimensional array after sorting the gray value matrix I of the original image from small to large, and row×col represents the resolution of the original image.

[0036] The present invention uses statistical analysis to determine the proportion of noise points with strong correlation affecting the image quality to the total number of image pixels, and after a large number of tests and statistics, obtains the noise point ratio coefficient with strong correlation of the microscope system. Through actual imaging tests, after one imaging is completed, the obtained system noise point ratio coefficient is deducted, and the maximum grayscale value and the minimum value of the image after deducting the noise points with strong correlation are remapped. Therefore, the present invention not only has a low impact on signal fidelity and a significant noise point deduction effect, but also has low time complexity and fast calculation speed, and can fully meet the real-time requirements in the microscopic imaging process, making it convenient for users to track the dynamic process of cells and improve the quality of imaging experiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic flow chart of a microscope real-time image optimization method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0039] like Figure 1 As shown, the microscope real-time image optimization method provided by the embodiment of the present invention includes:

[0040] Step 1: Obtain the noise ratio coefficient of the original image under various imaging conditions and calculate the noise ratio coefficient P of the microscope system. system .

[0041] Step 2, obtain the original image after deducting the P system The maximum grayscale value G of the image after the relevant noise max .

[0042] Step 3: remove the original image that is larger than G max The pixel value is set to G max , while the original image is lower than G max The pixel value remains unchanged to obtain the optimized image.

[0043] This embodiment starts from the composition structure of the micro-microscope system and analyzes the noise influence on the signal during the transmission process. It is mainly affected by three types of noise, namely, shot noise related to the signal, shot noise related to the dark current (also called dark noise), and readout noise. For the readout noise, it satisfies the Gaussian distribution, and for the micro-microscope system, the dark noise accounts for a very small and fixed proportion, and its proportion can be considered to be a constant C. The readout noise is related to the camera readout speed, and the noise point proportion coefficient is obtained by comprehensively considering the imaging field of view and exposure time of the system. The shot noise related to the signal can obtain its noise point proportion coefficient by counting the noise points under different laser powers. Considering the stability of the microscope system, a large amount of experimental imaging data is combined for statistical analysis to obtain the noise point proportion coefficient of the system with high correlation to the imaging quality. Therefore, in one embodiment, the multiple imaging conditions in step 1 can be understood as including the imaging field of view, imaging exposure time and laser power at the same time, or as a combination of one of the three imaging conditions or any two imaging conditions, or as conditions other than the three imaging conditions.

[0044] For any imaging condition listed in the above embodiments, the corresponding noise point ratio coefficient can be obtained by, but not limited to, using the following method.

[0045] For example, under the conditions of the longest exposure time and the lowest laser power of a given system, the noise ratio coefficient is statistically analyzed considering different imaging fields of view.

[0046]

[0047] Among them, P roi It represents the noise ratio coefficient of the system under the given imaging field of view, N i Indicates the number of noise points in the i-th image, M i represents the number of pixels in the i-th image, and total represents the number of image data counted. The noise ratios of different imaging fields are different, and the corresponding noise ratio coefficients of the microscope system can be calculated for different imaging fields.

[0048] Under the condition of selecting the largest imaging field of view and the lowest laser power, the noise ratio coefficient is statistically analyzed under different exposure times.

[0049]

[0050] Among them, P exposure Represents the noise ratio coefficient under different exposure time imaging conditions, N i Indicates the number of noise points in the i-th image, M iRepresents the number of pixels in the i-th image, and total represents the number of image data counted.

[0051] Under the given maximum imaging field of view and longest exposure time, different laser powers are considered and the noise ratio coefficient is statistically analyzed.

[0052]

[0053] Among them, P LaserPower Represents the noise point ratio coefficient under different laser power imaging conditions, N i Indicates the number of noise points in the i-th image, M i Represents the number of pixels in the i-th image, and total represents the number of image data counted.

[0054] In one embodiment, the multiple imaging conditions include imaging field of view, imaging exposure time, and laser power. system Calculated by formula (1):

[0055] P system =αP roi +βP exposure +γPL aserPower +C (1)

[0056] Among them, P roi represents the noise ratio coefficient of the preset imaging field of view, and α represents the noise ratio of the preset imaging field of view to the P system The weight of the proportion; P exposure represents the noise ratio coefficient of the preset imaging exposure time, and β represents the effect of the preset imaging exposure time on the P system The weight of the proportion; P LaserPower represents the noise point ratio coefficient of the preset laser power, and γ represents the effect of the preset laser power on the P system The weight of the dark current shot noise is represented by C, which is a very small constant and is usually 0.

[0057] The value range of α, β, and γ is (0-1). The values ​​of α, β, and γ are different in different microscope systems. For different microscope systems and imaging conditions, it is necessary to actually observe the effects of different values, and then determine the optimal weight to obtain the best real-time microscopic imaging image denoising effect. It can also be understood that for a specific microscope system, the optimal α, β, and γ values ​​can be given by statistically analyzing the noise ratio coefficient under different imaging conditions, so as to be applied to the real-time preview image optimization processing of the microscope system. Ideally, the values ​​of α, β, and γ are all 1. When the imaging condition is that the imaging field of the microscope system is the largest and the exposure time is the longest, the values ​​of α and β can be approximately 0, and the value of γ can be 1; when the imaging condition is that the imaging field of the microscope system is the largest and the laser power is the lowest, the values ​​of α and γ can be approximately 0, and the value of β can be 1; when the imaging condition is that the exposure time of the microscope system is the longest and the laser power is the lowest, the values ​​of β and γ can be approximately 0, and the value of α can be 1.

[0058] In one embodiment, the microscope real-time image optimization method further comprises:

[0059] Step 4: transform the two-dimensional matrix I of the original image pixels into origin Convert to a one-dimensional array I linear , the I linea The pixels in r are sorted by pixel value to obtain a one-dimensional array I sort :I linear =R(I origin ), I sort =S(I linear ), where R() and S() represent functions.

[0060] The purpose of step 3 is to obtain the maximum grayscale value of the original image calculated by deducting the noise ratio coefficient. Specifically, the two-dimensional pixel matrix of the original image is converted into a one-dimensional array, sorted by size, and then multiplied by the system noise ratio coefficient to obtain the maximum grayscale value of the image after deducting the noise. In other words, step 3 is described by formula (2):

[0061] G max =G[I sort (row×col×(1-P system ))] (2)

[0062] Among them, G[] max represents the acquisition function of the maximum grayscale value of the image, G[] represents the acquisition of the corresponding pixel value, G max Indicates the maximum grayscale value of the image, I sortIt represents a one-dimensional array after sorting the gray value matrix I of the original image from small to large, and row×col represents the resolution of the original image.

[0063] The embodiment of the present invention provides a microscope real-time image optimization system including a noise point ratio coefficient calculation unit, an image processing unit and an image optimization unit, wherein:

[0064] The noise ratio coefficient calculation unit is used to obtain the noise ratio coefficient of the original image under various imaging conditions and calculate the noise ratio coefficient P of the microscope system. system .

[0065] The image processing unit is used to obtain the original image after deducting the P system The maximum grayscale value G of the image after the relevant noise max .

[0066] The image optimization unit is used to optimize the original image max The pixel value is set to G max , while the original image is lower than G max The pixel value remains unchanged to obtain the optimized image.

[0067] In one embodiment, the plurality of imaging conditions include one or more of imaging field of view, imaging exposure time and laser power.

[0068] In one embodiment, the multiple imaging conditions include imaging field of view, imaging exposure time, and laser power. system It is calculated by formula (1).

[0069] In one embodiment, the microscope real-time image optimization system provided by the embodiment of the present invention further includes an image preprocessing unit, the image preprocessing unit is used to convert the two-dimensional matrix I of the original image pixel points into origin Convert to a one-dimensional array I linear , the I linear The pixels in the array are sorted by pixel value to obtain a one-dimensional array I sort :I linear =R(I origin ), I sort =S(I linear ), where R() and S() represent functions.

[0070] The image optimization unit is described by formula (2):

[0071] G max =G[I sort (row×col×(1-P system ))] (2)

[0072] Among them, G[] max represents the acquisition function of the maximum grayscale value of the image, G[] represents the acquisition of the corresponding pixel value, G max Indicates the maximum grayscale value of the image, I sort It represents a one-dimensional array after sorting the gray value matrix I of the original image from small to large, and row×col represents the resolution of the original image.

[0073] The microscope real-time image optimization method and system proposed in the present invention calculate the noise scale coefficient of the microscope system based on the statistics of the noise scale coefficient under different imaging conditions, and then apply this scale coefficient to deduct the noise of the image to obtain the maximum gray value of the image and remap the image. This method not only has a low impact on signal fidelity and a significant noise deduction effect, but also has low time complexity and fast calculation speed, which can fully meet the real-time requirements in the microscopic imaging process, facilitate users to track the dynamic process of cells, and improve the quality of imaging experiments.

[0074] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Those skilled in the art should understand that the technical solutions described in the above embodiments may be modified, or some of the technical features thereof may be replaced by equivalents; these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A microscope real-time image optimization method, characterized in that: include: Step 1: Obtain the noise ratio coefficient of the original image under various imaging conditions and calculate the noise ratio coefficient P of the microscope system. system ; Step 2: transform the two-dimensional matrix I of the original image pixels into origin Convert to a one-dimensional array I linear , the I linear The pixels in the array are sorted by pixel value to obtain a one-dimensional array I sort , where I linear =R(I origin ), I sort =S(I linear ), wherein R () and S () represent functions; obtaining the original image is deducted from the P system The maximum grayscale value G of the image after the relevant noise max ; said G max Through formula (2), we can get: G max =G[I sort (row×col×(1-P system ))] max (2) Among them, G[ ] max represents a function for obtaining the maximum grayscale value of the image, and row×col represents the resolution of the original image; Step 3: remove the original image that is larger than G max The pixel value is set to G max , while the original image is lower than G max The pixel value remains unchanged, and the optimized image is obtained; The multiple imaging conditions also include imaging field of view, imaging exposure time, and laser power; The P system Calculated by formula (1): P system =αP roi +βP exposure +γP LaserPower +C (1) Among them, P roi represents the noise ratio coefficient of the preset imaging field of view, and α represents the noise ratio of the preset imaging field of view to the P system The weight of the proportion; P exposure represents the noise ratio coefficient of the preset imaging exposure time, and β represents the effect of the preset imaging exposure time on the P sytem The weight of the proportion; P LaserPower represents the noise point ratio coefficient of the preset laser power, and γ represents the effect of the preset laser power on the P system The weight of the proportion of dark current shot noise is represented by C.

2. The microscope real-time image optimization method according to claim 1, characterized in that: α, β, and γ all have values ​​in the range of (0-1).

3. A microscope real-time image optimization system, characterized in that: include: The noise ratio coefficient calculation unit is used to obtain the noise ratio coefficient of the original image under various imaging conditions and calculate the noise ratio coefficient P of the microscope system. system ; An image preprocessing unit is used to convert the two-dimensional matrix I of the original image pixels into origin Convert to a one-dimensional array I linear , the I linear The pixels in the array are sorted by pixel value to obtain a one-dimensional array I sort , where I linear =R(I origin ), I sort =S(I linear ), where R() and S() represent functions; An image processing unit is used to obtain the original image after the P system The maximum grayscale value G of the image after the relevant noise max ; The G max Through formula (2), we can get: G max =G[I sort (row×col×(1-P system ))] max (2) Among them, G[ ] max represents a function for obtaining the maximum grayscale value of the image, and row×col represents the resolution of the original image; The image optimization unit is used to optimize the original image max The pixel value is set to G max , while the original image is lower than G max The pixel value remains unchanged, and the optimized image is obtained; The multiple imaging conditions also include imaging field of view, imaging exposure time, and laser power; The P system Calculated by formula (1): P system =αP roi +βP exposure +γP LaserPower +C (1) Among them, P roi represents the noise ratio coefficient of the preset imaging field of view, and α represents the noise ratio of the preset imaging field of view to the P system The weight of the proportion; P exposure represents the noise ratio coefficient of the preset imaging exposure time, and β represents the effect of the preset imaging exposure time on the P system The weight of the proportion; P LaserPower represents the noise point ratio coefficient of the preset laser power, and γ represents the effect of the preset laser power on the P system The weight of the proportion of dark current shot noise is represented by C.

4. The microscope real-time image optimization system according to claim 3, characterized in that: α, β, and γ all have values ​​in the range of (0-1).

Images