STED microscope image processing method and STED microscope system

Abstract

The present invention provides a STED microscope image processing method and a STED microscope system, the method comprising the steps of: obtaining a first STED image by using a first main image, which is obtained by emitting an excitation laser beam and a first STED laser beam at an object being measured, and a first auxiliary image, which is obtained by emitting only the first STED laser beam at the object being measured; obtaining a second STED image by using a second main image, which is obtained by emitting, at the object being measured, the excitation laser beam and a second STED laser beam of an intensity different from that of the first STED laser beam, and a second auxiliary image, which is obtained by emitting only the second STED laser beam at the object being measured; and using the first STED image and the second STED image so as to obtain a final image from which background noise is removed.

Description

STED microscope image processing method and STED microscope system

[0001] The present invention relates to a STED microscope image processing method and a STED microscope system.

[0002] Recently, fluorescence microscopy, which boasts relatively superior vertical resolution compared to conventional optical microscopy, has been applied for imaging samples such as biological samples. Among fluorescence microscopy techniques, stimulated emission depletion (STED) microscopy (hereinafter referred to as "STED microscopy") is a type of fluorescence microscopy that overcomes the diffraction limit. By applying stimulated emission, it achieves resolution far superior to that of conventional fluorescence microscopy.

[0003] These STED microscopes are based on the confocal microscopy method. More specifically, STED microscopes, similar to confocal microscopes, excite fluorescent substances dyed in a sample by incidenting an excitation laser beam, and during this excitation process, the fluorescent substances in the focal region transition to an excited state.

[0004] Conventional fluorescence microscopes capture images by detecting spontaneous emission from excited fluorescent material, and the resolution of the image is determined by the specifications of the lens. In contrast, STED microscopy focuses a donut-shaped depletion beam on the outer edge of the focal plane, resulting in stimulated emission at the outer edge of the excited region and spontaneous emission at the center. By selectively capturing only the spontaneously emitted photons to form an image, an effective point spread function much smaller than that of conventional confocal microscopes can be obtained, resulting in better resolution.

[0005] Meanwhile, images obtained using conventional STED microscopy exhibit specific background noise. This can be categorized into direct excitation noise (DEN) caused by the strong STED laser beam and incomplete depletion noise (IDN) caused by the insufficient suppression of the excitation laser beam by the STED laser beam.

[0006] In images obtained with conventional STED microscopy, background noise may appear stronger than the actual signal, or the presence of background noise may increase the intensity of low-frequency regions, resulting in relative signal loss in high-frequency regions, significantly degrading image quality. Furthermore, DEN and IDN have a conflicting relationship depending on the intensity of the STED laser beam, so reducing one often increases the other.

[0007] The purpose of the present invention is to provide a STED microscope image processing method and a STED microscope system capable of effectively removing background noise by obtaining a STED image using STED laser beams of different intensities and performing image calculations by applying a differential coefficient thereto.

[0008] In order to achieve the above object, one aspect of the present invention provides a STED microscope image processing method, including the steps of: obtaining a first STED image using a first main image obtained by irradiating a measurement target with an excitation laser beam and a first STED laser beam, and a first auxiliary image obtained by irradiating only the first STED laser beam to the measurement target; obtaining a second STED image using a second main image obtained by irradiating the measurement target with a second STED laser beam having a different intensity from the excitation laser beam and the first STED laser beam, and a second auxiliary image obtained by irradiating only the second STED laser beam to the measurement target; and obtaining a final image from which background noise is removed using the first STED image and the second STED image.

[0009] A STED microscope image processing method according to one embodiment of the present invention can effectively suppress two types of background noise (DEN and IDN) that appear in images obtained through conventional STED microscopy. At the same time, it can enhance image resolution, thereby improving image quality and analysis accuracy.

[0010] A STED microscope image processing method according to one embodiment of the present invention can reduce the intensity of a STED laser beam that must be irradiated on a sample to obtain an image with the same resolution, thereby reducing the risk of bleaching of a fluorescent substance and damage to the sample.

[0011] A STED microscope system according to one embodiment of the present invention can obtain images with improved background noise and resolution using only a controller without additional equipment.

[0012] FIG. 1 is a diagram illustrating a STED microscope system according to one embodiment of the present invention.

[0013] FIG. 2 and FIG. 3 are diagrams illustrating a STED microscope image processing method according to one embodiment of the present invention.

[0014] Figure 4 is a conceptual diagram showing the relationship between images acquired at each stage of Figure 2.

[0015] Figures 5 and 6 are drawings comparing images obtained from a conventional microscope system with images obtained from the STED microscope system of Figure 1.

[0016] One aspect of the present invention provides a STED microscope image processing method, comprising: a step of obtaining a first STED image using a first main image obtained by irradiating a measurement target with an excitation laser beam and a first STED laser beam, and a first auxiliary image obtained by irradiating only the first STED laser beam to the measurement target; a step of obtaining a second STED image using a second main image obtained by irradiating the measurement target with a second STED laser beam having a different intensity from the excitation laser beam and the first STED laser beam, and a second auxiliary image obtained by irradiating only the second STED laser beam to the measurement target; and a step of obtaining a final image from which background noise is removed using the first STED image and the second STED image.

[0017] Additionally, the step of obtaining the final image may include a step of determining a differential coefficient using the first STED image and the second STED image, and a step of generating a final image using the differential coefficient, the first STED image, and the second STED image.

[0018] In addition, the step of generating the final image may include the step of obtaining a result value from an intermediate image obtained by performing a first operation using the differential coefficient, the first STED image, and the second STED image, and the step of outputting the intermediate image as the final image when the result value satisfies a preset standard.

[0019] In addition, the step of generating the final image may further include a step of resetting the differential coefficient and re-acquiring the result value from an intermediate image obtained by re-performing the first operation, if the result value does not meet a preset standard.

[0020] Additionally, the first operation may multiply the second STED image by the differential coefficient and subtract it from the first STED image.

[0021] Additionally, the method may further include a step of adjusting the ratio of the intensities of the first STED laser beam and the second STED laser beam based on the background noise score and the resolution score of the final image.

[0022] Additionally, the intensity of the first STED laser beam may be greater than the intensity of the second STED laser beam.

[0023] Another aspect of the present invention provides a STED microscope system, comprising: a first laser generating unit for generating an excitation laser beam; a second laser generating unit for generating a first STED laser beam and a second STED laser beam of different intensities; an optical unit for setting an optical path so that the excitation laser beam, the first STED laser beam, and the second STED laser beam are irradiated to a measurement object; and a controller for controlling the operations of the first laser generating unit and the second laser generating unit to obtain a final image, wherein the controller obtains a plurality of STED images by irradiating at least two of the excitation laser beam, the first STED laser beam, and the second STED laser beam to the measurement object, and obtains a final image from which background noise is removed using the plurality of STED images.

[0024] In addition, the controller can determine a differential coefficient using the plurality of STED images, perform a first operation using the differential coefficient and the plurality of STED images, obtain a result value from an intermediate image obtained by performing the first operation, and output the intermediate image as the final image when the result value satisfies a preset standard.

[0025] Additionally, if the result value does not meet a preset standard, the controller can reset the differential coefficient and obtain a result value from an intermediate image obtained by re-performing the first operation.

[0026] The configuration and operation of the present invention will be described in detail with reference to embodiments of the present invention illustrated in the attached drawings below.

[0027] The present invention is capable of various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, as well as the methods for achieving them, will become clearer with reference to the embodiments described in detail below, along with the drawings. However, the present invention is not limited to the embodiments disclosed below and can be implemented in various forms.

[0028] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same reference numerals and redundant descriptions thereof will be omitted.

[0029] In the examples below, singular expressions include plural expressions unless the context clearly indicates otherwise.

[0030] In the examples below, terms such as “include” or “have” mean that a feature or component described in the specification is present, and do not preclude the possibility that one or more other features or components may be added.

[0031] In some embodiments, where implementations are otherwise feasible, specific process sequences may be performed in a different order than described. For example, two processes described in succession may be performed substantially simultaneously, or in a reverse order from the described order.

[0032] For convenience of explanation, the sizes of components in the drawings may be exaggerated or reduced. For example, the sizes and thicknesses of each component shown in the drawings are arbitrarily indicated for convenience of explanation, and thus the following embodiments are not necessarily limited to those shown.

[0033] FIG. 1 is a drawing illustrating a STED microscope system (10) according to one embodiment of the present invention.

[0034] Referring to FIG. 1, the STED microscope system (10) may include a first laser generation unit (100), a second laser generation unit (200), an optical unit (300), and a controller (400).

[0035] The first laser generating unit (100) can generate an excitation laser beam (EX). The first laser generating unit (100) can emit the excitation laser beam (EX) to a measurement target (SP) containing a fluorescent material. At this time, the first laser generating unit (100) can be an excitation light source that generates the excitation laser beam (EX).

[0036] The second laser generation unit (200) can generate a STED laser beam (STED). The second laser generation unit (200) can be a type of stimulated emission suppression light source that emits a STED laser beam (STED) to a measurement target (SP).

[0037] The second laser generation unit (200) can de-excite the fluorescent material of the measurement target (SP) excited by the excitation laser beam (EX). At this time, the wavelength of the STED laser beam (STED) generated by the second laser generation unit (200) may be different from the wavelength of the excitation laser beam (EX). Specifically, the wavelength of the STED laser beam (STED) may be shorter than that of the excitation laser beam (EX).

[0038] The second laser generation unit (200) can control the intensity of the generated STED laser beam (STED). Since the second laser generation unit (200) can generate STED laser beams (STED) with different intensities, the STED microscope system (10) can obtain a plurality of different STED images.

[0039] The optical unit (300) may include a light separation unit (310), a phase conversion unit (320), and a detection unit (330). The optical unit (300) may align the excitation laser beam (EX) and the STED laser beam (STED) and guide them to a measurement target (SP).

[0040] The optical separation unit (310) may include at least one beam splitter or polarizing beam splitter. The optical separation unit (310) may separate the excitation laser beam (EX) or the STED laser beam and guide them along different paths. For convenience of explanation, the following description will focus on an embodiment in which the optical separation unit (310) includes two polarizing beam splitters as in FIG. 1.

[0041] The optical separation unit (310) may include a first polarizing beam splitter (311) and a second polarizing beam splitter (312).

[0042] The first polarizing beam splitter (311) can separate the STED laser beam (STED) generated from the second laser generating unit (200). At this time, the first polarizing beam splitter (311) can guide the STED laser beam (STED) to be separated into two paths and pass through the first phase plate (321) and the second phase plate (322) which will be described later, respectively.

[0043] The first polarizing beam splitter (311) can transmit or reflect the STED laser beam (STED) depending on the polarization state of the STED laser beam (STED). The STED laser beam (STED) split into two paths by the first polarizing beam splitter (311) can pass through the first phase plate (321) and the second phase plate (322), which will be described later, and pass through the second polarizing beam splitter (312) with different phases.

[0044] The second polarizing beam splitter (312) can recombine the STED laser beam (STED) separated by the first polarizing beam splitter (311). In addition, the second polarizing beam splitter (312) can guide the laser beam (EX) here toward the measurement target (SP).

[0045] The phase conversion unit (320) may be equipped with a plurality of phase plates. For convenience of explanation, the following description will focus on an example having three phase plates as in FIG. 1.

[0046] The phase conversion unit (320) may include a first phase plate (321), a second phase plate (322), and a third phase plate (323).

[0047] The first phase plate (321) can convert the phase on the plane of the P-polarized STED laser beam so that it has a donut-shaped phase with a light intensity of 0 in the central region.

[0048] The second phase plate (322) can convert the axial phase of the S-polarized STED laser beam so that it has a donut-shaped phase with a light intensity of 0 in the central region.

[0049] The third phase plate (323) can generate a phase difference between the STED laser beam (STED) and the excitation laser beam (EX). Specifically, the third phase plate (323) can include at least one of a half-wave plate (HWP) or a quarter-wave plate (QWP), and FIG. 1 illustrates an embodiment in which the third phase plate (323) includes both a half-wave plate (HWP) (3231) and a quarter-wave plate (QWP) (3232).

[0050] The detection unit (330) may include a lens module (331), a dichroic mirror (332), and a sensor module (333). The detection unit (330) may detect an image of a measurement target (SP) using an excitation laser beam (EX) or a STED laser beam (STED) guided to the measurement target (SP) by the optical unit (300).

[0051] The lens module (331) can focus the beam reflected from the dichroic mirror (332) described later onto the measurement target (SP). In addition, the lens module (331) can focus the light (FL) fluoresced by the laser beam (EX) or the STED laser beam (STED) from the measurement target (SP) and guide it to the dichroic mirror (332). At this time, the lens module (331) can include an objective lens.

[0052] A dichroic mirror (332) can reflect a laser beam (EX) or a STED laser beam (STED) here and guide it to a measurement target (SP). In addition, the dichroic mirror (332) can transmit fluorescent light (FL) emitted from the measurement target (SP) and guide it to a sensor module (333).

[0053] The detection unit (330) may include one or more dichroic mirrors (332). FIG. 1 illustrates an embodiment in which the detection unit includes two dichroic mirrors (332a, 332b). Each dichroic mirror (332a, 332b) may guide the path of light by transmitting or reflecting at least one of an excitation laser beam (EX), a STED laser beam (STED), and fluorescence-emitted light (FL).

[0054] The sensor module (333) can sense fluorescent light (FL) emitted from the measurement target (SP). Information obtained from the sensor module (333) can be transmitted to the controller (400).

[0055] For example, the sensor module (333) may include a multimode fiber (MMF) and a single photon counting module (SPCM).

[0056] The controller (400) can obtain a final image (FIM) using multiple STED images. The controller (400) can obtain a final image (FIM) with background noise removed based on information received from the sensor module (333), and the algorithm for this will be described in detail below.

[0057] The above describes one embodiment of a STED microscope system (10), but the configuration of the STED microscope system (10) is not limited thereto and may have additional configurations and operating principles. In addition, the positions and relationships of each component in FIG. 1 may be modified or adjusted.

[0058] FIGS. 2 and 3 are diagrams illustrating a STED microscope image processing method according to one embodiment of the present invention. In addition, FIG. 4 is a conceptual diagram illustrating the relationship between images acquired at each stage of FIG. 2, and FIGS. 5 and 6 are diagrams comparing images acquired from a conventional microscope system with images acquired from the STED microscope system (10) of FIG. 1.

[0059] Referring to FIG. 2, the STED microscope image processing method comprises a step (S100) of obtaining a first STED image (IM13) using a first main image (IM11) obtained by irradiating a measurement target (SP) with an excitation laser beam (EX) and a first STED laser beam (STED1), and a first auxiliary image (IM12) obtained by irradiating only the first STED laser beam (STED1) to the measurement target (SP), a step (S200) of obtaining a second STED image (IM23) using a second main image (IM21) obtained by irradiating the measurement target (SP) with a second STED laser beam (STED2) having a different intensity from the excitation laser beam (EX) and the first STED laser beam (STED1), and a second auxiliary image (IM22) obtained by irradiating only the second STED laser beam (STED2) to the measurement target (SP), and a step (S200) of obtaining a second STED image (IM23) using the first STED image (IM13) and the second STED image (IM23). It may include a step (S300) of obtaining a final image (FIM) from which background noise has been removed.

[0060] Hereinafter, 'DEN' refers to the direct excitation noise that appears in the STED image due to the strong STED laser beam, and 'IDN' refers to the incomplete depletion noise that appears when the STED laser beam does not sufficiently suppress the excitation laser beam (EX).

[0061] In the step (S100) of acquiring the first STED image (IM13), the first STED image (IM13) with DEN removed can be primarily acquired using the first main image (IM11) and the first auxiliary image (IM12).

[0062] The first main image (IM11) can be obtained by irradiating the measurement target (SP) with a laser beam (EX) and a first STED laser beam (STED1).

[0063] When the excitation laser beam (EX) is irradiated onto the measurement target (SP), the electrons of the fluorescent material (fluorophore) of the measurement target (SP) can be excited to have a high energy level. The first STED laser beam (STED1) can be irradiated onto the same measurement target (SP) with a donut-shaped cross-section by a phase conversion unit, and the energy level of the electrons excited by the excitation laser beam (EX) can be lowered. At this time, energy having the same wavelength as the first STED laser beam (STED1) is emitted, and the fluorescent material in the area corresponding to the donut-shaped cross-section of the first STED laser beam (STED1) has its original fluorescence emission phenomenon suppressed (depleted) and a fluorescence reaction can occur only in the central local area. Through this, the STED microscope system (10) can acquire the first main image (IM11), and at this time, the first main image (IM11) can include both DEN and IDN, two types of background noise.

[0064] The first auxiliary image (IM12) can be acquired by irradiating only the first STED laser beam (STED1) to the measurement target (SP). When acquiring the first auxiliary image (IM12), the excitation laser beam (EX) is not irradiated to the measurement target (SP), so the IDN, which occurs because the first STED laser beam (STED1) does not sufficiently suppress the excitation laser beam (EX), does not appear. Meanwhile, the DEN appears by the first STED laser beam (STED1) as in the first main image (IM11), and at this time, the amount of DEN may be substantially the same as the amount of DEN appearing in the first main image (IM11).

[0065] The first STED image (IM13) can be acquired using the first main image (IM11) and the first auxiliary image (IM12). In this case, the first STED image (IM13) may be an image from which DEN has been primarily removed.

[0066] In one embodiment, the first STED image (IM13) can be obtained by subtracting the first main image (IM11) and the first auxiliary image (IM12). Since the first main image (IM11) and the first auxiliary image (IM12) have the same amount of DEN, subtracting the first auxiliary image (IM12) from the first main image (IM11) can obtain the first STED image (IM13) in which only the main signal and IDN appear.

[0067] In the step (S200) of acquiring a second STED image (IM23), a second STED image (IM23) with DEN primarily removed can be acquired using a second main image (IM21) and a second auxiliary image (IM22).

[0068] The second main image (IM21) can be obtained by irradiating the measurement target (SP) with an excitation laser beam (EX) and a second STED laser beam (STED2). At this time, the intensity of the excitation laser beam (EX) may be the same as the intensity of the excitation laser beam (EX) irradiated to obtain the first main image (IM11). Meanwhile, the intensity of the second STED laser beam (STED2) may be set differently from the intensity of the first STED laser beam (STED1).

[0069] The second main image (IM21) may include both types of background noise, DEN and IDN. Since the intensities of the first STED laser beam (STED1) and the second STED laser beam (STED2) are different, the amount of DEN and IDN appearing in the second main image (IM21) may be different from the amount of DEN and IDN appearing in the first main image (IM11).

[0070] In one embodiment, the intensity of the second STED laser beam (STED2) may be set to be weaker than the intensity of the first STED laser beam (STED1). In this case, since the IDN is background noise that appears when the STED laser beam does not sufficiently suppress the excitation laser beam (EX), the amount of IDN appearing in the second STED image (IM23) may be greater than the amount of IDN appearing in the first STED image (IM13).

[0071] The method of obtaining the second main image (IM21) and the second auxiliary image (IM22) and using them to obtain the second STED image (IM23) will be described with reference to the description of the step (S100) of obtaining the first STED image (IM13) described above.

[0072] Referring to FIGS. 3 and 4, in the step (S300) of obtaining a final image (FIM), the final image (FIM) may be obtained using a first STED image (IM13) obtained using a first main image (IM11) and a first auxiliary image (IM12), and a second STED image (IM23) obtained using a second main image (IM21) and a second auxiliary image (IM22). At this time, the final image (FIM) may be an image from which both the DEN and the IDN have been removed.

[0073] The step (S300) of obtaining the final image (FIM) may include a step (S310) of determining a differential coefficient (γ) using the first STED image (IM13) and the second STED image (IM23), and a step (S320) of generating the final image (FIM) using the differential coefficient (γ), the first STED image (IM13), and the second STED image (IM23).

[0074] In the step (S310) of determining the differential coefficient (γ), the differential coefficient (γ) can be set to obtain an intermediate image to be described later. For example, the controller (400) can set an arbitrary value as the initial differential coefficient (γ), and the initial differential coefficient (γ) can be applied to an operation to initially obtain an intermediate image. In addition, if the result value to be described later does not meet a preset standard, the controller (400) can set the differential coefficient (γ) again and apply it to an operation to re-obtain an intermediate image.

[0075] In the step (S320) of generating the final image (FIM), the determined differential coefficient (γ), the first STED image (IM13), and the second STED image (IM23) can be used to obtain the final image (FIM) from which both DEN and IDN are removed.

[0076] The step (S320) of generating a final image (FIM) may include a step (S321) of obtaining a result value from an intermediate image obtained by performing a first operation using the differential coefficient (γ), the first STED image (IM13), and the second STED image (IM23), and a step (S322) of outputting the intermediate image as the final image (FIM) when the result value satisfies a preset standard.

[0077] In the step of obtaining a result value (S321), the determined differential coefficient (γ) is applied to the first operation to obtain an intermediate image from the first STED image (IM13) and the second STED image (IM23).

[0078] In one embodiment, the first operation may be a method of multiplying the second STED image (IM23) by a difference coefficient (γ) and subtracting it from the first STED image (IM13). In this case, the image obtained by performing the first operation may be defined as an intermediate image, and a result value may be calculated to determine whether to output the intermediate image as the final image (FIM).

[0079] In one embodiment, the result value can be set to various variables to determine whether background noise has been sufficiently removed. For example, the result value can be the minimum value of the point spread function (PSF) of the intermediate image. Alternatively, the result value can be the average value of the point spread function (PSF) of the intermediate image. For convenience of explanation, the following description focuses on an embodiment in which the minimum value of the point spread function (PSF) of the intermediate image is set as the result value.

[0080] In the step (S322) of outputting as a final image (FIM), if the result value satisfies a preset standard, the intermediate image can be output as a final image (FIM).

[0081] In one embodiment, the controller (400) can determine and output the intermediate image as the final image (FIM) if the result value has a positive value close to 0 or 0.

[0082] Specifically, when performing the first operation of multiplying the second STED image (IM23) by the differential coefficient (γ) and then subtracting it from the first STED image (IM13), if the differential coefficient (γ) is set too small, the IDN may not be sufficiently removed, so the minimum pixel value may appear as a positive number. In addition, if the differential coefficient (γ) is set too large at this time, the main signal may also be removed, and the minimum pixel value may appear as a negative number. Therefore, the intermediate image can be output as the final image (FIM) based on whether the result value has a value of 0 or a sufficiently small positive number.

[0083] The step (S320) of generating the final image (FIM) may further include a step (S323) of resetting the differential coefficient (γ) and re-acquiring the result value from the intermediate image obtained by re-performing the first operation if the result value does not meet the preset standard.

[0084] In one embodiment, if the result value has a positive value, the differential coefficient (γ) can be increased and the first operation can be performed again. At this time, the amount by which the differential coefficient (γ) is increased can be appropriately set based on the result value.

[0085] In one embodiment, if the result value has a negative value, the differential coefficient (γ) may be reduced and the first operation may be performed again. At this time, the amount of reduction in the differential coefficient (γ) may be appropriately set based on the result value.

[0086] Hereinafter, an embodiment is described in which the controller (400) sets the initial differential coefficient (γ) value to 0 and acquires the final image (FIM) by increasing the differential coefficient (γ) value by 0.001. However, as mentioned above, the initially set differential coefficient (γ) value, the amount of change in the differential coefficient (γ), and the first operation method are not limited thereto.

[0087] The controller (400) can initially perform the first operation by setting the differential coefficient (γ) value to 0. At this time, the first operation is to multiply the second STED image (IM23) by the differential coefficient (γ) and subtract it from the first STED image (IM13). If the differential coefficient (γ) is 0, the intermediate image obtained as a result of the first operation appears identical to the first STED image (IM13).

[0088] Thereafter, the controller (400) can repeat the execution of the first operation and acquisition of intermediate images by increasing the differential coefficient (γ) value by 0.001. As the differential coefficient (γ) value increases, the amount subtracted from the first STED image (IM13) increases, so that more IDNs can be removed. However, as explained above, the amount of change in the differential coefficient (γ) value is not limited to this and can be set in various ways considering conditions such as the computational load.

[0089] When the intermediate image obtained while updating the differential coefficient (γ) value has a negative result, that is, the minimum value of the image pixel values, the IDN may be excessively removed. Therefore, the controller (400) may determine that the intermediate image obtained by applying the previous differential coefficient (γ) value is an optimal image with the IDN removed as much as possible and output it as the final image (FIM). In other words, the controller (400) may repeat the process of resetting the differential coefficient (γ) and obtaining the intermediate image until the result becomes negative.

[0090] The STED microscope image processing method may further include a step (S400) of adjusting the ratio (η) of the intensities of the first STED laser beam (STED1) and the second STED laser beam (STED2) based on the background noise score and the resolution score of the final image (FIM).

[0091] In the step (S400) of adjusting the ratio (η) of the intensities of the first STED laser beam (STED1) and the second STED laser beam (STED2), the ratio (η) of the intensities of the first STED laser beam (STED1) and the second STED laser beam (STED2) can be adjusted to achieve a balanced effect of suppressing background noise and improving resolution.

[0092] Since the two effects of background noise suppression and resolution enhancement are in conflict with each other depending on the ratio (η) of the intensities of the STED laser beams, the controller (400) can assign weights to the background noise score and the resolution score respectively depending on the purpose of using the image. In addition, the controller (400) can obtain the intensity ratio when the total score defined based on the two scores reflecting the weights is maximized, and determine it as the ratio (η) of the intensities of the first STED laser beam (STED1) and the second STED laser beam (STED2). From this, the STED microscope system (10) can obtain an optimal final image (FIM) that implements the two effects in a balanced manner.

[0093] Specifically, the background noise score and the resolution score can be defined using various parameters of the point spread function (PSF) of the image. Since the point spread function of the final image (FIM) is formed differently depending on the ratio (η) of the intensities of the first STED laser beam (STED1) and the second STED laser beam (STED2), the parameters used to define the background noise score and the resolution score can have as variables the ratio (η) of the intensities of the first STED laser beam (STED1) and the second STED laser beam (STED2).

[0094] In one embodiment, the background noise score may be defined as a normalized value of the signal-to-background noise ratio (SBR) of the jump spread function. Furthermore, the resolution score may be defined as a normalized value of the full-width at half maximum (FWHM) of the jump spread function, minus 1.

[0095] However, the definitions of background noise score and resolution score are not limited to this, and can be defined in various ways that can quantitatively calculate the effect of background noise suppression and resolution enhancement.

[0096] If the weight given to the background noise score is defined as the first weight and the weight given to the resolution score is defined as the second weight, the sum of the first and second weights is defined to have a constant value of 1, and accordingly, when one increases, the other decreases.

[0097] The weights assigned to the background noise score and resolution score can be varied, depending on the intended use of the final image. For example, the controller (400) may select weights based on a pre-obtained table, depending on the intended use of the image. Alternatively, the controller (400) may input and utilize weights appropriately selected based on the user's judgment regarding the intended use of the image.

[0098] In order to determine the ratio (η) of the intensities of the first STED laser beam (STED1) and the second STED laser beam (STED2), the controller (400) can calculate the η value when the total score calculated by applying weights to the background noise score and the resolution score is maximized. At this time, the controller (400) can calculate the η value using a numerical analysis method.

[0099] The total score can be defined based on the background noise score, the resolution score, and each weight. For example, the total score can be obtained by multiplying each value by the power of the background noise score and the resolution score, as shown in Equation 1.

[0100]

[0101] In mathematical expression 1, Q is the total score, is the background noise score, is the resolution score, and represents the weights assigned to the background noise score and resolution score, respectively. Meanwhile, mathematical expression 1 is an example of the definition of the total score, and the total score can be defined in various ways depending on the numerical analysis method, etc.

[0102] The ratio (η) of the intensities of the first STED laser beam (STED1) and the second STED laser beam (STED2) can be determined by the η value when the total score calculated by reflecting weights on the background noise score and the resolution score respectively is maximum. Since the total score is a function with η as a variable, the controller (400) can determine the ratio (η) of the intensities of the first STED laser beam (STED1) and the second STED laser beam (STED2) by calculating the η value when the total score is maximum.

[0103] Fig. 5 is a drawing comparing images obtained through a conventional microscope and the STED microscope system (10) of Fig. 1, using fluorescent beads arranged at regular intervals as measurement targets (SP). In addition, Fig. 6 is a drawing comparing images obtained through a conventional microscope and the STED microscope system (10) of Fig. 1, using a microtubule sample having a fluorescent substance as a measurement target (SP).

[0104] Referring to FIG. 5, the image (CONF) obtained through a conventional confocal microscope, the first main image (IM11) obtained through a conventional STED microscope, and the first STED image (IM13) obtained using the first main image (IM11) and the first auxiliary image (IM12) do not clearly show each fluorescent bead arranged at a certain interval, and multiple fluorescent beads appear as one object.

[0105] Meanwhile, the final image (FIM) obtained through the STED microscope system (10) of Fig. 1 can distinguish and identify each fluorescent bead. In addition, it can be confirmed that each fluorescent bead appears more distinctly in the final image (FIM) obtained through the STED microscope system (10) of Fig. 1 not only in the xy direction image but also in the xz direction image.

[0106] Likewise, referring to FIG. 6, the final image (FIM) obtained through the STED microscope system (10) of FIG. 1 clearly shows each microtubule more clearly than the first main image (IM11) obtained through the conventional STED microscopy and the first STED image (IM13) obtained using the first main image (IM11) and the first auxiliary image (IM12). In addition, it can be confirmed that the cross-section of the microtubule appears more clearly in the final image (FIM) in the xz direction than in the image obtained through the conventional microscopy.

[0107] From this, it can be confirmed that the STED microscope system (10) of FIG. 1 has improved resolution and background noise suppression ability compared to existing microscope systems, and that the axial resolution is also improved, so that images of better quality can be obtained.

[0108] Through the differential coefficient (γ) and STED laser beam intensity ratio (η) determined in the STED microscope image processing method, the controller (400) can calculate the full width at half maximum (FWHM) of the jump spread function (PSF) of the final image (FIM).

[0109] First, the FWHM of a general confocal microscope can be calculated as in mathematical equation 2.

[0110]

[0111] Here, n represents the refractive index of the medium, α represents the half-angle of light incident on the objective lens, and λ represents the wavelength of light.

[0112] The FWHM of a conventional STED microscope can be calculated as in Equation 3 by applying an additional variable to Equation 2.

[0113]

[0114] Here, ζ is the ratio of the intensity of the STED laser beam to the minimum intensity of the STED laser beam required to suppress fluorescence. Accordingly, it can be seen that the resolution of the STED microscope improves as the intensity of the STED laser beam increases.

[0115] In a STED microscope system (10) according to one embodiment of the present invention, the controller (400) can calculate FWHM as in mathematical expression 4.

[0116]

[0117] Since both γ and η are values ​​between 0 and 1, by substituting them into mathematical expression 4 and calculating the FWHM, it can be confirmed that the STED microscope system (10) of the present invention has a resolution that is improved even more than that of a conventional STED microscope.

[0118] The STED microscope image processing method of the present invention can effectively suppress the characteristic background noise (DEN, IDN) that appears in images obtained by a STED microscope. Furthermore, it can simultaneously enhance image resolution, thereby improving image quality.

[0119] While the present invention has been described with reference to the embodiments illustrated in the drawings, these are merely exemplary, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible. Therefore, the true scope of technical protection of the present invention should be determined by the technical spirit of the appended claims.

[0120] The specific implementations described in the examples are exemplary and do not limit the scope of the examples in any way. Furthermore, unless specifically stated as "essential," "important," or the like, an element may not be absolutely necessary for the application of the present invention.

[0121] The use of the term "above" and similar referential terms in the specification of embodiments (especially the claims) may refer to both singular and plural. Furthermore, if a range is described in the embodiments, it is intended that the invention includes individual values ​​within the range (unless otherwise stated), and is equivalent to describing each individual value constituting the range in the detailed description. Finally, unless the order of steps constituting a method according to an embodiment is explicitly stated or otherwise stated to the contrary, the steps may be performed in any suitable order. The embodiments are not necessarily limited by the order in which the steps are described. The use of all examples or exemplary terms (e.g., "etc.") in the embodiments is merely intended to describe the embodiments in detail, and the scope of the embodiments is not limited by the examples or exemplary terms, unless otherwise defined by the claims. Furthermore, those skilled in the art will recognize that various modifications, combinations, and variations may be made within the scope of the appended claims or their equivalents, depending on design conditions and factors.

[0122] According to one embodiment of the present invention, a STED microscope image processing method and a STED microscope system are provided. Furthermore, embodiments of the present invention can be applied to techniques for acquiring and processing microscope images.

Claims

1. A step of obtaining a first STED image using a first main image obtained by irradiating a measurement target with a laser beam and a first STED laser beam, and a first auxiliary image obtained by irradiating only the first STED laser beam to the measurement target; A step of obtaining a second STED image using a second main image obtained by irradiating the measurement target with the second STED laser beam having a different intensity from the first STED laser beam and the second auxiliary image obtained by irradiating only the second STED laser beam to the measurement target; and A method for processing STED microscopy images, comprising: obtaining a final image from which background noise has been removed using the first STED image and the second STED image.

2. In paragraph 1, The steps of obtaining the above final image are: A step of determining a differential coefficient using the first STED image and the second STED image; and A method for processing STED microscopy images, comprising: generating the final image using the differential coefficient, the first STED image, and the second STED image.

3. In paragraph 2, The steps for generating the final image are: A step of obtaining a result value from an intermediate image obtained by performing a first operation using the above differential coefficient, the first STED image, and the second STED image; and A STED microscope image processing method, comprising: a step of outputting the intermediate image as the final image when the above result value satisfies a preset criterion.

4. In paragraph 3, The steps for generating the final image are: A STED microscope image processing method, further comprising: a step of resetting the differential coefficient and re-acquiring the result value from an intermediate image obtained by re-performing the first operation if the result value does not meet a preset standard.

5. In paragraph 3, The above first operation is, A method for processing STED microscopy images, wherein the second STED image is multiplied by the differential coefficient and subtracted from the first STED image.

6. In paragraph 1, A STED microscope image processing method, further comprising: a step of adjusting the ratio of the intensities of the first STED laser beam and the second STED laser beam based on the background noise score and the resolution score of the final image.

7. In paragraph 1, A method for processing STED microscopy images, wherein the intensity of the first STED laser beam is stronger than the intensity of the second STED laser beam.

8. A first laser generating unit that generates a laser beam; A second laser generating unit for generating a first STED laser beam and a second STED laser beam of different intensities; An optical unit that sets an optical path so that the above-described laser beam, the first STED laser beam, and the second STED laser beam are irradiated to a measurement target; and A controller for controlling the operation of the first laser generating unit and the second laser generating unit to obtain a final image; The above controller, Obtaining multiple STED images by irradiating at least one of the above-mentioned laser beam, the first STED laser beam, and the second STED laser beam to the above-mentioned measurement target, A STED microscope system that obtains a final image by removing background noise using the above multiple STED images.

9. In paragraph 8, The above controller, Using the above multiple STED images, the differential coefficient is determined, Using the above differential coefficient and the plurality of STED images, a result value is obtained from an intermediate image obtained by performing a first operation, A STED microscope system that outputs the intermediate image as the final image when the above result value satisfies a preset criterion.

10. In paragraph 9, The above controller, A STED microscope system that obtains a result value from an intermediate image obtained by resetting the differential coefficient and re-performing the first operation when the above result value does not meet a preset standard.

Images