Denoising and deconvolution super-resolution imaging method and apparatus based on photoswitchable molecules

Abstract

The present disclosure relates to the technical field of biological microscopic imaging. Disclosed are a denoising and deconvolution super-resolution imaging method and apparatus based on photoswitchable molecules, and an electronic device and a computer-readable storage medium. The method comprises: acquiring a plurality of fluorescence image sequences, wherein structures to be imaged of images in the fluorescence image sequences are labeled by photoswitchable fluorescent molecules, each sequence is generated by using photoswitchable characteristics of the photoswitchable fluorescent molecules, and each sequence comprises at least one bright-state image and at least one dark-state image; performing denoising processing on each bright-state image and each dark-state image in the plurality of fluorescence image sequences, so as to obtain denoised images; and performing deconvolution processing on the denoised images, so as to obtain corresponding super-resolution images. The present disclosure has relatively low requirements for an imaging device, can be applied to various imaging techniques, and does not limit the size of an imaging region, such that requirements for large-field imaging of fixed biological samples in application scenarios can be met, thereby realizing large-field super-resolution imaging of fixed cells and tissues.

Description

Denoising and deconvolution super-resolution imaging method and device based on photoswitch molecules

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 28, 2023, with application number 202311839967.4 and application name “Denoising and Deconvolution Super-resolution Imaging Method and Device Based on Photo-Switch Molecules”, the entire contents of which are incorporated by reference into the application. Technical Field

[0002] The present disclosure relates to the field of biological microscopic imaging technology, and in particular to a denoising and deconvolution super-resolution imaging method, device, electronic device, and computer-readable storage medium based on photoswitch molecules. Background Art

[0003] Super-resolution imaging technology is the process of converting low-resolution images into high-resolution images using computer algorithms or models. This allows us to see more details and obtain a clearer, more realistic visual experience. In recent years, various super-resolution imaging technologies have emerged, such as stimulated emission depletion microscopy (STED), stochastic optical reconstruction microscopy (STORM), and photoactivated localization microscopy (PALM).

[0004] In fields such as biology and medicine, super-resolution imaging technology has broken the limitations of optical diffraction, enabling researchers to study the fine features of subcellular structures in cells at nanometer-scale resolution. However, existing super-resolution imaging technology requires complex equipment to achieve imaging, resulting in high imaging costs and a small imaging field of view, making it difficult to meet the needs of practical applications for imaging samples with a large field of view.

[0005] Contents of this application

[0006] In response to the above situation, the embodiments of the present disclosure provide a denoising and deconvolution super-resolution imaging method, device, electronic device and computer-readable storage medium based on photoswitch molecules, aiming to solve the above problems or at least partially solve the above problems.

[0007] In a first aspect, the present disclosure provides a method for denoising and deconvolution super-resolution imaging based on photoswitch molecules, the method comprising:

[0008] Acquiring a plurality of fluorescence image sequences, wherein the structures to be imaged in the images of the fluorescence image sequences are labeled with photoswitchable fluorescent molecules; each sequence is generated using the photoswitchable properties of the photoswitchable fluorescent molecules, and the sequence includes at least one bright state image and at least one dark state image;

[0009] performing denoising on each bright-state image and each dark-state image in the plurality of fluorescence image sequences to obtain a denoised image;

[0010] The denoised image is subjected to deconvolution processing to obtain a corresponding super-resolution imaging image.

[0011] In a second aspect, the present disclosure further provides a denoising and deconvolution super-resolution imaging device based on photoswitch molecules, the device comprising:

[0012] an acquisition module for acquiring a plurality of fluorescence image sequences, wherein the structures to be imaged in the images of the fluorescence image sequences are labeled with photoswitchable fluorescent molecules; each sequence is generated using the photoswitchable properties of the photoswitchable fluorescent molecules, and the sequence includes at least one bright state image and at least one dark state image;

[0013] a denoising module, configured to perform denoising on each bright-state image and each dark-state image in the plurality of fluorescence image sequences to obtain a denoised image;

[0014] The deconvolution module is used to perform deconvolution processing on the denoised image to obtain a corresponding super-resolution imaging image.

[0015] In a third aspect, an embodiment of the present disclosure further provides an electronic device comprising: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the steps of the above-mentioned denoising and deconvolution super-resolution imaging method based on photoswitch molecules.

[0016] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device performs the steps of the above-mentioned denoising and deconvolution super-resolution imaging method based on photoswitch molecules.

[0017] By means of the above-described technical solution, the embodiments of the present application provide a denoising and deconvolution super-resolution imaging method, apparatus, electronic device, and computer-readable storage medium based on photoswitch molecules. This method, based on the characteristics of common equipment in the fields of biology and medicine, proposes a highly versatile and robust super-resolution imaging technology solution. Using only an ordinary wide-field microscope, multiple fluorescence image sequences can be acquired by utilizing the principle that photoswitch fluorescent molecules are modulated by light while noise is not. By denoising each bright-state image and each dark-state image in these multiple fluorescence image sequences, the noise data therein can be effectively removed, thereby obtaining a denoised image with an extremely high signal-to-noise ratio. Further deconvolution of the denoised image can improve the resolution of the denoised image, thereby obtaining the corresponding super-resolution image. It can be seen that compared with the existing technology, this solution has lower requirements for imaging equipment, greatly reducing costs, and does not limit the size of the imaging area. It can meet the needs of large-field-of-view imaging of fixed biological samples in application scenarios, realizing large-field super-resolution imaging of fixed cells and tissues.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0020] FIG1 shows a schematic flow chart of a method for denoising and deconvolution of super-resolution imaging based on photoswitch molecules according to an embodiment of the present disclosure;

[0021] FIG2 is a schematic diagram showing a process flow of a denoising and deconvolution super-resolution imaging method based on photoswitch molecules provided by another embodiment of the present disclosure;

[0022] FIG3 shows a laser modulation timing diagram provided by an embodiment of the present disclosure;

[0023] FIG4 shows a comparison of image processing results of direct deconvolution without denoising and deconvolution after denoising, as well as the original image, provided by an embodiment of the present disclosure;

[0024] FIG5 shows a schematic structural diagram of a denoising and deconvolution super-resolution imaging device based on photoswitch molecules provided by an embodiment of the present disclosure;

[0025] FIG6 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] To make the objectives, technical solutions, and advantages of the present disclosure more clear, the technical solutions of the present disclosure will be clearly and completely described below in conjunction with the specific embodiments of the present disclosure and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present disclosure.

[0027] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0028] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that such usage is interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in sequences other than those illustrated or described herein. In addition, the term "including" and its variations are to be interpreted as open-ended terms meaning "including but not limited to."

[0029] Super-resolution imaging technology refers to the process of processing low-resolution images into high-resolution images through computer algorithms or models. It allows us to see more details and obtain a clearer and more realistic visual experience. Various super-resolution imaging technologies in recent years, such as stimulated emission depletion microscopy (STED), stochastic optical reconstruction microscopy (STORM), and photoactivated localization microscopy (PALM), have been developed. In the fields of biology and medicine, super-resolution imaging technology breaks the limitations of the optical diffraction limit, enabling researchers to study the fine features of subcellular structures in cells at a resolution level of nanometers. However, existing super-resolution imaging technology requires complex equipment to achieve imaging, which is costly and has a small imaging field of view, making it difficult to meet the needs of large-field-of-view sample imaging in actual application scenarios. Based on this, this application proposes a denoising and deconvolution super-resolution imaging method, device, electronic device, and computer-readable storage medium based on photoswitch molecules. The present disclosure is described in detail below through specific embodiments.

[0030] To facilitate understanding of this embodiment, a detailed introduction is first given to a denoising and deconvolution super-resolution imaging method based on photoswitch molecules disclosed in an embodiment of the present disclosure. The denoising and deconvolution super-resolution imaging method based on photoswitch molecules provided in an embodiment of the present disclosure is generally performed by a computer device with certain computing capabilities, such as a terminal device, a server, or other processing device. The terminal device may be a user equipment (UE), a mobile device, a user terminal, or a terminal. In some possible implementations, the denoising and deconvolution super-resolution imaging method based on photoswitch molecules can be implemented by a processor invoking computer-readable instructions stored in a memory.

[0031] FIG1 shows a flow chart of a method for denoising and deconvolution of super-resolution imaging based on photoswitch molecules according to an embodiment of the present disclosure. As can be seen from FIG1 , the embodiment of the present disclosure includes at least steps S101 to S103:

[0032] Step S101: Acquire multiple fluorescence image sequences, wherein the structure to be imaged in each sequence is labeled with a photoswitchable fluorescent molecule; each sequence is generated using the photoswitchable properties of the photoswitchable fluorescent molecule, and includes at least one bright state image and at least one dark state image.

[0033] Photoswitchable fluorescent molecules are molecules that undergo reversible changes in structure and properties under light stimulation. The photoswitchable property of a photoswitchable fluorescent molecule refers to its ability to switch between bright and dark states under different lighting conditions. Specifically, under one specific wavelength, the photoswitchable fluorescent molecule is in the "on" state, meaning it is bright or luminescent. Under another specific wavelength, the photoswitchable fluorescent molecule is in the "off" state, meaning it is dark or non-luminescent.

[0034] In a specific implementation, after labeling the structure to be imaged with photoswitchable fluorescent molecules, modulated light is used to illuminate the structure to be imaged. When the photoswitchable fluorescent molecules are in the luminescent state, fluorescence images are captured using a widefield microscope, resulting in one or more bright-state images. Since the structure to be imaged is labeled with the photoswitchable fluorescent molecules, each bright-state image contains signal data and noise data generated by the structure to be imaged, including Gaussian noise, Poisson noise, and sample autofluorescence. When the majority of the photoswitchable fluorescent molecules are in the non-luminescent state, fluorescence images are captured, resulting in one or more dark-state images, each containing weak signal data and noise data. Ultimately, multiple fluorescence image sequences can be obtained, such as fluorescence image sequence a: bright-state image a1, dark-state image a2; fluorescence image sequence b: bright-state image b1, bright-state image b2, dark-state image b3; and fluorescence image sequence c: bright-state image c1, bright-state image c2, dark-state image c3, and dark-state image c4. The number of fluorescence image sequences is not limited in the present embodiment and can be set according to actual conditions. For example, the number can be set to 500. Here, the structures to be imaged in the fluorescence image sequence are cell structures observed by fluorescence microscopy, including but not limited to: endoplasmic reticulum, mitochondria, chloroplasts, centrosomes, Golgi bodies, morphology of cell nucleoli, ribosomes, lysosomes, etc.

[0035] Step S102: performing denoising processing on each bright-state image and each dark-state image in the plurality of fluorescence image sequences to obtain a denoised image.

[0036] Research has found that for completely noise-free blurred images, deconvolution can be used to fully restore the original image. However, in the actual imaging process, due to uncertainties such as optical system noise and point spread function, deconvolution algorithms often result in a large number of artifacts in the image. Moreover, as the number of deconvolution iterations increases, the noise signal is amplified, and deconvolution further introduces more artifacts into the image.

[0037] Based on this, in this step, the bright-state images and dark-state images in multiple fluorescence image sequences can be processed first to remove the noise data in the image and retain the signal data generated by the structure to be imaged in the image, thereby obtaining a denoised image with a higher signal-to-noise ratio.

[0038] Step S103: performing deconvolution processing on the denoised image to obtain a corresponding super-resolution imaging image.

[0039] After obtaining the denoised image, deconvolution is performed on the denoised image to obtain a super-resolution image corresponding to the denoised image. In implementation, for example, the denoised image can be deconvolved using direct deconvolution to obtain the corresponding super-resolution image, or it can be deconvolved using iterative deconvolution to obtain the denoised image. The disclosed embodiments do not limit the deconvolution method.

[0040] In this step, since the denoised image has a high signal-to-noise ratio, performing a deconvolution operation on the denoised image can improve the resolution while suppressing reconstruction artifacts, thereby obtaining a high-quality super-resolution imaging image.

[0041] As can be seen from the method shown in FIG1 , the disclosed embodiment, based on the characteristics of common equipment in fields such as biology and medicine, proposes a highly versatile and robust super-resolution imaging technology solution. Using only an ordinary wide-field microscope, multiple fluorescence image sequences can be acquired by utilizing the principle that light-modulated fluorescent molecules are modulated by light while noise is not. Denoising each bright-state image and each dark-state image in these multiple fluorescence image sequences effectively removes the noise data, thereby obtaining a denoised image with an extremely high signal-to-noise ratio. Deconvolution of the denoised image improves the resolution of the denoised image, resulting in a corresponding super-resolution image. As can be seen, compared to existing technologies, this solution places lower demands on imaging equipment, significantly reducing costs, and without limiting the size of the imaging area. It can meet the needs of large-field-of-view imaging of fixed biological samples in application scenarios, enabling large-field super-resolution imaging of fixed cells and tissues.

[0042] Furthermore, in order to better illustrate the process of the above-mentioned denoising and deconvolution super-resolution imaging method based on photoswitch molecules, as a refinement and extension of the above-mentioned embodiment, the present application provides several optional embodiments, but is not limited thereto, as shown below:

[0043] For step S101:

[0044] Multiple fluorescence image sequences may be acquired first. Specifically, in one possible embodiment, the fluorescence image sequences are acquired using optical imaging techniques, including wide-field fluorescence microscopy, confocal fluorescence microscopy, spinning disk confocal microscopy, structured light illumination, optical wave super-resolution imaging, or photoacoustic imaging.

[0045] In a possible embodiment, in the above step S101, the photoswitchable fluorescent molecules include: fluorescent proteins with photoswitchable properties, fluorescent dyes with photoswitchable properties, or quantum dots.

[0046] In this embodiment, for fluorescent proteins with photoswitch properties, specifically, in one possible implementation, the fluorescent protein with photoswitch properties responds to light modulation and has two states, bright and dark, including: photoswitch fluorescent protein, reversible photoswitch fluorescent protein or light-blinking fluorescent protein, etc. In one possible implementation, the photoswitch fluorescent protein includes: Skylan-S, rsEGFP2 or Dronpa, etc. Here, Skylan-S (a new photoswitch protein) is a monomer that can be used for living cell imaging and has extremely high optical stability and switching properties. rsEGFP2 (reversible conversion enhanced green fluorescent protein 2) has the characteristics of easy crystallization and good optical properties. Dronpa (erasable optical protein) is a monomer fluorescent protein extracted from Pectiniidae (a petrified coral polyp) with unique photochromic properties, and can control the on and off of fluorescence with two different excitation lights.

[0047] Regarding fluorescent dyes with photoswitchable properties, specifically, in one possible embodiment, the fluorescent dyes with photoswitchable properties include photoswitchable dyes or photo-scintillation fluorescent dyes. In practice, fluorescent dyes with photoswitchable properties may include, for example, fluorescein, cyanine dyes, or rhodamine fluorescent dyes. It should be noted that the disclosed embodiments do not limit the types of fluorescent proteins and fluorescent dyes with photoswitchable properties.

[0048] Quantum dots, also known as semiconductor nanocrystals, can lock electrons in a very small three-dimensional space, restricting the movement of electrons. When a certain electric field or light is applied to them, they will emit light of a specific frequency. The frequency of the light emission changes with the change of particle size. By adjusting the size of the quantum dots, the color of their light emission can be controlled.

[0049] In one possible embodiment, in the above method, the light emitted by the photoswitchable fluorescent molecules includes blue light, green light, yellow light, red light, or far-infrared wavelength light. The disclosed embodiments do not limit the light emitted by the photoswitchable fluorescent molecules.

[0050] In one possible embodiment, in the above method, the photoswitchable fluorescent molecule is used to label the structure to be imaged by the following method A1 / method A2:

[0051] Mode A1: The photoswitchable fluorescent molecule is used to label the structure to be imaged by molecular cloning.

[0052] Mode A2: The photoswitchable fluorescent molecule labels the structure to be imaged by binding to a target antibody, wherein the target antibody specifically binds to the structure to be imaged.

[0053] In this embodiment, the type of target antibody can be a common antibody or a nano antibody. For example, a light-switch fluorescent molecule A can be used to express in fusion with the structure to be imaged (referring to constructing a fusion gene with the 3' end of an exogenous protein gene and another gene for expression) through a molecular cloning method (providing a method for purifying and amplifying specific DNA fragments at the molecular level) to mark the structure to be imaged. The structure to be imaged can also be marked by combining a light-switch fluorescent molecule B with a nano antibody C, and the target antibody C specifically binds to the structure to be imaged. It should be noted that the examples given here are only illustrative and do not limit the embodiments of the present disclosure.

[0054] Regarding step S102:

[0055] After obtaining multiple fluorescence image sequences, denoising can be performed on the multiple fluorescence image sequences to obtain denoised images. Specifically, in one possible implementation, denoising is performed on each bright-state image and each dark-state image in the multiple fluorescence image sequences to obtain denoised images using the following method B1 / method B2:

[0056] Mode B1: for any multiple groups of fluorescence image pairs in the multiple fluorescence image sequences, subtract the bright state image from the dark state image in each of the fluorescence image pairs to obtain each subtracted image; and generate the denoised image based on each of the subtracted images.

[0057] Mode B2: Using an optical lock-in amplification algorithm, the plurality of fluorescence image sequences are processed to obtain the denoised image.

[0058] In this embodiment, for method B1, for any number of fluorescence image pairs in multiple fluorescence image sequences, the pixel values ​​of corresponding pixels in the bright-state image and the dark-state image in each fluorescence image pair can be subtracted to obtain a subtracted image. Then, a denoised image is generated based on each subtracted image. For example, the multiple fluorescence image sequences include fluorescence image sequence d and fluorescence image sequence e. For fluorescence image sequence d, the pixel values ​​of corresponding pixels in the bright-state image d1 and the dark-state image d2 in fluorescence image sequence d can be subtracted to obtain subtracted image 1. For fluorescence image sequence e, the pixel values ​​of corresponding pixels in the bright-state image e1 and the dark-state image e3 in fluorescence image sequence e can be subtracted to obtain subtracted image 2. Then, a weighted average process can be performed on each of the second subtracted images to obtain a denoised image. For example, the weights of subtracted images 1 and 2 can be set to 1 / 2. In this case, the pixel value of each pixel in the denoised image = 1 / 2 of the pixel value of each pixel in subtracted image 1 + 1 / 2 of the pixel value of each pixel in subtracted image 2. The weights of the subtracted images are not limited in the present embodiment and can be set according to actual needs. It should be noted that the examples given here are only illustrative and do not limit the present embodiment.

[0059] For method B2, for example, an optical lock-in amplification algorithm (OLID algorithm) can be used to process the fluorescence image sequence f, the fluorescence image sequence j, the fluorescence image sequence h, and the fluorescence image sequence i to obtain a denoised image. It should be noted that the examples given here are merely illustrative and do not limit the embodiments of the present disclosure.

[0060] Regarding step S103:

[0061] After obtaining the denoised image, the denoised image may be subjected to deconvolution processing to obtain a corresponding super-resolution imaging image. Specifically, in one possible implementation, the deconvolution processing of the denoised image to obtain a corresponding super-resolution imaging image includes:

[0062] Based on the Richardson-Lucy algorithm, deconvolution processing is performed on the denoised image to obtain a super-resolution imaging image corresponding to the denoised image.

[0063] In specific implementation, the system point spread function can be determined according to the following formula (1):

[0064] Where I0 is the normalization coefficient, J1 is the Bessel function of the first kind, λ is the wavelength, NA is the numerical aperture, and ρ is the polar diameter in polar coordinates. When fluorescence emitted by a sample passes through an optical system, due to diffraction, the image formed on the camera is the convolution of the sample fluorescence and the system's point spread function. Therefore, to better reflect the diffraction process of an optical system, the system's point spread function can be represented using the Bessel function of the first kind.

[0065] Taking into account the influence of noise in imaging, too many deconvolution iterations will produce artifacts, while insufficient iterations cannot improve the image resolution well. Therefore, the number of deconvolution iterations can be determined according to the quality of the denoised image. Specifically, the number of iterations can be determined by calculating the Fourier ring correlation resolution (FRC resolution) of two consecutive frames of images in the actual sample to ensure that the image resolution reaches the highest and there are no obvious reconstruction artifacts.

[0066] In a possible implementation, in the above method, the imaging objects of the method include: fixed cells, fixed tissues, etc.

[0067] FIG2 shows a flow chart of a method for denoising and deconvolution of super-resolution imaging based on photoswitch molecules according to another embodiment of the present disclosure. As shown in FIG2 , this embodiment includes the following steps S201 to S204:

[0068] Step S201: Using photoswitchable fluorescent molecules to label the structure to be imaged, multiple fluorescence image sequences are acquired. Each fluorescence image sequence includes at least one bright-state image and at least one dark-state image. During implementation, the fluorescence image sequences are acquired using optical imaging techniques, including widefield fluorescence microscopy, confocal fluorescence microscopy, spinning disk confocal microscopy, structured light illumination imaging, optical wave super-resolution imaging, or photoacoustic imaging. Photoswitchable fluorescent molecules include fluorescent proteins with photoswitchable properties, fluorescent dyes with photoswitchable properties, or quantum dots. Fluorescent proteins with photoswitchable properties respond to light modulation and have two states: a bright state and a dark state. They include photoswitchable fluorescent proteins, reversible photoswitchable fluorescent proteins, or photoblinking fluorescent proteins. Examples of photoswitchable fluorescent proteins include Skylan-S, rsEGFP2, or Dronpa. Fluorescent dyes with photoswitchable properties include photoswitchable dyes or photoblinking fluorescent dyes. Light emitted by photoswitchable fluorescent molecules includes blue, green, yellow, red, or far-infrared wavelengths. The structure to be imaged is labeled with a photoswitchable fluorescent molecule. For example, the photoswitchable fluorescent molecule can be used to label the structure to be imaged through molecular cloning. Another example is that the photoswitchable fluorescent molecule can be combined with a target antibody to label the structure to be imaged, where the target antibody specifically binds to the structure to be imaged. In practice, the imaging target, i.e., the structure to be imaged, includes fixed cells, fixed tissues, etc.

[0069] The following uses photoswitchable fluorescent protein as an example to specifically describe how to obtain multiple fluorescence image sequences of the target molecule. First, through cell transfection technology (a technology that introduces exogenous molecules such as DNA, RNA, etc. into eukaryotic cells), F-actin is fused with photoswitchable fluorescent protein (RSFP) and expressed in eukaryotic cells: In human osteosarcoma cells U-2OS, after 36 hours, the cytoskeleton is fixed with a fixative, and fluorescence images are collected using a homemade or commercial fluorescence microscopy imaging platform. For example, the main model of the microscope is model A, using 405nm laser and 488nm laser, 150×, 1.45NA objective lens, the camera model is model B, and the pixel size is 6.5μm. Figure 3 shows the laser modulation timing diagram provided in an embodiment of the present disclosure. Referring to the laser modulation mode shown in Figure 3, the 405nm and 488nm lasers can be turned on at the same time, and the 405nm laser can be turned off after starting to take pictures and collect images to obtain an image sequence; the 488nm laser can be turned on, and the 405nm laser can be turned on after starting to take pictures and collect images to obtain an image sequence. Ultimately, a series of fluorescence images with repeated switching states, i.e., multiple fluorescence image sequences, can be obtained.

[0070] Step S202 : for any plurality of fluorescence image pairs in the plurality of fluorescence image sequences, subtract the bright state image from the dark state image in each fluorescence image pair to obtain subtracted images.

[0071] Step S203: Generate a denoised image based on each subtracted image. During implementation, an optical lock-in amplification algorithm can also be used to process multiple fluorescence image sequences to obtain a denoised image.

[0072] Step S204: performing deconvolution processing on the denoised image based on the Richardson-Lucy algorithm to obtain a super-resolution imaging image corresponding to the denoised image.

[0073] FIG4 shows a comparison of the image processing results of direct deconvolution without denoising and deconvolution after denoising provided by the embodiment of the present disclosure and the original image. Among them, a corresponds to the wide-field image, b corresponds to the image obtained by direct deconvolution of the wide-field image, c corresponds to the image obtained by deconvolution after denoising, a1 is the corresponding magnified image of a, b1 is the corresponding magnified image of b, and c1 is the corresponding magnified image of c (scale: 2μm (a); 1μm (b)), and e is a comparison of the signal-to-noise ratio of the image without direct deconvolution and after deconvolution after denoising. Referring to FIG4, according to a and c, it can be seen that the image processed by deconvolution after denoising is clearer, indicating higher image resolution and fewer artifacts. According to e, it can be seen that the signal-to-noise ratio of the image processed by deconvolution after denoising is higher, indicating higher resolution. It can be seen that the denoised image obtained based on the photoswitching properties of the photoswitching molecule has extremely low noise. Then, the denoised image is deconvolved, thereby greatly improving the spatiotemporal resolution of the denoised image and obtaining a super-resolution imaging image.

[0074] Those skilled in the art will understand that in the above method of the specific embodiment, the writing order of each step does not mean a strict execution order, but does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0075] It should be noted that, in practical applications, all possible implementation methods described above can be arbitrarily combined to form possible embodiments of the present application, which will not be described one by one here.

[0076] Based on the same concept, the present disclosure also provides a denoising and deconvolution super-resolution imaging device based on photoswitch molecules. FIG5 shows a schematic structural diagram of the denoising and deconvolution super-resolution imaging device based on photoswitch molecules provided by the present disclosure. Referring to FIG5 , the denoising and deconvolution super-resolution imaging device 500 based on photoswitch molecules provided by the present disclosure includes:

[0077] An acquisition module 501 is configured to acquire a plurality of fluorescence image sequences, wherein the structures to be imaged in the fluorescence image sequences are labeled with photoswitchable fluorescent molecules; each sequence is generated using the photoswitchable properties of the photoswitchable fluorescent molecules, and includes at least one bright-state image and at least one dark-state image;

[0078] a denoising module 502 for performing denoising on each bright-state image and each dark-state image in the plurality of fluorescence image sequences to obtain a denoised image;

[0079] The deconvolution module 503 is configured to perform deconvolution processing on the denoised image to obtain a corresponding super-resolution imaging image.

[0080] In a possible embodiment, in the above device, the photoswitchable fluorescent molecules include: fluorescent proteins with photoswitchable properties, fluorescent dyes with photoswitchable properties, or quantum dots.

[0081] In one possible embodiment, in the above device, the fluorescent protein with photoswitch properties responds to light modulation and has two states: bright state and dark state, including: photoswitchable fluorescent protein, reversible photoswitchable fluorescent protein or light-blinking fluorescent protein, etc.

[0082] In a possible embodiment, in the above device, the photoswitchable fluorescent protein includes: Skylan-S, rsEGFP2 or Dronpa, etc.

[0083] In a possible implementation, in the above device, the fluorescent dye with photoswitch properties includes: photoswitch dye or light-scintillation fluorescent dye, etc.

[0084] In a possible implementation, in the above device, the types of light emitted by the optical switch fluorescent molecules include: blue light, green light, yellow light, red light, or light with far-infrared wavelengths.

[0085] In one possible embodiment, in the above device, the photoswitchable fluorescent molecule is used to label the structure to be imaged by the following method:

[0086] The light-switch fluorescent molecule is labeled with the structure to be imaged by molecular cloning; or

[0087] The light-switch fluorescent molecule labels the structure to be imaged by binding to a target antibody, wherein the target antibody specifically binds to the structure to be imaged.

[0088] In one possible embodiment, in the above-mentioned device, the fluorescence image sequence is obtained by optical imaging technology, and the optical imaging technology includes: wide-field fluorescence microscopy imaging technology, confocal fluorescence microscopy imaging technology, spinning disk confocal microscopy imaging technology, structured light illumination imaging technology, optical wave super-resolution imaging technology or photoacoustic imaging technology, etc.

[0089] In a possible implementation, in the above device, the denoising module 502 is configured to:

[0090] For any plurality of fluorescence image pairs in the plurality of fluorescence image sequences, subtract the bright-state image from the dark-state image in each of the fluorescence image pairs to obtain subtracted images; and generate the denoised image based on each of the subtracted images; or

[0091] The plurality of fluorescence image sequences are processed using an optical lock-in amplification algorithm to obtain the denoised image.

[0092] In a possible implementation, in the above device, the deconvolution module 503 is configured to:

[0093] Based on the Richardson-Lucy algorithm, deconvolution processing is performed on the denoised image to obtain a super-resolution imaging image corresponding to the denoised image.

[0094] In a possible implementation, in the above-mentioned device, the imaging objects of the method include: fixed cells, fixed tissues, etc.

[0095] It should be noted that any of the above-mentioned denoising and deconvolution super-resolution imaging devices based on photoswitch molecules can implement the above-mentioned denoising and deconvolution super-resolution imaging method based on photoswitch molecules in a one-to-one correspondence, which will not be repeated here.

[0096] FIG6 shows a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. As shown in FIG6 , at the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0097] The processor, network interface, and memory can be interconnected via an internal bus, such as an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, FIG6 shows only one bidirectional arrow, but this does not imply that there is only one bus or only one type of bus.

[0098] The memory is used to store programs. Specifically, the program may include program code, which includes computer operating instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0099] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs it, forming a denoising and deconvolution super-resolution imaging device based on photoswitchable molecules at the logical level. The processor executes the program stored in the memory and is specifically used to perform the aforementioned method.

[0100] The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method may be completed by hardware integrated logic circuits in the processor or by software instructions. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure may be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of this disclosure may be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0101] The electronic device can execute the denoising and deconvolution super-resolution imaging method based on photoswitch molecules provided by multiple embodiments of the present disclosure, and realize the functions of the denoising and deconvolution super-resolution imaging device based on photoswitch molecules in the embodiment shown in Figure 5. The embodiments of the present disclosure will not be repeated here.

[0102] An embodiment of the present disclosure also proposes a computer-readable storage medium, which stores one or more programs, and the one or more programs include instructions. When the instructions are executed by an electronic device including multiple application programs, the electronic device can perform the denoising and deconvolution super-resolution imaging method based on photoswitch molecules provided by multiple embodiments of the present disclosure.

[0103] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0105] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0107] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0108] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0109] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0110] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0111] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0112] The above are merely examples of the present disclosure and are not intended to limit the present disclosure. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure are intended to be included within the scope of the claims of the present disclosure.

Claims

1. A denoising and deconvolution super-resolution imaging method based on light-switching molecules, wherein, The method includes: Obtaining a plurality of fluorescence image sequences, where the structure to be imaged in the images of the fluorescence image sequences is labeled with a photoswitchable fluorescent molecule; each of the sequences is generated using the photoswitching property of the photoswitchable fluorescent molecule, and the sequence includes at least one bright-state image and at least one dark-state image; Performing denoising processing on each bright-state image and each dark-state image in the plurality of fluorescence image sequences to obtain denoised images; Performing deconvolution processing on the denoised images to obtain corresponding super-resolution imaging images.

2. The de-noising and de-convolution super-resolution imaging method based on optical switch molecules according to claim 1, wherein The photoswitchable fluorescent molecule includes: a fluorescent protein with photoswitching properties, a fluorescent dye with photoswitching properties, or a quantum dot.

3. The denoising and deconvolution super-resolution imaging method based on a photoswitchable molecule according to claim 2, wherein, The fluorescent protein with photoswitching properties responds to light modulation and has two states, bright state and dark state, including: a photoswitchable fluorescent protein, a reversibly photoswitchable fluorescent protein, or a photoblinking fluorescent protein, etc.

4. The method for denoising and deconvolving super-resolution imaging based on optical switch molecules according to claim 3, wherein, The photoswitchable fluorescent protein includes: Skylan-S, rsEGFP2, or Dronpa, etc.

5. The de-noising and de-convolution super-resolution imaging method based on optical switch molecules according to claim 2, wherein, The fluorescent dye with photoswitching properties includes: a photoswitchable dye or a photoblinking fluorescent dye, etc.

6. The denoising and deconvolution super-resolution imaging method based on a photoswitchable molecule according to claim 1, wherein, The types of light emitted by the photoswitchable fluorescent molecule include: blue light, green light, yellow light, red light, or light with a far-infrared wavelength, etc.

7. The denoising and deconvolution super-resolution imaging method based on optical switch molecules according to claim 1, wherein, The photoswitchable fluorescent molecule labels the structure to be imaged by the following method: The photoswitchable fluorescent molecule labels the structure to be imaged by molecular cloning; or The photoswitchable fluorescent molecule binds to a target antibody to label the structure to be imaged, where the target antibody specifically binds to the structure to be imaged.

8. The de-noising and de-convolution super-resolution imaging method based on photo-switchable molecules according to claim 1, wherein, The fluorescence image sequence is obtained by an optical imaging technique, and the optical imaging technique includes: wide-field fluorescence microscopy imaging technique, confocal fluorescence microscopy imaging technique, spinning disk confocal microscopy imaging technique, structured illumination imaging technique, optical fluctuation super-resolution imaging technique, or photoacoustic imaging technique, etc.

9. The de-noising and de-convolution super-resolution imaging method based on a photoswitchable molecule according to claim 1, wherein, The performing denoising processing on each bright-state image and each dark-state image in the plurality of fluorescence image sequences to obtain denoised images includes: For any multiple groups of fluorescence image pairs in the plurality of fluorescence image sequences, subtracting the bright-state image from the dark-state image in each fluorescence image pair to obtain each subtracted image; generating the denoised image according to each subtracted image; or Using an optical lock-in amplification algorithm to process the plurality of fluorescence image sequences to obtain the denoised images.

10. The denoising and deconvolution super-resolution imaging method based on optical switch molecules according to claim 1, wherein, The performing deconvolution processing on the denoised images to obtain corresponding super-resolution imaging images includes: Based on the Richardson-Lucy algorithm, performing deconvolution processing on the denoised images to obtain the super-resolution imaging images corresponding to the denoised images.

11. The de-noising and de-convolution super-resolution imaging method based on a photoswitchable molecule according to any one of claims 1-10, wherein, The imaging objects of the method include: fixed cells, fixed tissues, etc.

12. A denoising and deconvolution super-resolution imaging device based on a photoswitchable molecule, wherein, The device includes: An acquisition module for acquiring a plurality of fluorescence image sequences, where the structure to be imaged in the images of the fluorescence image sequences is labeled with a photoswitchable fluorescent molecule; each of the sequences is generated using the photoswitching property of the photoswitchable fluorescent molecule, and the sequence includes at least one bright-state image and at least one dark-state image; A denoising module for performing denoising processing on each bright-state image and each dark-state image in the plurality of fluorescence image sequences to obtain denoised images; A deconvolution module for performing deconvolution processing on the denoised image to obtain a corresponding super-resolution imaging image.

13. An electronic device, comprising: A processor; And A memory arranged to store computer-executable instructions, wherein the executable instructions, when executed, cause the processor to perform the steps of the method according to any one of claims 1-11.

14. A computer-readable storage medium storing one or more programs, wherein, When the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device is caused to perform the steps of the method according to any one of claims 1-11.

Images