A method, apparatus, device, system and storage medium for microscopic imaging

Abstract

This application provides a method, apparatus, device, system, and storage medium for microscopic imaging, relating to the field of microscopic imaging technology. The method includes: acquiring the current state and brightness change of a fluorescently labeled sample; determining a light intensity adjustment amount based on the current state and brightness change; acquiring a first microscopic image of the fluorescently labeled sample using a structured illumination microscope when the illumination intensity is adjusted to a target illumination intensity, wherein the target illumination intensity is the illumination intensity obtained based on the light intensity adjustment amount; and obtaining a target microscopic image of the fluorescently labeled sample based on the first microscopic image. This method can reduce the phototoxicity to the sample, thereby improving imaging quality.

Description

Technical Field

[0001] This application relates to the field of microscopic imaging technology, and more specifically, to a method, apparatus, device, system, and storage medium for microscopic imaging. Background Technology

[0002] Currently, in the field of microscopic imaging, compared with traditional optical microscopes, structured illumination microscopes are widely used in biological research, materials science, and medical diagnostics due to their advantages of high resolution and high-speed imaging.

[0003] However, while structured illumination microscopy can acquire high-resolution microscopic images of the internal structure of a sample, it requires high light intensity during the microscopic imaging process. This may cause phototoxicity to the sample, thereby affecting the sample's activity and imaging quality.

[0004] Therefore, how to reduce the phototoxicity of samples and thus improve imaging quality is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] This application provides a method, apparatus, device, system, and storage medium for microscopic imaging, which can reduce the phototoxic effects on samples and thus improve imaging quality.

[0006] In a first aspect, a microscopic imaging method is provided, applied to a control device connected to a structured illumination microscope. The method includes: acquiring the current state and brightness change of a fluorescently labeled sample; determining an illumination intensity adjustment amount based on the current state and brightness change; acquiring a first microscopic image of the fluorescently labeled sample acquired by the structured illumination microscope when the illumination intensity of the structured illumination microscope is adjusted to a target illumination intensity, wherein the target illumination intensity is the illumination intensity obtained based on the illumination intensity adjustment amount; and obtaining a target microscopic image of the fluorescently labeled sample based on the first microscopic image.

[0007] In the embodiments of this application, the current state and brightness change of the fluorescently labeled sample are obtained; based on the current state and brightness change, the light intensity adjustment amount is determined; when the light intensity of the structured illumination microscope is adjusted to the target light intensity, a first microscopic image of the fluorescently labeled sample is acquired by the structured illumination microscope. Since this scheme can determine how to adjust the light intensity of the structured illumination microscope based on the current state and brightness change, the fluorescently labeled sample can adapt to the current target light intensity, thereby reducing the phototoxic effect of the structured illumination microscope's light intensity on the sample. Since the target light intensity is the light intensity obtained based on the light intensity adjustment amount, when the light intensity of the structured illumination microscope is adjusted to the target light intensity, the first microscopic image is acquired, and based on the first microscopic image, a target microscopic image with high imaging quality is obtained. Based on this, this scheme can reduce the phototoxic effect on the sample during the microscopic imaging of the sample using a structured illumination microscope, thereby improving the imaging quality.

[0008] In conjunction with the first aspect, in some possible implementations, the light intensity adjustment amount is determined based on the current state and the amount of brightness change, including: if the current state is an active state and the amount of brightness change is greater than or equal to a first change threshold, the light intensity adjustment amount is determined to be less than zero; if the current state is a stable state and the absolute value of the amount of brightness change is less than or equal to a second change threshold, the light intensity adjustment amount is determined to be greater than zero, and the second change threshold is less than the first change threshold.

[0009] In the embodiments of this application, by combining the current state of the fluorescently labeled sample and the amount of brightness change, the amount of light intensity adjustment is comprehensively determined. This enables the fluorescently labeled sample to adapt to the currently adjusted target light intensity, thereby reducing the phototoxic effect of the light intensity of the structure illumination microscope on the sample.

[0010] In conjunction with the first aspect and the above implementation methods, in some possible implementation methods, the control device is also connected to an event camera to acquire the current state and brightness change of the fluorescently labeled sample, including: acquiring the event signal from the event camera; obtaining the current state based on the frequency of the event signal; and obtaining the brightness change based on the intensity of the event signal.

[0011] In the embodiments of this application, by acquiring the event signal from the event camera and based on the characteristics (frequency and intensity) of the event signal, the current state and brightness change of the fluorescently labeled sample can be obtained quickly and accurately, so as to adjust the light intensity of the structure illumination microscope in a timely manner, thereby reducing the phototoxic effect of the light intensity of the structure illumination microscope on the sample.

[0012] One possible way to acquire event signals from an event camera is to acquire event signals every first preset period.

[0013] As one possible implementation, the control device is also connected to an atomic force microscope to acquire the current state and brightness change of the fluorescently labeled sample, including: acquiring the transverse force signal in the atomic force microscope; obtaining the current state based on the transverse force signal; and acquiring the brightness signal in the structure illumination microscope every second preset period, the brightness signal including the brightness change.

[0014] In the embodiments of this application, the sample state is dynamically determined by the lateral force signal from an atomic force microscope and the brightness signal from a structure illumination microscope; then, based on the sample state, the illumination intensity of the structure illumination microscope is adjusted to reduce phototoxicity.

[0015] Combining the first aspect and the above implementation methods, in some possible implementation methods, the current state is obtained based on the frequency of the event signal, including: if the frequency is greater than or equal to a first frequency threshold, the current state is determined to be an active state; if the frequency is less than or equal to a second frequency threshold, the current state is determined to be a stable state, where the second frequency threshold is less than the first frequency threshold.

[0016] In the embodiments of this application, the current state of the fluorescently labeled sample can be quickly and accurately determined by the frequency of the event signal, which facilitates the subsequent determination of the light intensity adjustment amount so that the fluorescently labeled sample can adapt to the current target light intensity, thereby reducing the phototoxic effect of the light intensity of the structure illumination microscope on the sample.

[0017] In conjunction with the first aspect and the above-described implementations, in some possible implementations, the control device is also connected to an atomic force microscope. The method provided in this application embodiment further includes: acquiring a second microscopic image of a fluorescently labeled sample collected by an atomic force microscope; and obtaining a target microscopic image of the fluorescently labeled sample based on the first microscopic image, including: obtaining the target microscopic image based on the first microscopic image, the second microscopic image, and the timestamp of the event signal.

[0018] In the embodiments of this application, by acquiring a second microscopic image from an atomic force microscope, and based on the first microscopic image, the second microscopic image, and the timestamp of the event signal, a target microscopic image with high imaging quality can be obtained, thereby improving the microscopic imaging quality.

[0019] Combining the first aspect and the above implementation methods, in some possible implementation methods, obtaining a target microscopic image based on the timestamps of the first microscopic image, the second microscopic image, and the event signal includes: synchronizing the timestamps of the event signal with the acquisition times of the first and second microscopic images to obtain aligned first and second microscopic images; and fusing the aligned first and second microscopic images in a three-dimensional spatial coordinate system to obtain the target microscopic image.

[0020] In the embodiments of this application, the acquisition time of the first and second microscopic images is synchronized by the timestamp of the event signal, and the target microscopic image is constructed in a three-dimensional spatial coordinate system, which may further improve the quality of microscopic imaging.

[0021] Secondly, a microscopic imaging apparatus is provided, applied to a control device connected to a structure illumination microscope, the apparatus comprising:

[0022] The acquisition module is used to acquire the current state and brightness change of the fluorescently labeled sample;

[0023] The determination module is used to determine the amount of light intensity adjustment based on the current state and the amount of brightness change;

[0024] The acquisition module is also used to acquire the first microscopic image of the fluorescently labeled sample acquired by the structure illumination microscope when the illumination intensity of the structure illumination microscope is adjusted to the target illumination intensity, wherein the target illumination intensity is the illumination intensity obtained based on the illumination intensity adjustment amount;

[0025] An imaging module is used to obtain a target microscopic image of the fluorescently labeled sample based on a first microscopic image.

[0026] As one possible implementation, the determination module is specifically used to: if the current state is an active state and the brightness change is greater than or equal to a first change threshold, determine that the light intensity adjustment amount is less than zero; if the current state is a stable state and the absolute value of the brightness change is less than or equal to a second change threshold, determine that the light intensity adjustment amount is greater than zero, and the second change threshold is less than the first change threshold.

[0027] As one possible implementation, the acquisition module is specifically used to: acquire the event signal from the event camera; obtain the current state based on the frequency of the event signal; and obtain the brightness change based on the intensity of the event signal.

[0028] As one possible implementation, the determination module is specifically used to: determine the current state as an active state if the frequency is greater than or equal to a first frequency threshold; and determine the current state as a stable state if the frequency is less than or equal to a second frequency threshold, wherein the second frequency threshold is less than the first frequency threshold.

[0029] As one possible implementation, the determining module is also used to acquire a second microscopic image of a fluorescently labeled sample acquired by an atomic force microscope; specifically, the determining module is used to obtain a target microscopic image based on the first microscopic image, the second microscopic image, and the timestamp of the event signal.

[0030] As one possible implementation, the determination module is specifically used to: synchronize the timestamp of the event signal with the acquisition time of the first and second microscopic images to obtain aligned first and second microscopic images; and fuse the aligned first and second microscopic images in a three-dimensional spatial coordinate system to obtain the target microscopic image.

[0031] Thirdly, an electronic device is provided, including a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, causing the electronic device to perform the methods of the first aspect or any possible implementation thereof.

[0032] Fourthly, a microscopic imaging system is provided, comprising: a control device and a structured illumination microscope, wherein the control device is connected to the structured illumination microscope and performs the method described in the first aspect or any possible implementation thereof.

[0033] As one possible implementation, the control device is also connected to an event camera and an atomic force microscope.

[0034] Fifthly, a computer program product is provided, comprising: computer program code, which, when run on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0035] In a sixth aspect, a computer-readable storage medium is provided that stores computer program code, which, when executed on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof. Attached Figure Description

[0036] Figure 1(a) is a schematic diagram of the structure of a microscopic imaging system provided in an embodiment of this application;

[0037] Figure 1(b) is a schematic diagram of a microscopic imaging system provided in an embodiment of this application;

[0038] Figure 2 This is a flowchart illustrating a microscopic imaging method provided in an embodiment of this application;

[0039] Figure 3 This is a flowchart illustrating another microscopic imaging method provided in an embodiment of this application;

[0040] Figure 4 This is a schematic diagram of the structure of a microscopic imaging device provided in an embodiment of this application;

[0041] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0042] The technical solutions in this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0043] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0044] Before introducing the solutions of the embodiments of this application, the technical terms that may be involved in the embodiments of this application will be explained first.

[0045] (1) Structured Illumination Microscopy (SIM): A super-resolution optical microscopy technique that achieves super-resolution by spatially modulating the illumination field. It illuminates the sample using a specific optical structure illumination mode and reconstructs a high-resolution image using computer algorithms. SIM can break through the resolution limit of traditional optical microscopes and provide clearer imaging means for research in fields such as cell biology and molecular biology.

[0046] (2) Atomic Force Microscope (AFM): A high-resolution scanning probe microscope based on the interaction force between the probe and the sample surface, capable of imaging and analyzing the surface of various materials at the atomic level. AFM obtains information on the morphology and properties of the sample surface by detecting the interaction force between the probe and the sample surface.

[0047] (3) Event Camera: Also known as a Dynamic Vision Sensor (DVS), it is a bio-inspired vision sensor based on the principle of asynchronous pixel response. Unlike traditional cameras, event cameras do not capture images at fixed time intervals, but rather trigger event output based on pixel-level brightness changes. For example, when the brightness change of a pixel exceeds a certain threshold, the event camera will output an event signal containing information such as the pixel's location and the timestamp of the brightness change. This photosensitive imaging principle gives event cameras the characteristics of high temporal resolution and low data redundancy, enabling them to quickly capture dynamic changes in a scene.

[0048] (4) Phototoxicity: The oxidative damage effect of high-energy photons on biological samples (such as living cells).

[0049] (5) Fluorescent Labeled Sample: A biological sample in which a fluorescent probe is specifically bound to a target substance (such as a protein or organelle) through physical or chemical means.

[0050] Currently, in the field of microscopic imaging, compared with traditional optical microscopes, structured illumination microscopes are widely used in biological research, materials science, and medical diagnostics due to their advantages of high resolution and high-speed imaging.

[0051] However, while structured illumination microscopy can acquire high-resolution microscopic images of the internal structure of a sample, it requires high light intensity during the microscopic imaging process. This may cause phototoxicity to the sample, thereby affecting the sample's activity and imaging quality.

[0052] Therefore, how to reduce the phototoxicity of samples and improve the imaging quality during the process of microscopic imaging of samples using structured illumination microscopy is a technical problem that urgently needs to be solved.

[0053] In view of this, this solution provides a method, apparatus, device, system and storage medium for microscopic imaging, which can reduce the phototoxic effects on the sample during the process of microscopic imaging of the sample by structured illumination microscope.

[0054] To illustrate the technical solutions described in this application, specific embodiments are provided below. First, a microscopic imaging system according to an embodiment of this application will be introduced.

[0055] Figure 1(a) is a schematic diagram of the structure of a microscopic imaging system provided in an embodiment of this application; Figure 1(b) is a schematic diagram of a scene of a microscopic imaging system provided in an embodiment of this application.

[0056] As exemplarily shown in FIG1(a), the microscopic imaging system 100 includes a control device 110 and a structured illumination microscope 120.

[0057] Optionally, the control device 110 is also connected to the event camera 130 and the atomic force microscope 140.

[0058] For example, the control device 110 can be connected to the structure illumination microscope 120, the event camera 130, and the atomic force microscope 140 via either a wired or wireless communication connection. For instance, a wired communication connection could be an Ethernet connection or other wired communication connections, and this embodiment does not impose specific limitations on this. Similarly, a wireless communication connection could be a Wi-Fi (Wireless Fidelity) connection, a Bluetooth (BT) connection, or a Near Field Communication (NFC) connection, and this embodiment does not impose specific limitations on this.

[0059] For example, the control device 110 can be a device with data processing capabilities, such as a computer, mobile phone, server, etc., and the embodiments of this application do not impose specific limitations on it.

[0060] For example, the control device 110 can be used to integrate data from other devices (e.g., structure illumination microscope 120, event camera 130, and atomic force microscope 140), generate illumination adjustment for the structure illumination microscope 120, and combine microscopic images from each device to obtain a target microscopic image. As shown in FIG1(b), the control device 110 may include a control unit and a data processing unit.

[0061] For example, the structured illumination microscope 120 can be an optical imaging module. As shown in Figure 1(b), the structured illumination microscope 120 may include a light source, a spatial light modulator, a total internal reflection (TIR) ​​prism, a zoom lens, and a mirror. The light source can be a high-power laser (such as a 488 nm blue laser) to provide coherent illumination for the structured illumination microscope and excite fluorescent labels on the sample. The illumination intensity of the structured illumination microscope 120 can be adjusted by the light source. The spatial light modulator modulates the laser wavefront by loading a preset grating pattern (such as sinusoidal fringes) to achieve the structured illumination required for super-resolution imaging. The TIR prism can excite fluorescent labels near a 200 nm thin layer on the sample surface, reducing interference from deep background. The zoom lens dynamically adjusts the size of the imaging area to adapt to different resolution requirements.

[0062] For example, the event camera 130 can be used to capture the state of a sample at a microsecond time resolution and output a pixel-level event stream (i.e., an event signal hereinafter).

[0063] For example, the atomic force microscope 140 can be a detection module, as shown in Figure 1(b). The atomic force microscope 140 may include an AFM probe, a piezoelectric scanning stage, a laser, and a photodetector. The probe is used to carry a nanoscale sharp probe (radius of curvature <10 nm) to scan the sample surface morphology through contact or tapping modes. The piezoelectric scanning stage controls the three-dimensional movement of the sample stage with sub-nanometer precision. The displacement of the laser reflected by the cantilever beam of the laser and photodetector converts the probe deformation into an electrical signal, generating a surface height map and mechanical parameters (such as elastic modulus) in real time.

[0064] Optionally, Figure 1(b) also includes a reflector and a beam splitter. The reflector, located to the right of the beam splitter, connects the beam splitter to the spatial light modulator, redirecting the working optical path transmitted by the beam splitter to the spatial light modulator or subsequent optical elements (such as a TIR prism). The beam splitter, located below the event camera, connects the laser, photodetector, and spatial light modulator. The beam splitter splits the main beam output from the laser into two paths: one reflected to the photodetector, and the other transmitted to the spatial light modulator.

[0065] Referring to the architecture shown in Figure 1, the microscopic imaging method provided in the embodiments of this application will be described in detail below.

[0066] Figure 2 This is a schematic flowchart of a microscopic imaging method provided in an embodiment of this application.

[0067] It should be understood that the embodiments of this application do not specifically limit the executing entity of the microscopic imaging method, as long as communication can be performed according to the microscopic imaging method of the embodiments of this application by running a program that records the code of the microscopic imaging method of the embodiments of this application. For example, the executing entity of the microscopic imaging method provided in the embodiments of this application can be the electronic device described in the above embodiments, or it can be a functional module in the electronic device that can call and execute the program, such as a chip. This application does not limit this.

[0068] For example, the electronic device can be a control device connected to the structure illumination microscope. The control device can be a device with data processing capabilities, such as a computer, mobile phone, server, etc., and this application embodiment does not impose specific limitations on it.

[0069] like Figure 2 As shown, the following describes a microscopic imaging method provided by the embodiments of this application using S201 to S204 as examples.

[0070] S201. Obtain the current state and brightness change of the fluorescently labeled sample.

[0071] For example, the fluorescently labeled sample can be a living cell (e.g., stem cell, primary cell, etc.) or a subcellular structure (e.g., mitochondria, endoplasmic reticulum, cytoskeleton, etc.), and the embodiments of this application do not impose specific limitations on this.

[0072] For example, a fluorescently labeled sample can refer to a biological sample in which a target substance (e.g., protein, organelle, etc.) specifically binds to a fluorescent probe (e.g., fluorescent dye or fluorescent protein); the fluorescence distribution in the fluorescently labeled sample can be obtained using a fluorescence microscope (e.g., structured illumination microscope), thereby obtaining a microscopic image of the fluorescently labeled sample. The fluorescently labeled sample can emit light when excited at a specific wavelength.

[0073] For example, the current state can be used to represent the dynamic characteristics of a fluorescently labeled sample, such as the behavioral characteristics of cells under different physiological conditions.

[0074] For example, the current state can be a steady state. Fluorescently labeled samples in a steady state have stable basal metabolic levels, stable energy consumption, and no significant cell division, thus maintaining cell morphology, such as the resting phase of adherent cells.

[0075] For example, the current state can also be an active state; in an active state, the metabolic activity of fluorescently labeled samples is significantly enhanced, and the cells are undergoing dynamic processes such as division, migration, phagocytosis, and signal transduction.

[0076] For example, the change in brightness can refer to the magnitude of the change in brightness of the fluorescence signal in the fluorescently labeled sample.

[0077] For a detailed description of S201, please refer to the following embodiments, which will not be repeated here.

[0078] S202. Determine the light intensity adjustment amount based on the current state and the amount of brightness change.

[0079] For example, the light intensity adjustment amount can refer to the amount of adjustment to the light intensity of the structure illumination microscope. For instance, if the light intensity adjustment amount is greater than zero, it means increasing the light intensity of the structure illumination microscope; as another example, if the light intensity adjustment amount is less than zero, it means decreasing the light intensity of the structure illumination microscope.

[0080] Understandably, the amount of light intensity adjustment is determined by comprehensively considering the dynamic characteristics and brightness variation range of the fluorescently labeled sample.

[0081] For example, when the fluorescently labeled sample is in an active state and the brightness of the fluorescence signal varies greatly, it is necessary to reduce the light intensity of the structure illumination microscope to avoid phototoxicity to the fluorescently labeled sample caused by excessive light intensity.

[0082] For example, when the fluorescently labeled sample is in a stable state and the brightness variation of the fluorescence signal is small, it is necessary to increase the light intensity of the structure illumination microscope to avoid the microscopic images acquired by the structure illumination microscope being of low quality due to insufficient light.

[0083] For a detailed description of S202, please refer to the following embodiments, which will not be repeated here.

[0084] S203. When the illumination intensity of the structured illumination microscope is adjusted to the target illumination intensity, acquire the first microscopic image of the fluorescently labeled sample acquired by the structured illumination microscope.

[0085] The target light intensity is the light intensity obtained based on the light intensity adjustment amount.

[0086] As an example, given a determined amount of light intensity adjustment, the light intensity adjustment can be sent to a structured illumination microscope, which can then adjust its light intensity to the target light intensity based on the adjustment amount.

[0087] As another example, the current illumination intensity of the structure illumination microscope can be obtained after determining the illumination intensity adjustment amount; the target illumination intensity can be determined based on the illumination intensity adjustment amount and the current illumination intensity; and then the target illumination intensity can be sent to the structure illumination microscope.

[0088] For example, by adjusting the illumination intensity of the structured illumination microscope, the illumination intensity of the structured illumination microscope can be matched with the current condition of the fluorescently labeled sample in real time, avoiding the influence of a fixed illumination intensity on the fluorescently labeled sample or on the quality of the microscopic imaging.

[0089] For information on how to acquire fluorescently labeled samples using structured illumination microscopy, please refer to existing techniques; details will not be elaborated here.

[0090] For example, the control device can periodically acquire a first microscopic image from a structured illumination microscope. The first microscopic image can be a set of multiple frames of raw images with structured light fringes, typically a set of two-dimensional images in multiple directions (e.g., 0°, 120°, 240°) and multiple phases (e.g., 0, 1 / 3 period, 2 / 3 period).

[0091] For example, a first microscopic image of a fluorescently labeled sample acquired by a structured illumination microscope can be used to reflect the internal structure of the fluorescently labeled sample.

[0092] S204. Based on the first microscopic image, obtain the target microscopic image of the fluorescently labeled sample.

[0093] For example, the target microscopic image can be a super-resolution image obtained by processing a first microscopic image (e.g., a multi-frame fringe image) acquired by a structured illumination microscope using a specific algorithm. This super-resolution image breaks through the optical diffraction limit and can clearly display subcellular structures (such as microtubules, mitochondrial cristae, etc.). For example, the specific algorithm can be a multi-directional interpolation algorithm, etc., and this application embodiment does not impose specific limitations on it.

[0094] For example, nine frames of the first microscopic image (three-direction, three-phase first microscopic image) are processed to obtain a single frame of the target microscopic image.

[0095] In the embodiments of this application, the current state and brightness change of the fluorescently labeled sample are obtained; based on the current state and brightness change, the light intensity adjustment amount is determined; when the light intensity of the structured illumination microscope is adjusted to the target light intensity, a first microscopic image of the fluorescently labeled sample is acquired by the structured illumination microscope. Since this scheme can determine how to adjust the light intensity of the structured illumination microscope based on the current state and brightness change, the fluorescently labeled sample can adapt to the current target light intensity, thereby reducing the phototoxic effect of the structured illumination microscope's light intensity on the sample. Since the target light intensity is the light intensity obtained based on the light intensity adjustment amount, when the light intensity of the structured illumination microscope is adjusted to the target light intensity, the first microscopic image is acquired, and based on the first microscopic image, a target microscopic image with high imaging quality is obtained. Based on this, this scheme can reduce the phototoxic effect on the sample during the microscopic imaging of the sample using a structured illumination microscope, thereby improving the imaging quality.

[0096] In one possible embodiment of this application, the control device is also connected to an event camera, and the above-mentioned S201 includes: acquiring an event signal from the event camera; obtaining the current state based on the frequency of the event signal; and obtaining the brightness change based on the intensity of the event signal.

[0097] For example, an event camera can asynchronously trigger event signals based on changes in pixel brightness in a fluorescently labeled sample; for instance, the event camera can output an event signal when the pixel brightness change exceeds a certain threshold. For example, at the instant when a fluorescently labeled sample is undergoing cell division, local fluorescence changes can trigger multiple event signals, while no event signal is output in stable regions.

[0098] For example, the event signal may include location information, intensity, and a timestamp. The location of the event signal can be used to reflect the coordinates of a pixel in the fluorescently labeled sample, for example, (x, y) = (320, 240). The intensity of the event signal can represent the absolute magnitude of the pixel brightness change detected by the event camera, i.e., the magnitude of the brightness change of the pixel over a specific time period (e.g., ΔL = 30 grayscale value or photon count). The timestamp of the event signal can be used to reflect the time when the event occurred. Optionally, the event signal may also include polarity; polarity can be used to reflect the direction of the brightness change. For example, polarity may include +1 and -1; +1 may indicate an increase in brightness (e.g., excitation of fluorescent molecules), and -1 may indicate a decrease in brightness (e.g., photobleaching).

[0099] As an example, the current state is determined based on the frequency of the event signal, including: if the frequency is greater than or equal to a first frequency threshold, the current state is determined to be an active state; if the frequency is less than or equal to a second frequency threshold, the current state is determined to be a stable state.

[0100] The second frequency threshold is less than the first frequency threshold.

[0101] For example, the first frequency threshold and the second frequency threshold can be values ​​pre-configured by the event camera and control device, or they can be manually set values. This application embodiment does not impose specific limitations on this. For example, the first frequency threshold can be 100 Hz, and the second frequency threshold can be 20 Hz.

[0102] For example, the frequency of the event signal can represent the number of event signals output by the event camera within a preset time period. The frequency of the event signal can indicate the activity level during the dynamic change process of the fluorescently labeled sample. It is understood that since the event signal can be triggered when the pixel brightness change of a pixel in the fluorescently labeled sample exceeds a certain threshold, the number of event signals within the preset time period can be used to reflect the current state of the fluorescently labeled sample.

[0103] For example, the frequency of event signals can be determined using a fixed time window method. For instance, by setting a fixed period (e.g., 10 ms), the number of event signals within each fixed period is counted. For example, if 50 event signals are detected within 10 ms, the frequency of the event signals can be 5000 Hz.

[0104] As an example, if the frequency of the event signal is greater than or equal to the first frequency threshold, it can indicate that the number of event signals output by the event camera within a preset time period is large. At this time, the brightness change of a large number of pixels in the fluorescently labeled sample exceeds the set threshold, and the fluorescently labeled sample is in an active state.

[0105] For example, if the frequency of the event signal is 120Hz (greater than 100Hz (the first frequency threshold is 100Hz)), that is, 12,000 event signals are detected within 1 second, it can be said that the fluorescently labeled sample is in an active state such as cell division and rapid movement, resulting in violent fluctuations in local fluorescence. When acquiring microscopic images of fluorescently labeled samples, if the brightness change is large, it is necessary to reduce the light intensity of the structure illumination microscope to reduce phototoxicity.

[0106] As another example, if the frequency of the event signal is less than or equal to the second frequency threshold, it can indicate that the number of event signals output by the event camera within a preset time is small. At this time, the brightness change value of a small number of pixels in the fluorescently labeled sample exceeds the set threshold, and the fluorescently labeled sample is in a stable state.

[0107] For example, if the frequency of the event signal is 10Hz (less than 20Hz (the second frequency threshold is 20Hz)), and 2000 event signals are detected within 1 second, it can be said that the fluorescently labeled sample is in a stable state and the fluorescence signal tends to be stable. When acquiring microscopic images of fluorescently labeled samples, if the brightness change is small, it is necessary to increase the illumination intensity of the structured illumination microscope to improve the quality of microscopic imaging.

[0108] As another example, if the frequency of the event signal is less than the first frequency threshold and greater than the second frequency threshold, it can indicate that the number of event signals output by the event camera within a preset time is moderate. In this case, the illumination intensity of the structure illumination microscope can be left unadjusted to avoid frequent switching.

[0109] Understandably, by using the frequency of the event signal, the current state of the fluorescently labeled sample can be quickly and accurately determined, which facilitates the subsequent determination of the light intensity adjustment amount so that the fluorescently labeled sample can adapt to the current target light intensity, thereby reducing the phototoxic effect of the light intensity of the structure illumination microscope on the sample.

[0110] In the embodiments of this application, by acquiring the event signal from the event camera and based on the characteristics (frequency and intensity) of the event signal, the current state and brightness change of the fluorescently labeled sample can be obtained quickly and accurately, so as to adjust the light intensity of the structure illumination microscope in a timely manner, thereby reducing the phototoxic effect of the light intensity of the structure illumination microscope on the sample.

[0111] One possible way to acquire event signals from an event camera is to acquire event signals every first preset period.

[0112] As an example, the first preset period can be a value pre-configured by the event camera and control device, or it can be a manually set value. This application embodiment does not impose specific limitations on this. For example, the first preset period can be 100 microseconds (μs).

[0113] As another example, the first preset period can be determined based on the content of the fluorescently labeled sample being detected.

[0114] For example, when detecting rapid dynamic processes such as calcium ion transients in fluorescently labeled samples, the first preset period can be set to 100 μs; when detecting medium-speed dynamic processes such as cell migration in fluorescently labeled samples, the first preset period can be set to 10 milliseconds (ms); when detecting slow processes such as cell cycle phases in fluorescently labeled samples, the first preset period can be set to 100 ms.

[0115] As one possible implementation, the control device is also connected to an atomic force microscope, and the above-mentioned S201 includes: acquiring a transverse force signal in the atomic force microscope; obtaining the current state based on the transverse force signal; and acquiring a brightness signal in a structure illumination microscope every second preset period.

[0116] The luminance signal includes the amount of luminance change.

[0117] For example, the second preset period can be a value pre-configured by the atomic force microscope and control equipment, or it can be a manually set value; this application embodiment does not impose specific limitations on this. For example, the first preset period can be 10ms. The second preset period is longer than the first preset period.

[0118] For example, the lateral force signal can reflect the friction, viscoelasticity, or dynamic deformation of the sample surface when the probe of an atomic force microscope scans the surface of a fluorescently labeled sample. For instance, if the lateral force signal fluctuates significantly (e.g., ±10 nanonewtons (nN)), it indicates that the fluorescently labeled sample is in an active state. As another example, if the lateral force signal changes gradually (e.g., ±1 nN), it indicates that the fluorescently labeled sample is in a stable state.

[0119] For example, the structured illumination microscope detects the change in brightness of the fluorescently labeled sample every second preset period, and then sends a signal carrying the change in brightness to the control device.

[0120] For example, if the transverse force of the probe detected by atomic force microscopy fluctuates greatly periodically (e.g., ±15nN), it indicates that the fluorescently labeled sample is undergoing severe deformation. If the fluorescence intensity is detected by structured illumination microscopy, the control equipment can determine that the fluorescently labeled sample is in an active state and that the fluorescent molecules are changing drastically, thereby reducing the illumination intensity of the structured illumination microscopy.

[0121] For example, if the periodic fluctuation of the probe's lateral force detected by the atomic force microscope is small (e.g., ±0.5 nN), it indicates that the fluorescently labeled sample is in a stable state. If the structure illumination microscope detects a small change in fluorescence intensity, the control equipment can determine that the fluorescently labeled sample is in a stable state and that the fluorescent molecules are changing little, thereby increasing the illumination intensity of the structure illumination microscope.

[0122] In the embodiments of this application, the sample state is dynamically determined by the lateral force signal from an atomic force microscope and the brightness signal from a structure illumination microscope; then, based on the sample state, the illumination intensity of the structure illumination microscope is adjusted to reduce phototoxicity.

[0123] In one possible embodiment of this application, the above-mentioned S202 includes: if the current state is an active state and the brightness change is greater than or equal to a first change threshold, determining that the light intensity adjustment amount is less than zero; if the current state is a stable state and the absolute value of the brightness change is less than or equal to a second change threshold, determining that the light intensity adjustment amount is greater than zero.

[0124] The second change threshold is less than the first change threshold.

[0125] For example, the first and second change thresholds can be values ​​pre-configured by the control device or manually set values; this application embodiment does not impose specific limitations on this. For instance, the first change threshold is 20%, and the second change threshold is 5%.

[0126] For example, if the light intensity adjustment is less than zero, it means that the light intensity of the structure illumination microscope needs to be reduced; if the light intensity adjustment is greater than zero, it means that the light intensity of the structure illumination microscope needs to be increased.

[0127] As an example, if the current state is active and the change in brightness is greater than or equal to the first change threshold, then there is a positive correlation between the change in brightness and the absolute value of the light intensity adjustment.

[0128] As another example, if the current state is a stable state and the absolute value of the brightness change is less than or equal to the second change threshold, then the brightness change is negatively correlated with the absolute value of the light intensity adjustment.

[0129] As an example, if the current state is active and the change in brightness is greater than or equal to the first change threshold, it means that the fluorescently labeled sample is in an active state (e.g., the frequency of the event signal is 120 Hz, and / or the fluctuation of the transverse force signal of the atomic force microscope is ±12 nN). The fluorescent molecules in the fluorescently labeled sample fluctuate violently. At this time, it is necessary to reduce the light intensity of the structured illumination microscope to reduce the phototoxic effect of the light intensity of the structured illumination microscope on the sample.

[0130] For example, if the detected event signal frequency is 120 Hz and the brightness change is 30%, it indicates that the fluorescence fluctuates violently due to microtubule reorganization during cell division. The fluorescence intensity in the cell division region jumps from 100 photons to 130 photons in 0.1 seconds, thus determining that the fluorescently labeled sample is in an active state. To avoid high-intensity light accelerating photobleaching and cell damage, the light intensity adjustment can be set to -150 mW / cm², so that the light intensity of the structured illumination microscope is reduced from 500 mW / cm² (initial light intensity) to 350 mW / cm² (target light intensity).

[0131] As another example, if the current state is a stable state and the change in brightness is less than or equal to the second change threshold, it indicates that the fluorescently labeled sample is in a stable state (e.g., the frequency of the event signal is 15 Hz, and / or the fluctuation of the transverse force signal of the atomic force microscope is ±0.3 nN). The fluorescent molecules in the fluorescently labeled sample are stable, and it is necessary to increase the illumination intensity of the structured illumination microscope to improve the quality of the microscopic imaging.

[0132] For example, if the detected event signal frequency is 15 Hz and the brightness change is 3%, it indicates that the cell has entered a metabolic stationary phase after completing cell division. The fluorescence intensity fluctuates slightly from 200 photons to 194 photons within 1 second, thus confirming that the fluorescently labeled sample is in a stable state. To improve imaging resolution, the illumination intensity adjustment can be set to 100 mW / cm², thereby increasing the illumination intensity of the structured illumination microscope from 300 mW / cm² to 400 mW / cm².

[0133] In the embodiments of this application, by combining the current state of the fluorescently labeled sample and the amount of brightness change, the amount of light intensity adjustment is comprehensively determined. This enables the fluorescently labeled sample to adapt to the currently adjusted target light intensity, thereby reducing the phototoxic effect of the light intensity of the structure illumination microscope on the sample.

[0134] In one possible embodiment of this application, the control device is also connected to an atomic force microscope. The method provided in this application embodiment further includes: acquiring a second microscopic image of a fluorescently labeled sample collected by an atomic force microscope; the above-mentioned S204 includes: obtaining a target microscopic image based on the first microscopic image, the second microscopic image and the timestamp of the event signal.

[0135] For example, the second microscopic image may be an image of the nanoscale morphology or mechanical properties of the sample surface independently acquired by an atomic force microscope, that is, the second microscopic image is used to reflect the surface morphology of the fluorescently labeled sample.

[0136] For example, the control device can periodically acquire a second microscopic image from the atomic force microscope. For instance, the atomic force microscope completes a scan of the fluorescently labeled sample every 30 ms and records the contraction and deformation of the cell membrane.

[0137] For example, the target microscopic image can be a fused microscopic image of a three-dimensional structure. The target microscopic image can simultaneously reflect the morphology and internal structure of the fluorescently labeled sample, and can dynamically display the changes in the fluorescently labeled sample.

[0138] For example, the first microscopic image acquired by structured illumination microscopy is a fluorescence signal image inside the fluorescently labeled sample (such as organelle structures), the second microscopic image generated by atomic force microscopy is a surface morphology image of the fluorescently labeled sample (such as cell membrane topological features), and the event camera records a precise time stamp of the event signal. Therefore, the first microscopic image, the second microscopic image, and the time stamp of the event signal can be integrated to obtain the target microscopic image.

[0139] Regarding how to obtain the target microscopic image based on the timestamps of the first microscopic image, the second microscopic image, and the event signal, please refer to the following embodiments, which will not be repeated here.

[0140] In the embodiments of this application, by acquiring a second microscopic image from an atomic force microscope, and based on the first microscopic image, the second microscopic image, and the timestamp of the event signal, a target microscopic image with high imaging quality can be obtained, thereby improving the microscopic imaging quality.

[0141] In one possible embodiment of this application, obtaining a target microscopic image based on a first microscopic image, a second microscopic image, and a timestamp of an event signal includes: synchronizing the timestamp of the event signal with the acquisition time of the first and second microscopic images to obtain aligned first and second microscopic images; and fusing the aligned first and second microscopic images in a three-dimensional spatial coordinate system to obtain the target microscopic image.

[0142] For example, the first and second microscopic images can be synchronized onto the same timeline based on the timestamp of the event signal to eliminate timing discrepancies caused by differences in equipment. For instance, a structured illumination microscope acquires the first microscopic image at time t=10.5ms, and an atomic force microscope acquires the second microscopic image at time t=10.3ms; the event signal detects a dynamic change at time t=10.4ms; the control device can align these three data points using an interpolation algorithm.

[0143] For example, a first microscopic image (e.g., XY plane) reflecting the internal structure of a fluorescently labeled sample and a second microscopic image (e.g., Z-axis height) reflecting the surface features of the fluorescently labeled sample can be fused in a three-dimensional coordinate system. For instance, by combining the location of mitochondria (X=5μm, Y=3μm) shown in a first microscopic image acquired by a structure illumination microscope after timestamping of event signals with the cell membrane indentation depth data (Z=120nm) shown in a second microscopic image acquired by an atomic force microscope, the true distribution of organelles in three-dimensional space can be reconstructed.

[0144] In the embodiments of this application, the acquisition time of the first and second microscopic images is synchronized by the timestamp of the event signal, and the target microscopic image is constructed in a three-dimensional spatial coordinate system, which may further improve the quality of microscopic imaging.

[0145] Figure 3 This is a schematic flowchart of another microscopic imaging method provided in the embodiments of this application.

[0146] For example, Figure 3 The microscopic imaging method shown can be performed by an electronic device or by a microscopic imaging device in an electronic device, such as a chip.

[0147] like Figure 3 As shown, the microscopic imaging method may include S301 to S312, and S301 to S312 are described in detail below.

[0148] S301. Acquire event signals from the event camera.

[0149] S302. Determine the current state of the fluorescently labeled sample based on the frequency of the event signal.

[0150] S303. Determine the amount of brightness change of the fluorescently labeled sample based on the intensity of the event signal.

[0151] S304. Determine whether the current state of the fluorescently labeled sample is active. If yes, proceed to S305; otherwise, proceed to S307.

[0152] S305. Determine whether the change in brightness is greater than or equal to the first change threshold. If so, proceed to S306.

[0153] S306. Ensure that the light intensity adjustment amount is less than zero.

[0154] S307. Determine whether the current state of the fluorescently labeled sample is static. If yes, proceed to S308.

[0155] S308. Determine whether the change in brightness is less than or equal to the second change threshold. If so, proceed to S309.

[0156] S309. Ensure that the light intensity adjustment is greater than zero.

[0157] S310. Send the illumination intensity adjustment to the structured illumination microscope.

[0158] S311. Acquire a first microscopic image in a structured illumination microscope and a second microscopic image in an atomic force microscope.

[0159] S312. Obtain the target microscopic image based on the first microscopic image, the second microscopic image, and the timestamp of the event signal.

[0160] For a detailed description of S301 to S312, please refer to Figures 1 to 12 above. Figure 2 And related descriptions, which will not be repeated here.

[0161] In the embodiments of this application, during the microscopic imaging of a fluorescently labeled sample using a structured illumination microscope, an event signal is acquired from an event camera; based on the event signal, the illumination intensity adjustment amount of the structured illumination microscope is determined; the illumination intensity adjustment amount is sent to the structured illumination microscope so that the structured illumination microscope adjusts its illumination intensity accordingly; since the event camera in this scheme can monitor the state of the fluorescently labeled sample, and the event signal is used to indicate that the brightness change of the fluorescently labeled sample is greater than a preset value, the illumination intensity adjustment amount of the structured illumination microscope can be determined through the event signal sent by the event camera, and the illumination intensity adjustment amount is sent to the structured illumination microscope; since the illumination intensity adjustment amount is related to the brightness change, the illumination intensity of the structured illumination microscope can be matched with the illumination intensity required for microscopic imaging of the sample, thereby reducing the phototoxicity of the sample caused by the illumination intensity of the structured illumination microscope, and thus improving the activity of the sample and the imaging quality.

[0162] It should be understood that the above examples are provided to help those skilled in the art understand the embodiments of this application, and are not intended to limit the embodiments of this application to the specific values ​​or scenarios exemplified. Those skilled in the art can obviously make various equivalent modifications or variations based on the above examples, and such modifications or variations also fall within the scope of the embodiments of this application.

[0163] The above text combined Figures 2 to 3 The microscopic imaging method provided in the embodiments of this application has been described in detail; the following will be combined with Figure 4 and Figure 5The apparatus embodiments of this application are described in detail below. It should be understood that the apparatus in the embodiments of this application can perform the various methods described in the foregoing embodiments of this application, that is, the specific working processes of the various products described below can be referred to the corresponding processes in the foregoing method embodiments.

[0164] The following is combined with Figure 4 The microscopic imaging apparatus provided in the embodiments of this application will be described in detail.

[0165] Figure 4 This is a schematic diagram of the structure of a microscopic imaging device provided in an embodiment of this application.

[0166] For example, such as Figure 4 As shown, the microscopic imaging apparatus 400 includes:

[0167] The acquisition module 410 is used to acquire the current state and brightness change of the fluorescently labeled sample;

[0168] The determination module 420 is used to determine the amount of light intensity adjustment based on the current state and the amount of brightness change;

[0169] The acquisition module 410 is also used to acquire the first microscopic image of the fluorescently labeled sample acquired by the structure illumination microscope when the illumination intensity of the structure illumination microscope is adjusted to the target illumination intensity, wherein the target illumination intensity is the illumination intensity obtained based on the illumination intensity adjustment amount.

[0170] Imaging module 430 is used to obtain a target microscopic image of the fluorescently labeled sample based on the first microscopic image.

[0171] As one possible implementation, the determining module 420 is specifically used to: if the current state is an active state and the brightness change is greater than or equal to a first change threshold, determine that the light intensity adjustment amount is less than zero; if the current state is a stable state and the absolute value of the brightness change is less than or equal to a second change threshold, determine that the light intensity adjustment amount is greater than zero and the second change threshold is less than the first change threshold.

[0172] As one possible implementation, the acquisition module 410 is specifically used to: acquire the event signal of the event camera; obtain the current state based on the frequency of the event signal; and obtain the brightness change based on the intensity of the event signal.

[0173] As one possible implementation, the determining module 420 is specifically used to: determine the current state as an active state if the frequency is greater than or equal to a first frequency threshold; and determine the current state as a stable state if the frequency is less than or equal to a second frequency threshold, wherein the second frequency threshold is less than the first frequency threshold.

[0174] As one possible implementation, the determining module 420 is also used to acquire a second microscopic image of the fluorescently labeled sample acquired by an atomic force microscope; specifically, the determining module 420 is used to obtain a target microscopic image based on the first microscopic image, the second microscopic image, and the timestamp of the event signal.

[0175] As one possible implementation, the determining module 420 is specifically used to: synchronize the timestamp of the event signal with the acquisition time of the first and second microscopic images to obtain aligned first and second microscopic images; and fuse the aligned first and second microscopic images in a three-dimensional spatial coordinate system to obtain the target microscopic image.

[0176] It should be noted that the aforementioned microscopic imaging device is embodied in the form of functional units. The term "module" here can be implemented in software and / or hardware, without specific limitations.

[0177] For example, a "module" can be a software program, hardware circuit, or a combination of both that implements the above functions. Hardware circuits may include application-specific integrated circuits (ASICs), electronic circuits, processors (e.g., shared processors, proprietary processors, or combined processors) and memory for executing one or more software or firmware programs, combined logic circuits, and / or other suitable components that support the described functions.

[0178] Therefore, the units of the various examples described in the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0179] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0180] For example, electronic device 500 includes: processor 510, memory 520 and executable program code 530.

[0181] For example, electronic device 500 includes one or more processors 510, which can support the electronic device 500 in implementing the method of generating the electronic device in the method embodiment. The processor 510 can be a general-purpose processor or a special-purpose processor. For example, the processor 510 can be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0182] For example, processor 510 can be used to control electronic device 500, execute software programs, and process data from the software programs. Electronic device 500 may also include a communication unit for receiving and transmitting signals.

[0183] For example, the electronic device 500 may include one or more memories 520 storing executable program code 530, which can be run by the processor 510 to generate instructions, causing the processor 510 to execute the microscopic imaging method described in the above method embodiments according to the instructions.

[0184] Optionally, the memory 520 may also store data. Optionally, the processor 510 may also read data stored in the memory 520, which may be stored at the same memory address as the executable program code 530, or the data may be stored at a different memory address than the executable program code 530.

[0185] For example, the processor 510 and memory 520 can be configured separately or integrated together, for example, integrated on a system-on-chip (SOC) of the terminal device.

[0186] For example, the memory 520 can be used to store related programs of the electronic device generation method provided in the embodiments of this application, and the processor 510 can be used to call the executable program code 530 stored in the memory 520 when controlling the electronic device 500 to execute the microscopic imaging method of the embodiments of this application.

[0187] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the microscopic imaging method of any of the foregoing embodiments.

[0188] The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, Digital Video Discs (DVDs), Compact Disc Read-Only Memory (CD-ROMs), microdrives, and magneto-optical disks, read-only memory (ROMs), random access memory (RAMs), erasable programmable read-only memory (EPROMs), electrically erasable programmable read-only memory (EEPROMs), dynamic random access memory (DRAMs), video random access memory (VRAMs), flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0189] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement a microscopic imaging method as described in the above embodiments.

[0190] In addition, the electronic device provided in the embodiments of this application may specifically be a chip, component or module. The electronic device may include a connected processor and a memory. The memory is used to store instructions. When the electronic device is running, the processor may call and execute the instructions to make the chip perform a microscopic imaging method in the above embodiments.

[0191] The electronic devices, computer-readable storage media, computer program products or chips provided in this application are all used to perform the corresponding microscopic imaging methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding microscopic imaging methods provided above, and will not be repeated here.

[0192] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0193] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0194] The above description is only a specific implementation of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application.

Claims

1. A method for microscopic imaging, characterized in that, The method is applied to a control device connected to a structure illumination microscope, and includes: The current state and brightness change of a fluorescently labeled sample are obtained, wherein the current state is used to represent the dynamic characteristics of the fluorescently labeled sample, and the current state includes an active state or a stable state, and the brightness change refers to the magnitude of the brightness change of the fluorescence signal in the fluorescently labeled sample. Based on the current state and the change in brightness, determine the amount of light intensity adjustment; When the illumination intensity of the structured illumination microscope is adjusted to the target illumination intensity, the first microscopic image of the fluorescently labeled sample is acquired by the structured illumination microscope, wherein the target illumination intensity is the illumination intensity obtained based on the illumination intensity adjustment amount; Based on the first microscopic image, a target microscopic image of the fluorescently labeled sample is obtained; The step of determining the light intensity adjustment amount based on the current state and the brightness change includes: If the current state is the active state, and the brightness change is greater than or equal to the first change threshold, then the light intensity adjustment amount is determined to be less than zero. If the current state is the stable state, and the absolute value of the brightness change is less than or equal to the second change threshold, then the light intensity adjustment amount is determined to be greater than zero, and the second change threshold is less than the first change threshold.

2. The method according to claim 1, characterized in that, The control device is also connected to an event camera, and the acquisition of the current state and brightness change of the fluorescently labeled sample includes: Acquire the event signal from the event camera; The current state is obtained based on the frequency of the event signal; The change in brightness is obtained based on the intensity of the event signal.

3. The method according to claim 2, characterized in that, The process of obtaining the current state based on the frequency of the event signal includes: If the frequency is greater than or equal to the first frequency threshold, the current state is determined to be the active state; If the frequency is less than or equal to the second frequency threshold, the current state is determined to be the stable state, where the second frequency threshold is less than the first frequency threshold.

4. The method according to claim 2, characterized in that, The control device is also connected to an atomic force microscope, and the method further includes: Obtain a second microscopic image of the fluorescently labeled sample using the atomic force microscope; The step of obtaining the target microscopic image of the fluorescently labeled sample based on the first microscopic image includes: The target microscopic image is obtained based on the timestamps of the first microscopic image, the second microscopic image, and the event signal.

5. The method according to claim 4, characterized in that, The process of obtaining the target microscopic image based on the timestamps of the first microscopic image, the second microscopic image, and the event signal includes: The timestamp of the event signal is synchronized with the acquisition time of the first and second microscopic images to obtain the aligned first and second microscopic images. The aligned first and second microscopic images are fused in a three-dimensional coordinate system to obtain the target microscopic image.

6. A device for microscopic imaging, characterized in that, An apparatus for use in a control device connected to a structure illumination microscope, the apparatus comprising: The acquisition module is used to acquire the current state and brightness change of the fluorescently labeled sample. The current state is used to represent the dynamic characteristics of the fluorescently labeled sample. The current state includes an active state or a stable state. The brightness change refers to the magnitude of the brightness change of the fluorescence signal in the fluorescently labeled sample. The determining module is used to determine the light intensity adjustment amount based on the current state and the brightness change amount; The acquisition module is further configured to acquire a first microscopic image of the fluorescently labeled sample when the illumination intensity of the structure illumination microscope is adjusted to the target illumination intensity, wherein the target illumination intensity is the illumination intensity obtained based on the illumination intensity adjustment amount; An imaging module is used to obtain a target microscopic image of the fluorescently labeled sample based on the first microscopic image; The determining module is specifically used to determine that if the current state is the active state and the brightness change is greater than or equal to the first change threshold, the light intensity adjustment amount is less than zero; if the current state is the stable state and the absolute value of the brightness change is less than or equal to the second change threshold, the light intensity adjustment amount is greater than zero, and the second change threshold is less than the first change threshold.

7. A microscopic imaging system, characterized in that, The microscopic imaging system includes: a control device and a structured illumination microscope, the control device being connected to the structured illumination microscope, and the control device being used to perform the method as described in any one of claims 1 to 5.

8. A control device, characterized in that, The control device includes: Memory, used to store executable program code; A processor for calling and running the executable program code from the memory, causing the control device to perform the method as described in any one of claims 1 to 5.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 5.

Images