Microscopic imaging method, device, equipment, system and storage medium

Abstract

The invention provides a microscopic imaging method, device, equipment and system and a storage medium, and relates to the technical field of microscopic imaging, and the method comprises the steps: obtaining the current state and brightness variation of a fluorescence labeled sample; determining an illumination intensity adjustment amount based on the current state and the brightness variation; when the illumination intensity of the structural illumination microscope is adjusted to target illumination intensity, a first microscopic image of the fluorescence labeling sample collected by the structural illumination microscope is obtained, and the target illumination intensity is the illumination intensity obtained based on the illumination intensity adjusting amount; and obtaining a target microscopic image of the fluorescence labeled sample based on the first microscopic image. According to the method, the influence of phototoxicity on the sample can be reduced, so that the imaging quality is improved.

Description

Technical Field

[0001] The present application relates to the field of microscopic imaging technology, and more specifically, to a method, apparatus, device, system and storage medium for microscopic imaging in the field of microscopic imaging technology. Background Art

[0002] Currently, in the field of microscopic imaging, compared with traditional optical microscopes, structured illumination microscopes are widely used in the fields of biological research, materials science or medical diagnosis due to their advantages of high resolution and high-speed imaging.

[0003] However, although structured illumination microscopes can obtain high-resolution microscopic images of the internal structure of samples, they require a relatively high illumination intensity during the microscopic imaging process, which may cause phototoxicity to the samples, thereby affecting the activity and imaging quality of the samples.

[0004] Therefore, how to reduce the impact of phototoxicity on samples and thus improve the imaging quality is a technical problem that urgently needs to be solved at present. Summary of the Invention

[0005] The present application provides a method, apparatus, device, system and storage medium for microscopic imaging, and this method can reduce the impact of phototoxicity on samples and thus improve the imaging quality.

[0006] In a first aspect, a method for microscopic imaging is provided, which is applied to a control device. The control device is connected to a structured illumination microscope. The method includes: obtaining the current state and the brightness change amount of a fluorescently labeled sample; determining the illumination intensity adjustment amount based on the current state and the brightness change amount; when the illumination intensity of the structured illumination microscope is adjusted to a target illumination intensity, obtaining a first microscopic image of the fluorescently labeled sample collected by the structured illumination microscope, where the target illumination intensity is the illumination intensity obtained based on the illumination intensity adjustment amount; and obtaining a target microscopic image of the fluorescently labeled sample based on the first microscopic image.

[0007] In an embodiment of the present application, the current state and the brightness change amount of the fluorescently labeled sample are obtained; based on the current state and the brightness change amount, the illumination intensity adjustment amount is determined; when the illumination intensity of the structured illumination microscope is adjusted to the target illumination intensity, the first microscopic image of the fluorescently labeled sample collected by the structured illumination microscope is obtained; since this solution can determine how to adjust the illumination intensity of the structured illumination microscope through the current state and the brightness change amount, the fluorescently labeled sample can be adapted to the current target illumination intensity, thereby reducing the impact of the illumination intensity of the structured illumination microscope on the sample caused by phototoxicity; since the target illumination intensity is the illumination intensity obtained based on the illumination intensity adjustment amount, when the illumination intensity of the structured illumination microscope is adjusted to the target illumination intensity, the first microscopic image is obtained, and based on the first microscopic image, the target microscopic image with higher imaging quality is obtained; based on this, this solution can reduce the impact of phototoxicity on the sample during the microscopic imaging of the sample by the structured illumination microscope, thereby improving the imaging quality.

[0008] Combined with the first aspect, in some possible implementation manners, determining the illumination intensity adjustment amount based on the current state and the brightness change amount includes: if the current state is the active state and the brightness change amount is greater than or equal to the first change threshold, determining that the illumination intensity adjustment amount is less than zero; if the current state is the stable state and the absolute value of the brightness change amount is less than or equal to the second change threshold, determining that the illumination intensity adjustment amount is greater than zero, and the second change threshold is less than the first change threshold.

[0009] In an embodiment of the present application, by combining the current state and the brightness change amount of the fluorescently labeled sample, the illumination intensity adjustment amount is comprehensively determined, so that the fluorescently labeled sample can be adapted to the current adjusted target illumination intensity, thereby reducing the impact of the illumination intensity of the structured illumination microscope on the sample caused by phototoxicity.

[0010] Combined with the first aspect and the above implementation manners, in some possible implementation manners, the control device is further connected to the event camera, and obtaining the current state and the brightness change amount of the fluorescently labeled sample includes: obtaining the event signal of the event camera; obtaining the current state based on the frequency of the event signal; obtaining the brightness change amount based on the intensity of the event signal.

[0011] In an embodiment of the present application, by obtaining the event signal of the event camera and based on the characteristics (frequency and intensity) of the event signal, the current state and the brightness change amount of the fluorescently labeled sample are quickly and accurately obtained, so as to be able to timely adjust the illumination intensity adjustment amount of the structured illumination microscope, thereby reducing the impact of the illumination intensity of the structured illumination microscope on the sample caused by phototoxicity.

[0012] As a possible implementation manner, obtaining the event signal of the event camera includes: obtaining the event signal every first preset period.

[0013] As a possible implementation, the control device is also connected to an atomic force microscope to obtain the current state and the amount of brightness change of the fluorescently labeled sample, including: obtaining the lateral force signal in the atomic force microscope; based on the lateral force signal, obtaining the current state; obtaining the brightness signal in the structured illumination microscope every second preset period, and the brightness signal includes the amount of brightness change.

[0014] In the embodiments of the present application, the state of the sample is dynamically judged through the lateral force signal of the atomic force microscope and the brightness signal of the structured illumination microscope; then, based on the sample state, the illumination intensity of the structured illumination microscope is adjusted to reduce phototoxicity.

[0015] Combined with the first aspect and the above implementation, in some possible implementations, based on the frequency of the event signal, the current state is obtained, including: if the frequency is greater than or equal to the first frequency threshold, determining that the current state is an active state; if the frequency is less than or equal to the second frequency threshold, determining that the current state is a stable state, and the second frequency threshold is less than the first frequency threshold.

[0016] In the embodiments of the present application, through the frequency of the event signal, the current state of the fluorescently labeled sample can be quickly and accurately determined, and then it is convenient to determine the amount of illumination intensity adjustment subsequently, so that the fluorescently labeled sample can adapt to the current target illumination intensity, and further reduce the impact of the illumination intensity of the structured illumination microscope on the sample caused by phototoxicity.

[0017] Combined with the first aspect and the above implementation, in some possible implementations, the control device is also connected to an atomic force microscope, and the method provided by the embodiments of the present application further includes: obtaining a second microscopic image of the fluorescently labeled sample collected by the atomic force microscope; based on the first microscopic image, obtaining a target microscopic image of the fluorescently labeled sample, including: based on the first microscopic image, the second microscopic image, and the time stamp of the event signal, obtaining the target microscopic image.

[0018] In the embodiments of the present application, by obtaining the second microscopic image in the atomic force microscope and based on the first microscopic image, the second microscopic image, and the time stamp of the event signal, a target microscopic image with higher imaging quality can be obtained, thereby improving the microscopic imaging quality.

[0019] Combined with the first aspect and the above implementation, in some possible implementations, based on the first microscopic image, the second microscopic image, and the time stamp of the event signal, obtaining the target microscopic image includes: synchronizing the time stamp of the event signal with the acquisition times of the first microscopic image and the second microscopic image to obtain the aligned first microscopic image and second microscopic image; performing a fusion process on the aligned first microscopic image and second microscopic image in a three-dimensional space coordinate system to obtain the target microscopic image.

[0020] In an embodiment of the present application, the acquisition times of the first microscopic image and the second microscopic image are synchronized by the timestamps of the event signals, and a target microscopic image is constructed in a three-dimensional space coordinate system, which may further improve the microscopic imaging quality.

[0021] In a second aspect, a microscopic imaging device is provided, which is applied to a control device. The control device is connected to a structured illumination microscope. The device includes: An acquisition module, configured to acquire the current state and the brightness change amount of the fluorescently labeled sample; A determination module, configured to determine the illumination intensity adjustment amount based on the current state and the brightness change amount; The acquisition module is further configured to acquire a first microscopic image of the fluorescently labeled sample by the structured illumination microscope when the illumination intensity of the structured illumination microscope is adjusted to a target illumination intensity, and the target illumination intensity is the illumination intensity obtained based on the illumination intensity adjustment amount; An imaging module, configured to obtain a target microscopic image of the fluorescently labeled sample based on the first microscopic image.

[0022] As a possible implementation, the determination module is specifically configured to: if the current state is an active state and the brightness change amount is greater than or equal to a first change threshold, determine that the illumination intensity adjustment amount is less than zero; if the current state is a stable state and the absolute value of the brightness change amount is less than or equal to a second change threshold, determine that the illumination intensity adjustment amount is greater than zero, and the second change threshold is less than the first change threshold.

[0023] As a possible implementation, the acquisition module is specifically configured to: acquire the event signals of the event camera; obtain the current state based on the frequency of the event signals; obtain the brightness change amount based on the intensity of the event signals.

[0024] As a possible implementation, the determination module is specifically configured to: if the frequency is greater than or equal to a first frequency threshold, determine that the current state is an active state; if the frequency is less than or equal to a second frequency threshold, determine that the current state is a stable state, and the second frequency threshold is less than the first frequency threshold.

[0025] As a possible implementation, the determination module is further configured to acquire a second microscopic image of the fluorescently labeled sample by the atomic force microscope; the determination module is specifically configured to: obtain the target microscopic image based on the first microscopic image, the second microscopic image, and the timestamp of the event signal.

[0026] As a possible implementation, the determination module is specifically configured to: synchronize the timestamp of the event signal with the acquisition times of the first microscopic image and the second microscopic image to obtain the aligned first microscopic image and second microscopic image; perform fusion processing on the aligned first microscopic image and second microscopic image in a three-dimensional space coordinate system to obtain the target microscopic image.

[0027] In a third aspect, an electronic device is provided, including a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the electronic device executes the method in the above first aspect or any possible implementation manner of the first aspect.

[0028] In a fourth aspect, a microscopy imaging system is provided, including: a control device and a structured illumination microscope. The control device is connected to the structured illumination microscope, and the control device executes the method in the above first aspect or any possible implementation manner of the first aspect.

[0029] As a possible implementation manner, the control device is further connected to an event camera and an atomic force microscope.

[0030] In a fifth aspect, a computer program product is provided, including: computer program code. When the computer program code runs on a computer, the computer is enabled to execute the method in the above first aspect or any possible implementation manner of the first aspect.

[0031] In a sixth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores computer program code. When the computer program code runs on a computer, the computer is enabled to execute the method in the above first aspect or any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG. 1(a) is a schematic structural diagram of a microscopy imaging system provided by an embodiment of the present application; FIG. 1(b) is a schematic scenario diagram of a microscopy imaging system provided by an embodiment of the present application; Figure 2 is a schematic flow diagram of a method for microscopy imaging provided by an embodiment of the present application; Figure 3 is a schematic flow diagram of another method for microscopy imaging provided by an embodiment of the present application; Figure 4 is a schematic structural diagram of a device for microscopy imaging provided by an embodiment of the present application; Figure 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] The technical solutions in the present application will be clearly and elaborately described below in conjunction with the accompanying drawings. Among them, in the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B may mean A or B. The "and / or" in the text is merely a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B may mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present application, "a plurality of" means two or more than two.

[0034] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features.

[0035] Before introducing the solutions of the embodiments of the present application, first, a glossary of technical terms that may be involved in the embodiments of the present application will be given.

[0036] (1) Structured Illumination Microscopy (SIM): An optical microscopy technique for achieving super-resolution by spatially modulating the illumination light field. The sample is illuminated with a specific optical structure illumination pattern, and a high-resolution image is reconstructed using computer algorithms. SIM can break through the resolution limit of traditional optical microscopes and provide clearer imaging means for research in fields such as cell biology and molecular biology.

[0037] (2) Atomic Force Microscope (AFM): A high-resolution scanning probe microscope based on the interaction force between a probe and the surface of a sample, capable of imaging and analyzing the surface of various materials at the atomic level. AFM obtains information about the topography and properties of the sample surface by detecting the interaction force between the probe and the sample surface.

[0038] (3) Event Camera: Also known as a Dynamic Vision Sensor (DVS), a bio-inspired vision sensor based on the asynchronous pixel response principle. Different from the principle of traditional cameras, the event camera does not capture images at fixed time intervals, but triggers event outputs based on pixel-level brightness changes. For example, when the brightness change of a pixel exceeds a certain threshold, the event camera will output an event signal containing information such as the position of the pixel and the timestamp of the brightness change. This photosensitive imaging principle enables the event camera to have the characteristics of high temporal resolution and low data redundancy, and can quickly capture dynamic changes in the scene.

[0039] (4) Phototoxicity: The oxidative damage effect produced by high-energy photons on biological samples (such as living cells).

[0040] (5) Fluorescent Labeled Sample: A biological sample in which a fluorescent probe is specifically bound to a target substance (such as a protein, an organelle, etc.) by physical or chemical means.

[0041] Currently, in the field of microscopic imaging, compared with traditional optical microscopes, structured illumination microscopes are widely used in the fields of biological research, materials science, or medical diagnosis due to their advantages of high resolution and high-speed imaging.

[0042] However, although structured illumination microscopes can obtain high-resolution microscopic images of the internal structure of samples, they require a relatively high illumination intensity during the microscopic imaging process, which may cause phototoxicity to the samples, thereby affecting the activity and imaging quality of the samples.

[0043] Therefore, in the process of microscopic imaging of samples by structured illumination microscopes, how to reduce the impact of phototoxicity on the samples and thus improve the imaging quality is a technical problem that urgently needs to be solved at present.

[0044] In view of this, this solution provides a method, device, equipment, system, and storage medium for microscopic imaging. In the process of microscopic imaging of samples by structured illumination microscopes, this method can reduce the impact of phototoxicity on the samples.

[0045] To illustrate the technical solutions described in this application, specific embodiments will be used for illustration below. First, a microscopic imaging system according to an embodiment of this application will be introduced.

[0046] Among them, FIG. 1(a) is a schematic structural diagram of a microscopic imaging system provided by an embodiment of this application; FIG. 1(b) is a schematic scene diagram of a microscopic imaging system provided by an embodiment of this application.

[0047] Exemplarily, as shown in FIG. 1(a), the microscopic imaging system 100 includes: a control device 110 and a structured illumination microscope 120.

[0048] Optionally, the control device 110 is further connected to an event camera 130 and an atomic force microscope 140.

[0049] Exemplarily, the control device 110 can be in a wired communication connection or a wireless communication connection with the structured illumination microscope 120, the event camera 130, and the atomic force microscope 140. For example, the wired communication connection can be an Ethernet connection or other wired communication connections, and the embodiments of the present application do not make specific limitations thereto. For example, the wireless communication connection can be a Wi-Fi (Wireless Fidelity) connection, a Bluetooth (BT) connection, a Near Field Communication (NFC), etc., and the embodiments of the present application do not make specific limitations thereto.

[0050] Exemplarily, the control device 110 can be a device with data processing functions, such as a computer, a mobile phone, a server, etc., and the embodiments of the present application do not make specific limitations thereto.

[0051] Exemplarily, the control device 110 can be used to integrate data from other devices (such as the structured illumination microscope 120, the event camera 130, and the atomic force microscope 140), can generate the light intensity adjustment amount of the structured illumination microscope 120, and can also combine the microscopic images in each device to obtain a target microscopic image. As shown in FIG. 1(b), the control device 110 can include a control unit and a data processing unit.

[0052] Exemplarily, the structured illumination microscope 120 can be an optical imaging module. As shown in FIG. 1(b), the structured illumination microscope 120 can include a light source, a spatial light modulator, a total internal reflection (TIR) prism, a zoom lens, a mirror, etc. Among them, the light source can use a high-power laser (such as a 488-nanometer (nm) blue laser) to provide coherent illumination for the structured illumination microscope and excite the fluorescence labeling of the sample; the light intensity of the structured illumination microscope 120 can be adjusted through the light source; the spatial light modulator modulates the laser wavefront by loading a preset grating pattern (such as a sine stripe) to achieve the structured illumination required for super-resolution imaging. The TIR prism can excite the fluorescence labeling near the 200-nm thin layer on the sample surface and reduce the interference of the deep background. The zoom lens dynamically adjusts the size of the imaging area to adapt to different resolution requirements.

[0053] Exemplarily, the event camera 130 can be used to capture the state of the sample with a time resolution of microseconds and output a pixel-level event stream (i.e., the event signal in the following text).

[0054] Exemplarily, the atomic force microscope 140 can be a detection module. As shown in Fig. 1(b), the atomic force microscope 140 can include: an AFM probe, a piezoelectric scanning stage, a laser, and a photoelectric detection. The probe is used to carry a nanoscale sharp probe (curvature radius < 10 nm), and scans the surface topography of the sample in contact or tapping mode. The piezoelectric scanning stage controls the three-dimensional movement of the sample stage with sub-nanometer accuracy. The laser and the photoelectric detection detect the displacement of the reflected laser of the cantilever beam, convert the deformation of the probe into an electrical signal, and generate a surface height map and mechanical parameters (such as elastic modulus) in real time.

[0055] Optionally, Fig. 1(b) further includes a mirror and a beam splitter. The mirror is located on the right side of the beam splitter, connects the beam splitter and the spatial light modulator, and is used to turn the working optical path transmitted by the beam splitter to the spatial light modulator or subsequent optical elements (such as a TIR prism). The beam splitter is located below the event camera and connects the laser, the photodetector, and the spatial light modulator. The beam splitter divides the main beam output by the laser into two paths, one path is reflected to the photodetector, and the other path is transmitted to the spatial light modulator.

[0056] Combined with the architecture shown in Fig. 1, the method for microscopic imaging provided by the embodiments of the present application will be described in detail below.

[0057] Figure 2 It is a schematic flow chart of a method for microscopic imaging provided by the embodiments of the present application.

[0058] It should be understood that the embodiments of the present application do not particularly limit the execution subject of the method for microscopic imaging. As long as it can communicate according to the method for microscopic imaging of the embodiments of the present application by running a program recorded with the code of the method for microscopic imaging of the embodiments of the present application. For example, the execution subject of the method for microscopic imaging provided by the embodiments of the present application can be the electronic device described in the above embodiments, or a functional module in the electronic device that can call and execute the program, such as a chip. The present application does not limit this.

[0059] Exemplarily, the electronic device can be a control device, and the control device is connected to the structured illumination microscope. The control device can be a device with data processing functions, such as a computer, a mobile phone, a server, etc. The embodiments of the present application do not make specific limitations on this.

[0060] As Figure 2 shown, hereinafter, S201 to S204 will be used to exemplarily illustrate the method for microscopic imaging provided by the embodiments of the present application.

[0061] S201. Obtain the current state and the brightness change amount of the fluorescently labeled sample.

[0062] Exemplarily, the fluorescently labeled sample can be a living cell (e.g., a stem cell, a primary cell, etc.) or a subcellular structure (e.g., a mitochondrion, an endoplasmic reticulum, a cytoskeleton, etc.). The embodiments of the present application do not make specific limitations thereto.

[0063] Exemplarily, the fluorescently labeled sample can refer to a biological sample in which a target substance (e.g., a protein, an organelle, etc.) is specifically bound to a fluorescent probe (e.g., a fluorescent dye or a fluorescent protein); the fluorescence distribution in the fluorescently labeled sample can be obtained through a fluorescence microscope (e.g., a structured illumination microscope), and then a microscopic image of the fluorescently labeled sample can be obtained. The fluorescently labeled sample can emit light under excitation at a specific wavelength.

[0064] Exemplarily, the current state can be used to represent the dynamic characteristics of the fluorescently labeled sample, such as the behavioral characteristics of cells under different physiological conditions.

[0065] For example, the current state can be a stable state. The basal metabolic level of the fluorescently labeled sample in the stable state is stable, the energy consumption is stable, and there is no significant state such as division, etc., and it can maintain the cell morphology, such as the resting period of adherent cells.

[0066] Again, for example, the current state can also be an active state; the metabolic activity of the fluorescently labeled sample in the active state is significantly enhanced, and the cell is undergoing dynamic processes such as division, migration, phagocytosis, signal transduction, etc.

[0067] Exemplarily, the amount of brightness change can refer to the amplitude of the brightness change of the fluorescence signal in the fluorescently labeled sample.

[0068] For the specific description of S201, reference can be made to the following embodiments, and details are not described herein again.

[0069] S202. Determine the amount of illumination intensity adjustment based on the current state and the amount of brightness change.

[0070] Exemplarily, the amount of illumination intensity adjustment can refer to the amount of adjustment of the illumination intensity of the structured illumination microscope. For example, if the amount of illumination intensity adjustment is greater than zero, it means to increase the illumination intensity of the structured illumination microscope; again, for example, if the amount of illumination intensity adjustment is less than zero, it means to decrease the illumination intensity of the structured illumination microscope.

[0071] It can be understood that the amount of illumination intensity adjustment is determined by comprehensively considering the dynamic characteristics and the amplitude of the brightness change of the fluorescently labeled sample. For example, when the fluorescently labeled sample is in the active state and the amplitude of the brightness change of the fluorescence signal is large, it is necessary to decrease the illumination intensity of the structured illumination microscope to avoid phototoxicity to the fluorescently labeled sample caused by too high illumination intensity of the structured illumination microscope.

[0072] For another example, when the fluorescently labeled sample is in a stable state and the change in the brightness of the fluorescence signal is small, it is necessary to increase the illumination intensity of the structured illumination microscope to avoid the quality of the microscopic image collected by the structured illumination microscope being too low due to insufficient illumination.

[0073] For the specific description of S202, reference can be made to the following embodiments and will not be elaborated here.

[0074] S203. When the illumination intensity of the structured illumination microscope is adjusted to the target illumination intensity, obtain the first microscopic image of the fluorescently labeled sample collected by the structured illumination microscope.

[0075] Among them, the target illumination intensity is the illumination intensity obtained based on the illumination intensity adjustment amount.

[0076] As an example, when the illumination intensity adjustment amount is determined, the illumination intensity adjustment amount can be sent to the structured illumination microscope, and then the structured illumination microscope adjusts its illumination intensity to the target illumination intensity based on the illumination intensity adjustment amount.

[0077] As another example, when the illumination intensity adjustment amount is determined, the current illumination intensity of the structured illumination microscope can be obtained; based on the illumination intensity adjustment amount and the current illumination intensity, the target illumination intensity is determined; and then the target illumination intensity is sent to the structured illumination microscope.

[0078] Exemplarily, by adjusting the illumination intensity of the structured illumination microscope, the illumination intensity of the structured illumination microscope can be matched in real time with the current situation of the fluorescently labeled sample, avoiding the influence of a fixed illumination intensity on the fluorescently labeled sample or on the microscopic imaging quality.

[0079] Regarding how the structured illumination microscope collects the fluorescently labeled sample, reference can be made to the prior art and will not be elaborated here.

[0080] Exemplarily, the control device can periodically obtain the first microscopic image from the structured illumination microscope. The first microscopic image can be a multi-frame raw image with structured light stripes, usually a two-dimensional image set in multiple directions (e.g., 0°, 120°, 240°) and multiple phases (e.g., 0, 1 / 3 cycle, 2 / 3 cycle).

[0081] Exemplarily, the first microscopic image of the fluorescently labeled sample collected by the structured illumination microscope can be used to reflect the internal structure of the fluorescently labeled sample.

[0082] S204. Based on the first microscopic image, obtain the target microscopic image of the fluorescently labeled sample.

[0083] Exemplarily, the target microscopic image can be a super-resolution image obtained by processing the first microscopic image (e.g., multiple-frame fringe images) collected by a structured illumination microscope through a specific algorithm. Its resolution breaks through the optical diffraction limit and can clearly display subcellular structures (such as microtubules, mitochondrial cristae, etc.). For example, the specific algorithm can be a multi-directional interpolation algorithm, etc., and the embodiments of the present application do not make specific limitations thereto.

[0084] For example, nine first microscopic images (first microscopic images with three directions and three phases) are processed to obtain a single-frame target microscopic image.

[0085] In an embodiment of the present application, the current state and the brightness change amount of the fluorescently labeled sample are obtained; based on the current state and the brightness change amount, the light intensity adjustment amount is determined; when the light intensity of the structured illumination microscope is adjusted to the target light intensity, the first microscopic image of the fluorescently labeled sample collected by the structured illumination microscope is obtained; since this solution can determine how to adjust the light intensity of the structured illumination microscope through the current state and the brightness change amount, the fluorescently labeled sample can be adapted to the current target light intensity, thereby reducing the impact of the light intensity of the structured illumination microscope on the sample due to phototoxicity; since the target light intensity is the light intensity obtained based on the light intensity adjustment amount, when the light intensity of the structured illumination microscope is adjusted to the target light intensity, the first microscopic image is obtained, and based on the first microscopic image, a target microscopic image with higher imaging quality is obtained; based on this, this solution can reduce the impact of phototoxicity on the sample during the microscopic imaging of the sample by the structured illumination microscope, thereby improving the imaging quality.

[0086] In a possible embodiment of the present application, the control device is further connected to an event camera, and the above S201 includes: obtaining the event signal of the event camera; obtaining the current state based on the frequency of the event signal; obtaining the brightness change amount based on the intensity of the event signal.

[0087] Exemplarily, the event camera can asynchronously trigger an event signal based on the brightness change of pixels in the fluorescently labeled sample; for example, the event camera can output an event signal when the pixel brightness change exceeds a certain threshold. For example, at the moment of cell division of the fluorescently labeled sample, local fluorescence changes can trigger multiple event signals, while no event signals are output in the stable region.

[0088] Exemplarily, the event signal may include position information, intensity, and timestamp. The position of the event signal can be used to reflect the coordinates of the pixel in the fluorescently labeled sample. For example, (x, y) = (320, 240). The intensity of the event signal can represent the absolute magnitude of the pixel brightness change detected by the event camera, that is, the amplitude of the pixel brightness change within a specific time (e.g., ΔL = 30 gray values or number of photons). The timestamp of the event signal can be used to reflect the moment when the event occurs. Optionally, the event signal may further include: polarity; the polarity can be used to reflect the direction of the brightness change. For example, the polarity may include +1 and -1; +1 may indicate an increase in brightness (e.g., excitation of fluorescent molecules), and -1 may indicate a decrease in brightness (e.g., photobleaching).

[0089] As an example, based on the frequency of the event signal, the current state is obtained, including: if the frequency is greater than or equal to the first frequency threshold, determining that the current state is an active state; if the frequency is less than or equal to the second frequency threshold, determining that the current state is a stable state.

[0090] Wherein, the second frequency threshold is less than the first frequency threshold.

[0091] Exemplarily, the first frequency threshold and the second frequency threshold can be pre-configured values of the event camera and the control device, or can be manually set values, and the embodiments of the present application do not make specific limitations thereon. For example, the first frequency threshold can be 100 Hertz (Hz), and the second frequency threshold can be 20 Hz.

[0092] Exemplarily, the frequency of the event signal can represent the number of event signals output by the event camera within a preset duration. Through the frequency of the event signal, the activity degree during the dynamic change process of the fluorescently labeled sample can be represented. It can be understood that since the event signal can be triggered when the pixel brightness change in the fluorescently labeled sample exceeds a certain threshold, the number of event signals within the preset duration can be used to reflect the current state of the fluorescently labeled sample.

[0093] Exemplarily, the frequency of the event signal can be determined by the fixed time window method. For example, by setting a fixed period (e.g., 10 ms), the number of event signals within each fixed period is counted. For example, if 50 event signals are detected within 10 ms, the frequency of the event signal can be 5000 Hz.

[0094] As an example, when the frequency of the event signal is greater than or equal to the first frequency threshold, it can indicate that the number of event signals output by the event camera within the preset duration is large. At this time, the brightness change amounts of a large number of pixels in the fluorescently labeled sample exceed the set threshold, and the fluorescently labeled sample is in an active state.

[0095] For example, if the frequency of the event signal is 120 Hz (greater than 100 Hz (the first frequency threshold is 100 Hz)), that is, 12,000 event signals are detected within 1 second, it can be said that the fluorescently labeled sample is in an active state such as cell division and rapid movement, resulting in violent fluctuations in local fluorescence; when collecting microscopic images of fluorescently labeled samples, if the brightness changes greatly, it is necessary to reduce the light intensity of the structured illumination microscope to reduce phototoxicity.

[0096] As another example, the frequency of the event signal is less than or equal to the second frequency threshold, which may indicate that the number of event signals output by the event camera within a preset time period is small. At this time, the brightness change value of a small number of pixels in the fluorescently labeled sample exceeds the set threshold, and the fluorescently labeled sample is in a stable state.

[0097] For example, if the frequency of the event signal is 10 Hz (less than 20 Hz (the second frequency threshold is 20 Hz)), and 2000 event signals are detected within 1 second, it can be said that the fluorescently labeled sample is in a stable state and the fluorescent signal tends to be stable; when collecting microscopic images of fluorescently labeled samples, if the brightness change is small, it is necessary to increase the illumination intensity of the structured illumination microscope to improve the microscopic imaging quality.

[0098] As another example, the frequency of the event signal is less than the first frequency threshold and greater than the second frequency threshold, which may indicate that the number of event signals output by the event camera within a preset time period is moderate. In this case, the illumination intensity of the structured illumination microscope may not be adjusted to avoid frequent switching.

[0099] It can be understood that the frequency of the event signal can be used to quickly and accurately determine the current state of the fluorescently labeled sample, thereby facilitating the subsequent determination of the light intensity adjustment amount so that the fluorescently labeled sample can adapt to the current target light intensity, thereby reducing the phototoxic effect of the light intensity of the structured illumination microscope on the sample.

[0100] In an embodiment of the present application, by acquiring the event signal of the event camera and based on the characteristics (frequency and intensity) of the event signal, the current state and brightness change of the fluorescent-labeled sample are quickly and accurately obtained, so that the light intensity of the structured illumination microscope can be adjusted in time, thereby reducing the phototoxic effect of the light intensity of the structured illumination microscope on the sample.

[0101] As a possible implementation manner, acquiring the event signal of the event camera includes: acquiring the event signal every first preset period.

[0102] As an example, the first preset period may be a value preconfigured by the event camera and the control device, or may be a manually set value, which is not specifically limited in the embodiment of the present application. For example, the first preset period may be 100 microseconds (μs).

[0103] As another example, the first preset period can be determined according to the content of the fluorescence-labeled sample to be detected.

[0104] For example, when the content of the fluorescence-labeled sample to be detected is the detection content of a fast dynamic process such as calcium ion transient, the first preset period can be set to 100 μs; when the content of the fluorescence-labeled sample to be detected is the detection content of a medium-speed dynamic process such as cell migration, the first preset period can be set to 10 milliseconds (ms); when the content of the fluorescence-labeled sample to be detected is the detection content of a slow process such as the cell cycle stage, the first preset period can be set to 100 ms.

[0105] As a possible implementation manner, the control device is further connected to an atomic force microscope. The above S201 includes: acquiring a lateral force signal in the atomic force microscope; obtaining the current state based on the lateral force signal; and acquiring a brightness signal in the structured illumination microscope every second preset period.

[0106] Wherein, the brightness signal includes a brightness change amount.

[0107] Exemplarily, the second preset period can be a value preconfigured by the atomic force microscope and the control device, or a value set manually. The embodiments of the present application do not make specific limitations thereto. For example, the first preset period can be 10 ms. The second preset period is greater than the first preset period.

[0108] Exemplarily, the lateral force signal can reflect the surface friction, viscoelasticity or dynamic deformation of the sample when the probe of the atomic force microscope scans the surface of the fluorescence-labeled sample. For example, if the lateral force signal fluctuates significantly (for example, ±10 nanonewtons (nN)), it indicates that the fluorescence-labeled sample is in an active state. For another example, if the lateral force signal changes gently (for example, ±1 nN), it indicates that the fluorescence-labeled sample is in a stable state.

[0109] Exemplarily, the structured illumination microscope detects the brightness change amount of the fluorescence-labeled sample every second preset period, and then sends the signal carrying the brightness change amount to the control device.

[0110] For example, if the probe lateral force detected by the atomic force microscope fluctuates periodically greatly (for example, ±15 nN), it indicates that the fluorescence-labeled sample is severely deformed at this time; if the structured illumination microscope detects that the fluorescence intensity fluctuates rapidly, the control device can determine that the fluorescence-labeled sample is in an active state and the fluorescence molecules change violently at this time, and then the illumination intensity of the structured illumination microscope can be reduced.

[0111] For another example, if the periodic fluctuation of the probe lateral force detected by the atomic force microscope is small (e.g., ±0.5 nN), it indicates that the fluorescently labeled sample is in a stable state at this time; if the structure illumination microscope detects a small change in fluorescence intensity, the control device can determine that the fluorescently labeled sample is in a stable state at this time, and the change in fluorescent molecules is small, and then the illumination intensity of the structure illumination microscope can be increased.

[0112] In the embodiments of the present application, the state of the sample is dynamically judged through the lateral force signal of the atomic force microscope and the brightness signal of the structure illumination microscope; then based on the state of the sample, the illumination intensity of the structure illumination microscope is adjusted to reduce the phototoxicity.

[0113] In a possible embodiment of the present application, the above S202 includes: if the current state is an active state and the change amount of brightness is greater than or equal to the first change threshold, it is determined that the illumination intensity adjustment amount is less than zero; if the current state is a stable state and the absolute value of the change amount of brightness is less than or equal to the second change threshold, it is determined that the illumination intensity adjustment amount is greater than zero.

[0114] Wherein, the second change threshold is less than the first change threshold.

[0115] Exemplarily, the first change threshold and the second change threshold can be values pre-configured by the control device or values set manually, and the embodiments of the present application do not make specific limitations thereto. For example, the first change threshold is 20% and the second change threshold is 5%.

[0116] Exemplarily, if the illumination intensity adjustment amount is less than zero, it indicates that the illumination intensity of the structure illumination microscope needs to be reduced at this time; if the illumination intensity adjustment amount is greater than zero, it indicates that the illumination intensity of the structure illumination microscope needs to be increased at this time.

[0117] As an example, if the current state is an active state and the change amount of brightness is greater than or equal to the first change threshold, there is a positive correlation between the change amount of brightness and the absolute value of the illumination intensity adjustment amount.

[0118] As another example, if the current state is a stable state and the absolute value of the change amount of brightness is less than or equal to the second change threshold, there is a negative correlation between the change amount of brightness and the absolute value of the illumination intensity adjustment amount.

[0119] As an example, if the current state is an active state and the change amount of brightness is greater than or equal to the first change threshold, it indicates that the fluorescently labeled sample is in an active state at this time (e.g., the frequency of the event signal is 120 Hz, and / or, the fluctuation of the lateral force signal of the atomic force microscope is ±12 nN), and the fluorescent molecules in the fluorescently labeled sample fluctuate violently. At this time, the illumination intensity of the structure illumination microscope needs to be reduced to reduce the impact of the illumination intensity of the structure illumination microscope on the sample phototoxicity.

[0120] For example, if the detected frequency of the event signal is 120 Hz and the brightness change amount is 30%, it can be shown that at this time, during cell division, the fluorescence fluctuates violently due to microtubule reorganization. The fluorescence intensity in the cell division area jumps from 100 photon units to 130 photon units within 0.1 second. Furthermore, it can be determined that the fluorescently labeled sample is in an active state at this time. To avoid high-intensity light accelerating photobleaching and cell damage, the light intensity adjustment amount can be determined to be -150 mW / cm², so that the light intensity of the structured illumination microscope decreases from 500 mW / cm² (initial light intensity) to 350 mW / cm² (target light intensity).

[0121] As another example, if the current state is a stable state and the brightness change amount is less than or equal to the second change threshold, it indicates that the fluorescently labeled sample is in a stable state at this time (for example, the frequency of the event signal is 15 Hz, and / or the fluctuation of the lateral force signal of the atomic force microscope is ±0.3 nN). The fluorescent molecules in the fluorescently labeled sample are stable. At this time, it is necessary to increase the light intensity of the structured illumination microscope to improve the microscopic imaging quality.

[0122] For another example, if the detected frequency of the event signal is 15 Hz and the brightness change amount is 3%, it can be shown that after the cell completes division and enters the metabolic steady state, the fluorescence intensity slightly fluctuates from 200 photon units to 194 photon units within 1 second. Furthermore, it can be determined that the fluorescently labeled sample is in a steady state at this time. To improve the imaging resolution, the light intensity adjustment amount can be determined to be 100 mW / cm², so that the light intensity of the structured illumination microscope increases from 300 mW / cm² to 400 mW / cm² In the embodiments of the present application, by combining the current state of the fluorescently labeled sample and the brightness change amount, the light intensity adjustment amount is comprehensively determined, so that the fluorescently labeled sample can adapt to the current adjusted target light intensity, thereby reducing the impact of the light intensity of the structured illumination microscope on the sample caused by phototoxicity.

[0123] In a possible embodiment of the present application, the control device is further connected to an atomic force microscope. The method provided by the embodiments of the present application further includes: acquiring a second microscopic image of the fluorescently labeled sample collected by the atomic force microscope; the above S204 includes: obtaining a target microscopic image based on the first microscopic image, the second microscopic image, and the time stamp of the event signal.

[0124] Exemplarily, the second microscopic image can be an image of the nano-scale topography or mechanical properties of the sample surface independently collected by the atomic force microscope, that is, the second microscopic image is used to reflect the surface topography of the fluorescently labeled sample.

[0125] Exemplarily, the control device may periodically obtain a second microscopic image from an atomic force microscope. For example, the atomic force microscope completes a scan of the fluorescently labeled sample every 30 ms and records the contraction deformation of the cell membrane.

[0126] Exemplarily, the target microscopic image may be a fused microscopic image of a three-dimensional structure. The target microscopic image can simultaneously reflect the morphology and internal structure of the fluorescently labeled sample, and can dynamically display the change process of the fluorescently labeled sample.

[0127] Exemplarily, since the first microscopic image acquired by the structured illumination microscope is an image of the fluorescent signal inside the fluorescently labeled sample (such as the organelle structure), the second microscopic image generated by scanning with the atomic force microscope is an image of the surface morphology of the fluorescently labeled sample (such as the topological features of the cell membrane), and the precise time stamps of the event signals recorded by the event camera. Therefore, the time stamps of the first microscopic image, the second microscopic image, and the event signals can be integrated to obtain the target microscopic image.

[0128] Regarding how to obtain the target microscopic image based on the time stamps of the first microscopic image, the second microscopic image, and the event signals, reference can be made to the following embodiments, which will not be elaborated here.

[0129] In the embodiments of the present application, by obtaining the second microscopic image in the atomic force microscope and based on the time stamps of the first microscopic image, the second microscopic image, and the event signals, a target microscopic image with higher imaging quality can be obtained, thereby improving the microscopic imaging quality.

[0130] In a possible embodiment of the present application, obtaining the target microscopic image based on the time stamps of the first microscopic image, the second microscopic image, and the event signals includes: synchronizing the time stamps of the event signals with the acquisition times of the first microscopic image and the second microscopic image to obtain the aligned first microscopic image and second microscopic image; performing a fusion process on the aligned first microscopic image and second microscopic image in a three-dimensional space coordinate system to obtain the target microscopic image.

[0131] Exemplarily, based on the time stamps of the event signals, the first microscopic image and the second microscopic image can be synchronized to the same time axis to eliminate the timing deviation caused by device differences. For example, the structured illumination microscope acquires the first microscopic image at time point t = 10.5 ms, the atomic force microscope acquires the second microscopic image at time point t = 10.3 ms; the event signal detects a dynamic change at time point t = 10.4 ms; the control device can align these three data through an interpolation algorithm.

[0132] Exemplarily, a first microscopic image (e.g., X-Y plane) for reflecting the internal structure of a fluorescently labeled sample and a second microscopic image (e.g., Z-axis height) for reflecting the surface features of the fluorescently labeled sample can be fused in a three-dimensional space coordinate system. For example, the position of mitochondria (X = 5μm, Y = 3μm) shown in the first microscopic image collected by structured illumination microscopy after aligning the timestamps of event signals is combined with the depth data of the cell membrane depression (Z = 120nm) shown in the second microscopic image collected by atomic force microscopy to reconstruct the true distribution of organelles in three-dimensional space.

[0133] In an embodiment of the present application, by synchronizing the acquisition times of the first microscopic image and the second microscopic image through the timestamps of event signals and constructing a target microscopic image in a three-dimensional space coordinate system, the microscopic imaging quality may be further improved.

[0134] Figure 3 It is a schematic flowchart of another microscopic imaging method provided by an embodiment of the present application.

[0135] Exemplarily, Figure 3 The shown microscopic imaging method can be executed by an electronic device or by a microscopic imaging device in the electronic device, such as a chip.

[0136] As Figure 3 shown, the microscopic imaging method may include S301 to S312, and S301 to S312 will be described in detail below.

[0137] S301. Obtain event signals in an event camera.

[0138] S302. Determine the current state of the fluorescently labeled sample based on the frequency of the event signals.

[0139] S303. Determine the amount of brightness change of the fluorescently labeled sample based on the intensity of the event signals.

[0140] S304. Determine whether the current state of the fluorescently labeled sample is an active state. If so, execute S305; if not, execute S307.

[0141] S305. Determine whether the amount of brightness change is greater than or equal to a first change threshold. If so, execute S306.

[0142] S306. Determine that the light intensity adjustment amount is less than zero.

[0143] S307. Determine whether the current state of the fluorescently labeled sample is a stationary state. If so, execute S308.

[0144] S308. Determine whether the brightness change amount is less than or equal to the second change threshold. If so, execute S309.

[0145] S309. Determine that the illumination intensity adjustment amount is greater than zero.

[0146] S310. Send the illumination intensity adjustment amount to the structured illumination microscope.

[0147] S311. Obtain the first microscopic image in the structured illumination microscope and the second microscopic image in the atomic force microscope.

[0148] S312. Obtain the target microscopic image based on the first microscopic image, the second microscopic image, and the time stamp of the event signal.

[0149] For the specific descriptions of S301 to S312, reference can be made to FIGS. 1 to Figure 2 , and the relevant descriptions, which will not be elaborated here.

[0150] In the embodiments of the present application, during the microscopic imaging of a sample labeled with fluorescence by a structured illumination microscope, an event signal in an event camera is obtained; based on the event signal, the illumination intensity adjustment amount of the structured illumination microscope is determined; the illumination intensity adjustment amount is sent to the structured illumination microscope so that the structured illumination microscope adjusts the illumination intensity of the structured illumination microscope according to the illumination intensity adjustment amount; since the event camera in this solution can monitor the state of the sample labeled with fluorescence, and the event signal is used to indicate that the brightness change amount of the sample labeled with fluorescence is greater than a preset value, the illumination intensity adjustment amount of the structured illumination microscope can be determined through the event signal sent by the event camera, and the illumination intensity adjustment amount is sent to the structured illumination microscope; since the illumination intensity adjustment amount is associated with the brightness change amount, the illumination intensity of the structured illumination microscope can be matched with the illumination intensity required for microscopic imaging of the sample, so as to reduce the photo-toxicity caused by the illumination intensity of the structured illumination microscope to the sample, and further improve the activity and imaging quality of the sample.

[0151] It should be understood that the above examples are for helping those skilled in the art to understand the embodiments of the present application, rather than limiting the embodiments of the present application to the specific numerical values or specific scenarios shown. Those skilled in the art can obviously make various equivalent modifications or changes according to the above examples, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0152] As described above in conjunction with Figures 2 to 3 The method for microscopic imaging provided by the embodiments of the present application is described in detail; below, in conjunction with Figure 4 and Figure 5Describe in detail the device embodiments of the present application. It should be understood that the devices in the embodiments of the present application can execute various methods in the foregoing embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the foregoing method embodiments.

[0153] Next, in conjunction with Figure 4 Describe in detail the device for microscopic imaging provided in the embodiments of the present application.

[0154] Figure 4 It is a schematic structural diagram of a device for microscopic imaging provided in an embodiment of the present application.

[0155] Exemplarily, as Figure 4 shown, the device 400 for microscopic imaging includes: An acquisition module 410, configured to acquire the current state and the brightness change amount of the fluorescently labeled sample; A determination module 420, configured to determine the light intensity adjustment amount based on the current state and the brightness change amount; The acquisition module 410 is further configured to acquire a first microscopic image of the fluorescently labeled sample collected by the structured illumination microscope when the light intensity of the structured illumination microscope is adjusted to a target light intensity, and the target light intensity is the light intensity obtained based on the light intensity adjustment amount; An imaging module 430, configured to obtain a target microscopic image of the fluorescently labeled sample based on the first microscopic image.

[0156] As a possible implementation manner, the determination module 420 is specifically configured to: if the current state is an active state and the brightness change amount is greater than or equal to a first change threshold, determine that the light intensity adjustment amount is less than zero; if the current state is a stable state and the absolute value of the brightness change amount is less than or equal to a second change threshold, determine that the light intensity adjustment amount is greater than zero, and the second change threshold is less than the first change threshold.

[0157] As a possible implementation manner, the acquisition module 410 is specifically configured to: acquire the event signal of the event camera; obtain the current state based on the frequency of the event signal; and obtain the brightness change amount based on the intensity of the event signal.

[0158] As a possible implementation manner, the determination module 420 is specifically configured to: if the frequency is greater than or equal to a first frequency threshold, determine that the current state is an active state; if the frequency is less than or equal to a second frequency threshold, determine that the current state is a stable state, and the second frequency threshold is less than the first frequency threshold.

[0159] As a possible implementation, the determining module 420 is further configured to obtain a second microscopic image of the fluorescently labeled sample collected by the atomic force microscope; specifically, the determining module 420 is configured to: obtain a target microscopic image based on the first microscopic image, the second microscopic image, and the time stamps of the event signals.

[0160] As a possible implementation, specifically, the determining module 420 is configured to: synchronize the time stamps of the event signals with the acquisition times of the first microscopic image and the second microscopic image to obtain the aligned first microscopic image and second microscopic image; perform a fusion process on the aligned first microscopic image and second microscopic image in a three-dimensional space coordinate system to obtain a target microscopic image.

[0161] It should be noted that the above-described microscopic imaging device is embodied in the form of functional units. The term "module" here can be implemented in software and / or hardware forms, and no specific limitation is made thereto.

[0162] For example, the "module" can be a software program, a hardware circuit, or a combination of both that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a combined processor, etc.) for executing one or more software or firmware programs, a memory, a merged logic circuit, and / or other suitable components that support the described functions.

[0163] Therefore, the units of the examples described in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0164] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0165] Exemplarily, the electronic device 500 includes: a processor 510, a memory 520, and an executable program code 530.

[0166] Exemplarily, the electronic device 500 includes one or more processors 510, and the one or more processors 510 can support the electronic device 500 to implement the generation method of the electronic device in the method embodiments. The processor 510 can be a general-purpose processor or a dedicated processor. For example, the processor 510 can be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0167] Exemplarily, the processor 510 can be used to control the electronic device 500, execute software programs, and process the data of software programs. The electronic device 500 can also include a communication unit for realizing the input (receiving) and output (sending) of signals.

[0168] Exemplarily, the electronic device 500 may include one or more memories 520, on which there is executable program code 530, and the executable program code 530 can be run by the processor 510 to generate instructions, so that the processor 510 executes the microscopic imaging method described in the above method embodiments according to the instructions.

[0169] Optionally, data can also be stored in the memory 520. Optionally, the processor 510 can also read the data stored in the memory 520. The data can be stored at the same storage address as the executable program code 530, or the data can be stored at a different storage address from the executable program code 530.

[0170] Exemplarily, the processor 510 and the memory 520 can be set separately or integrated together. For example, they can be integrated on a system on chip (SOC) of a terminal device.

[0171] Exemplarily, the memory 520 can be used to store the relevant programs of the generation method of the electronic device provided in the embodiments of the present application, and the processor 510 can be used to call the executable program code 530 stored in the memory 520 when controlling the electronic device 500 to execute the microscopic imaging method of the embodiments of the present application.

[0172] The present application also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the microscopic imaging method in any of the foregoing embodiments are implemented.

[0173] Among them, the computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, digital versatile disks (DVDs), compact disc read-only memories (CD-ROMs), microdrives, and magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), dynamic random access memories (DRAMs), video random access memories (VRAMs), flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0174] The present application also provides a computer program product. When the computer program product runs on a computer, it causes the computer to execute the above-related steps to implement a microscopic imaging method in the above embodiments.

[0175] In addition, the electronic device provided in the embodiments of the present application may specifically be a chip, component, or module. The electronic device may include a processor and a memory connected thereto; among them, the memory is used to store instructions. When the electronic device runs, the processor may call and execute the instructions to cause the chip to execute a microscopic imaging method in the above embodiments.

[0176] Among them, the electronic device, computer-readable storage medium, computer program product, or chip provided in the present application are all used to execute the corresponding microscopic imaging method provided above. Therefore, the beneficial effects that can be achieved by them can refer to the beneficial effects in the corresponding microscopic imaging method provided above, and will not be elaborated here.

[0177] Through the description of the above embodiments, those skilled in the art can understand that for the convenience and simplicity of description, only the above division of each functional module is used as an example. In actual applications, the above functions may be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0178] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical or other forms.

[0179] The above content is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application.

Claims

1. A method for microscopic imaging, characterized in that: Applied to a control device, the control device is connected to a structured illumination microscope, the method comprising: Get the current state and brightness change of the fluorescent labeled sample; Determining a light intensity adjustment amount based on the current state and the brightness change amount; When the illumination intensity of the structured illumination microscope is adjusted to a target illumination intensity, obtaining a first microscopic image of the fluorescent labeled sample acquired by the structured illumination microscope, wherein the target illumination intensity is an illumination intensity obtained based on the illumination intensity adjustment amount; Based on the first microscopic image, a target microscopic image of the fluorescently labeled sample is obtained.

2. The method according to claim 1, characterized in that The step of determining the light intensity adjustment amount based on the current state and the brightness change amount includes: If the current state is an active state, and the brightness change amount is greater than or equal to a first change threshold, determining that the illumination intensity adjustment amount is less than zero; If the current state is a stable state, and the absolute value of the brightness change amount is less than or equal to a second change threshold, it is determined that the illumination intensity adjustment amount is greater than zero, and the second change threshold is less than the first change threshold.

3. The method according to claim 1 or 2, characterized in that: The control device is also connected to the event camera, and the acquisition of the current state and brightness change of the fluorescent labeled sample includes: Acquiring an event signal from the event camera; Based on the frequency of the event signal, obtaining the current state; The brightness change amount is obtained based on the intensity of the event signal.

4. The method according to claim 3, characterized in that The obtaining the current state based on the frequency of the event signal comprises: If the frequency is greater than or equal to a first frequency threshold, determining that the current state is an active state; If the frequency is less than or equal to a second frequency threshold, it is determined that the current state is a stable state, and the second frequency threshold is less than the first frequency threshold.

5. The method according to claim 3, characterized in that: The control device is also connected to the atomic force microscope, and the method further includes: Acquire a second microscopic image of the fluorescent-labeled sample collected by the atomic force microscope; The step of obtaining a target microscopic image of the fluorescently labeled sample based on the first microscopic image comprises: The target microscopic image is obtained based on the first microscopic image, the second microscopic image and the time stamp of the event signal.

6. The method according to claim 5, characterized in that The step of obtaining the target microscopic image based on the first microscopic image, the second microscopic image and the timestamp of the event signal comprises: Synchronizing the acquisition time of the first microscopic image and the second microscopic image based on the timestamp of the event signal to obtain the aligned first microscopic image and the second microscopic image; The aligned first microscopic image and the second microscopic image are fused in a three-dimensional space coordinate system to obtain the target microscopic image.

7. A microscopic imaging device, characterized in that: Applied to a control device, the control device is connected to a structured illumination microscope, and the device comprises: An acquisition module is used to acquire the current state and brightness change of the fluorescent labeled sample; A determination module, configured to determine a light intensity adjustment amount based on the current state and the brightness change amount; The acquisition module is further used to acquire a first microscopic image of the fluorescent-labeled sample acquired by the structured illumination microscope when the illumination intensity of the structured illumination microscope is adjusted to a target illumination intensity, wherein the target illumination intensity is an illumination intensity obtained based on the illumination intensity adjustment amount; An imaging module is used to obtain a target microscopic image of the fluorescently labeled sample based on the first microscopic image.

8. A microscopic imaging system, characterized in that: The microscopic imaging system comprises: a control device and a structured illumination microscope, wherein the control device is connected to the structured illumination microscope, and the control device is used to execute the method according to any one of claims 1 to 6.

9. A control device, characterized in that: The control device comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the control device executes the method according to any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images