Optogenetic control system and its control method

Abstract

The present application provides an optogenetic control system and control method thereof. The system comprises: an image acquisition module for acquiring brightfield images of a live microbial sample at successive moments; a data processing module for preprocessing the brightfield images at successive moments to obtain images containing phenotypic information of target cells; a microprojection module for modulating the preprocessed images and projecting the modulated images onto a microscope module, wherein the imaging of the live microbial sample in the microscope module corresponds to the modulated images; and the data processing module is further configured to generate synchronization signals to control the synchronous operation of the light source module, the image acquisition module, and the microprojection module. This solution combines image tracking with microscopy imaging light stimulation to facilitate precise positioning and tracking of target cells, as well as observation of changes in target cells, thereby achieving continuous optical manipulation of target cells in live microbial samples.

Description

Technical Field

[0001] The present application belongs to the field of optical experimental technology, and in particular relates to an optogenetic control system and a control method thereof. Background Art

[0002] Optogenetic control mainly refers to the introduction of optogenetic circuits into experimental individuals, and through the expression or production of corresponding photoreceptor components, different light exposure is given to the experimental individuals to achieve control of the physiological state or gene expression of the experimental individuals.

[0003] Traditional optogenetic control experiments are often conducted on large experimental individuals in vivo or in vitro, or on tissue samples from large experimental individuals. They are not suitable for experiments on active microorganisms such as bacteria that are motile and grow rapidly.

[0004] For microbial populations such as bacteria, traditional optogenetic control experiments use light to globally control the response of bacteria or cell populations when shaking the bacteria in a test tube or growing as a single clone on an agar plate. However, due to the heterogeneity of bacterial cells, gene expression or bacterial phenotypes vary at the scale of individual cells. Although it is possible to focus light on a microscope sample to achieve optical control of a local sample, such schemes are only applicable to optogenetic control of static samples such as the aforementioned tissue samples, and are not suitable for active microbial samples such as bacteria that are motile and grow rapidly. In addition, during long-term continuous culture, the physiological state and position of bacteria are changing.

[0005] Therefore, how to achieve accurate optogenetic control of active microorganisms such as bacteria is a technical problem that needs to be solved urgently. Summary of the Invention

[0006] The embodiments of the present application provide an optogenetic control system and a control method thereof, which can improve the accuracy of optogenetic control of active microorganisms such as bacteria.

[0007] In a first aspect, an embodiment of the present application provides an optogenetic control system, which includes a light source module, an image acquisition module, a micro-projection module, a microscope module, and a data processing module; wherein,

[0008] The output end of the light source module is connected to the input end of the micro-projection module; the light source module is used to provide light stimulation illumination;

[0009] The input end of the image acquisition module is connected to the output end of the microscope module; the output end of the image acquisition module is connected to the input end of the data processing module; the image acquisition module is used to obtain bright field images of the active microbial sample at consecutive moments;

[0010] The output end of the data processing module is connected to the input end of the micro-projection module; the data processing module is used to: perform image preprocessing on the bright field images at consecutive moments to obtain preprocessed images corresponding to the bright field images, wherein the preprocessed images contain phenotypic information of target cells in the active microbial sample at consecutive moments;

[0011] The output end of the micro-projection module is connected to the input end of the microscope module. The micro-projection module is configured to: receive a pre-processed image from the data processing module; modulate the pre-processed image to obtain a modulated image, wherein the modulated image is carried by a light spot and images a pattern in the light spot; and project the modulated image into the objective lens field of view of the microscope module, wherein the image of the active microbial sample in the microscope module corresponds to the modulated image.

[0012] The data processing module is also used to generate a synchronization signal for controlling the light source module, the image acquisition module and the micro-projection module to work synchronously.

[0013] In one embodiment, the continuous moments include a first moment and a second moment, the second moment being later than the first moment; the phenotypic information includes static information and process information; and the data processing module is specifically configured to:

[0014] receiving a bright field image at a first moment from an image acquisition module, where the bright field image at the first moment is captured by the image acquisition module under the triggering of a synchronization signal at the first moment;

[0015] performing image preprocessing on the bright field image at the first moment to obtain a preprocessed image at the first moment;

[0016] Extracting static information of the target cell at the first moment from the preprocessed image at the first moment;

[0017] Sending the pre-processed image at the first moment to the micro-projection module so that the micro-projection module performs a modulation operation;

[0018] receiving a bright field image at a second moment from the image acquisition module, where the bright field image at the second moment is captured by the image acquisition module under the triggering of the synchronization signal at the second moment;

[0019] performing image preprocessing on the bright field image at the second moment to obtain a preprocessed image at the second moment;

[0020] extracting static information of the target cell at the second moment from the preprocessed image at the second moment;

[0021] The bright field image at the first moment and the bright field image at the second moment are tracked and analyzed to obtain the process information of the target cells.

[0022] In one embodiment, the micro-projection module includes a micro-projection chip, a lens, and a coaxial magnification optical path;

[0023] The input end of the micro-projection chip is connected to the output end of the light source module and the output end of the data processing module; the output end of the micro-projection chip is connected to the input end of the coaxial amplifier circuit; the micro-projection chip is used to modulate the pre-processed image from the data processing module to obtain a modulated image;

[0024] The input end of the lens is connected to the output end of the coaxial amplifying circuit; the output end of the lens is connected to the input end of the microscope module;

[0025] The coaxial amplifying optical path is used to receive the modulated image from the micro-projection chip and project the modulated image into the objective field of view of the microscope module through the lens.

[0026] In one embodiment, the micro-projection chip is a DMD chip; or the micro-projection chip includes a spatial light modulator, a first polarizer, and a second polarizer;

[0027] The first polarizer is set before the spatial light modulator and is the polarizer of the spatial light modulator; the second polarizer is set after the spatial light modulator and is the linear polarization beam splitter cube after the spatial light modulator;

[0028] The spatial light modulator is used to receive the pre-processed image from the data transmission module and perform phase modulation on the pre-processed image to obtain a patterned light spot. The pattern imaging in the light spot is the modulated image.

[0029] In one embodiment, the light source module includes a laser light source and a multimode optical fiber;

[0030] The output end of the laser light source is connected to the input end of the multimode optical fiber; the laser light source is used to generate lasers of multiple wavelengths;

[0031] The output end of the multimode optical fiber is connected to the input end of the micro-projection module; the multimode optical fiber is used to couple lasers of multiple wavelengths to form a laser spot; the laser spot is used to irradiate active microbial samples after beam expansion.

[0032] In one embodiment, the plurality of wavelengths include wavelengths corresponding to red, green, and blue color components, respectively.

[0033] In one embodiment, the synchronization signal is also used to control motor vibration, which in turn causes the coiled multimode optical fiber to vibrate. By controlling the start time of the motor's vibration using the synchronization signal, the multimode optical fiber can be vibrated, thereby balancing the illumination field.

[0034] In one embodiment, the synchronization signal is used to control the light source module and the micro-projection module to work synchronously to illuminate the target cells, and at the same time control the image acquisition module to capture the active microbial sample to obtain a bright field image.

[0035] In one embodiment, the synchronization signal is used to control the image acquisition module, the light source module and the micro-projection module to work synchronously to obtain a wide-field fluorescence image.

[0036] In one embodiment, the static information includes at least one of length, position, size, or fluorescence intensity; and the process information includes at least one of growth and division information or position change information.

[0037] In a second aspect, an embodiment of the present application provides an optogenetic control method, comprising: obtaining multiple bright-field images of an active microbial sample at consecutive moments, wherein the multiple bright-field images are taken when light stimulation is applied to target cells in the active microbial sample; processing the multiple bright-field images to obtain phenotypic information of the target cells at consecutive moments; and optogenetic control of the target cells based on the phenotypic information of the target cells at consecutive moments.

[0038] The control method of the second aspect can be executed using the control system of the first aspect.

[0039] In one embodiment, the phenotypic information includes static information and process information; when processing multiple bright field images to obtain phenotypic information of target cells in an active biological sample at consecutive moments, the information may include:

[0040] performing image preprocessing on the plurality of bright field images to obtain a plurality of preprocessed bright field images, wherein the plurality of preprocessed bright field images contain static information;

[0041] Extracting static information of target cells at consecutive moments from multiple pre-processed bright field images;

[0042] Tracking analysis is performed on multiple pre-processed bright field images to obtain the process information of the target cells at consecutive moments.

[0043] In one embodiment, the consecutive moments include a first moment and a second moment, the second moment being later than the first moment. When optogenetic control of a target cell is performed based on phenotypic information of the target cell at the consecutive moments, the method may include:

[0044] According to the process information of the target cell from the first moment to the second moment, the position of the target cell in the active microbial sample at the second moment is located; and the target cell is optically genetically controlled according to the position.

[0045] In one embodiment, the control method may further include:

[0046] The pre-processed image is modulated by using a micro-projection module to obtain a modulated image, where the modulated image is a pattern image in the light spot;

[0047] The modulated image is projected into the objective field of view in the microscope module, and the imaging of the active microbial sample in the microscope module corresponds to the modulated image, so that at the same moment, the position of the target cell in the preprocessed image is consistent with the position of the target cell in the active microbial sample.

[0048] In one embodiment, when the micro-projection module is used to modulate the pre-processed image to obtain the modulated image, the method may include: performing phase modulation on the pre-processed image to obtain pattern imaging in the light spot.

[0049] In one embodiment, the control method may further include:

[0050] Generate synchronization signals at consecutive moments, the synchronization signals being used to control the light source module, the micro-projection module and the image processing module to work synchronously at consecutive moments;

[0051] When acquiring multiple brightfield images of a live microbial sample at consecutive moments, this can include:

[0052] Under the triggering of the synchronization signal, the light source module and the micro-projection module are controlled to illuminate the active microorganism sample, while the image processing module is controlled to collect the bright field image of the active microorganism sample.

[0053] In one embodiment, the synchronization signal is further used to control the motor to operate synchronously at consecutive moments, and the control method further includes:

[0054] When triggered by a synchronization signal, the motor is controlled to vibrate. When the motor vibrates, the coil of the multimode optical fiber in the light source module is driven to vibrate.

[0055] In one embodiment, the static information includes at least one of length, position, size, or fluorescence intensity; and the process information includes at least one of growth and division information or position change information.

[0056] In a third aspect, an embodiment of the present application provides a control method for an optogenetic control system, wherein the control system is the control system of the first aspect, and the control method includes:

[0057] The image acquisition module acquires bright field images of the active microbial sample at consecutive moments;

[0058] The data processing module processes the bright field images at consecutive moments to obtain phenotypic information of target cells in the active microbial sample at consecutive moments and pre-processed images corresponding to the bright field images;

[0059] The micro-projection module modulates the pre-processed image to obtain a modulated image; the modulated image is projected into the objective lens field of view of the microscope module, and the imaging of the active microbial sample in the microscope module corresponds to the modulated image;

[0060] The data processing module generates a synchronization signal at each moment to control the light source module, the image acquisition module and the micro-projection module to work synchronously.

[0061] In one embodiment, the continuous moments include a first moment and a second moment, the second moment being later than the first moment; the phenotypic information includes static information and process information; and the data processing module processes the bright field images at the continuous moments to obtain the phenotypic information of the target cells in the active microbial sample at the continuous moments and pre-processed images corresponding to the bright field images, including:

[0062] receiving a bright field image at a first moment from an image acquisition module, where the bright field image at the first moment is captured by the image acquisition module under the triggering of a synchronization signal at the first moment;

[0063] performing image processing on the bright field image at the first moment to obtain static information of the target cell at the first moment and obtain a pre-processed image at the first moment;

[0064] Sending the pre-processed image at the first moment to the micro-projection module;

[0065] receiving a bright field image at a second moment from the image acquisition module, where the bright field image at the second moment is captured by the image acquisition module under the triggering of the synchronization signal at the second moment;

[0066] performing image processing on the bright field image at the second moment to obtain static information of the target cell at the second moment and a pre-processed image at the second moment;

[0067] The bright field image at the first moment and the bright field image at the second moment are tracked and analyzed to obtain the process information of the target cells.

[0068] In one embodiment, the synchronization signal is also used to control motor vibration, which in turn causes the coiled multimode optical fiber to vibrate. In this implementation, the synchronization signal can be used to control the start time of the motor's vibration, causing the multimode optical fiber to vibrate, thereby balancing the illumination field.

[0069] In one embodiment, the synchronization signal is specifically used to control the light source module and the micro-projection module to work synchronously to illuminate the target cells, and at the same time control the image acquisition module to capture the active microbial sample to obtain a bright field image.

[0070] In one embodiment, the synchronization signal is further used to control the image acquisition module, the light source module and the projection module to work synchronously to obtain a wide-field fluorescence image.

[0071] In one embodiment, the static information includes at least one of length, position, size, or fluorescence intensity; and the process information includes at least one of growth and division information or position change information.

[0072] In a fourth aspect, an embodiment of the present application provides a terminal device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the second aspect or the third aspect and any one of its implementation methods when executing the computer program.

[0073] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method of the second aspect or the third aspect and any one of its implementation methods.

[0074] Compared with the prior art, the beneficial effects of the embodiments of the present application are as follows: the bright field image of the active microbial sample is modulated mainly by using the micro-projection module, so that the image presented by the objective field of view of the microscope corresponds to the position of the modulated bright field image, thereby facilitating the positioning of the target cells and the extraction of information about the target cells. This solution makes the position of the target cells seen in the microscope and the pixel position of the target cells in the image correspond to the position of the real target cells by matching the image with the microscope imaging. This facilitates the precise positioning and tracking of target cells, as well as the observation of changes in target cells. The image acquisition module captures bright field images at consecutive moments, and the data processing module performs a comprehensive analysis of the bright field images at consecutive moments, that is, tracking, to obtain the phenotypic information of the target cells at consecutive moments. And several modules are controlled to work synchronously through synchronization signals. Therefore, optogenetic control at the single-cell level can be achieved, and optogenetic experiments on active microorganisms with strong motility and rapid growth, such as bacteria, can be accurately controlled.

[0075] That is, this scheme combines image tracking and microscope imaging light stimulation to facilitate the precise positioning and tracking of target cells, as well as the observation of changes in target cells, thereby achieving continuous light manipulation of target cells in active microbial samples.

[0076] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 It is a schematic structural diagram of an optogenetic control system according to an embodiment of the present application.

[0078] Figure 2 It is a schematic structural diagram of another optogenetic control system according to an embodiment of the present application.

[0079] Figure 3 This is a schematic diagram of the synchronization control of the synchronization signal of an embodiment of the present application.

[0080] Figure 4 This is a schematic flow chart of an optogenetic control method according to an embodiment of the present application.

[0081] Figure 5 This is a schematic flow chart of a control method of an optogenetic control system according to an embodiment of the present application.

[0082] Figure 6 This is a flow chart of another control method of an optogenetic control system according to an embodiment of the present application. DETAILED DESCRIPTION

[0083] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0084] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0085] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0086] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0087] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0088] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0089] Figure 1 This is a schematic structural diagram of an optogenetic control system according to an embodiment of the present application. Figure 1 As described above, the system 100 includes a light source module 110 , a micro-projection module 120 , a data processing module 130 , an image acquisition module 140 and a microscope module 150 .

[0090] The output end of the light source module 110 is connected to the input end of the micro-projection module 120. The light source module 110 is used to provide light stimulation illumination. Specifically, the micro-projection module 120 provides light stimulation illumination for the active microbial sample on the operating table of the microscope module 150. It can also be understood that the light source module 110 is used to illuminate the cells in the active microbial sample.

[0091] The input of the image acquisition module 140 is connected to the output of the microscope module 150; the output of the image acquisition module 140 is connected to the input of the data processing module 130. The image acquisition module 140 is used to acquire brightfield images of the active microbial sample. For example, brightfield images can be acquired at different times. In one example, the image acquisition module 140 acquires brightfield images at multiple consecutive times. For example, a brightfield image is acquired at time T1 and time T2, respectively, where time T1 and time T2 are consecutive times.

[0092] The input end of the micro-projection module 120 is also connected to the output end of the data processing module 130 ; the output end of the micro-projection module 120 is connected to the input end of the microscope module 150 .

[0093] The micro-projection module 120 can be used to modulate the bright field image to obtain a modulated bright field image, and transmit the modulated bright field image to the microscope module 150. The modulated bright field image is a patterned light spot. In one example, the micro-projection module 120 modulates the bright field image from the data processing module to obtain a modulated image, i.e., a patterned light spot. The micro-projection module 120 also projects the modulated image into the objective field of view in the microscope module. The bright field image from the data processing module 130 is the image obtained after the data processing module 130 performs image preprocessing on the bright field image uploaded by the image acquisition module 140, and therefore can be referred to as a preprocessed image of the bright field image, or an image after image preprocessing of the bright field image.

[0094] In other words, the data processing module 130 can be used to preprocess the brightfield images from the image acquisition module 110 to obtain preprocessed images. These preprocessed images contain phenotypic information of the target cells. Therefore, the data processing module 130 can further extract this phenotypic information from the preprocessed images.

[0095] Image preprocessing can be done by using a series of image processing algorithms such as image background correction and image recognition to obtain the solid outline information of the cells in the sample at that moment. Based on this outline information, phenotypic information such as the position and size of the cells can be extracted.

[0096] The imaging of the active microbial sample in the microscope module corresponds to the bright field image modulated by the micro-projection module. In other words, the position of a cell in the sample seen through the microscope and the pixel position of that cell in the modulated image both correspond to the actual location of the cell in the sample.

[0097] Target cells can be understood as active cells that one wishes to observe in an active microbial sample, or cells that are continuously optogenetically controlled.

[0098] In one example, data processing module 130 performs image preprocessing on brightfield images at consecutive moments to obtain preprocessed images. These preprocessed images contain phenotypic information of target cells in a live microbial sample at consecutive moments. For example, if a first moment and a second moment are two consecutive moments, and the second moment is later than the first, the brightfield image at the first moment can be processed to obtain static information such as size, position, and fluorescence intensity, which is part of the phenotypic information at the first moment; the brightfield image at the second moment can also be processed to obtain static information such as size, position, and fluorescence intensity, which is part of the phenotypic information at the second moment. Comprehensive analysis of the brightfield images at both moments can also provide process information about the target cells, such as positional changes and growth and division information between the first and second moments.

[0099] Alternatively, phenotypic information may include static information and process information (motion information). Static information may include at least one of length, position, size, or fluorescence intensity. Process information may include information such as growth and division information, position change information, etc. Process information can be understood as dynamic information or motion information generated during the experiment, such as cell growth, cell division, and cell movement (i.e., cell position changes at different times), which are all information that can only be inferred over time.

[0100] In this example, the data processing module 130 may further extract phenotypic information from the pre-processed image, for example, static information at each moment, and process information between consecutive moments.

[0101] The data processing module 130 is also used to generate a synchronization signal that controls the synchronization of the light source module, the micro-projection module, and the image acquisition module. For example, while the light source module and the micro-projection module illuminate the target cells, the image acquisition module simultaneously captures a brightfield image or a widefield fluorescence image.

[0102] Optionally, the synchronization signal can also be used to control the synchronous operation of the motor, so that the motor vibration drives the multimode optical fiber in the light source module 110 to vibrate, so that the illumination field is balanced. This part will be described in detail below and will not be repeated here for the sake of brevity.

[0103] The microscope module 150 is used to observe the biological characteristics of active microbial samples, and can be understood as being used by the experimenter to perform microscopic observation. The microscope module 150 can be regarded as a microscope optical system / device, that is, including a microscope and its accessories.

[0104] In the embodiments of this application, active microbial samples refer to samples of living microorganisms. In other words, they are motile and rapidly changing microorganisms. They can also be referred to as active microbial samples. Active microbial samples can include bacteria, single-celled organisms, and other living organisms that can be observed under a microscope. This is fundamentally different from the animals or animal tissues used in traditional protocols.

[0105] Figure 1The system shown mainly uses a micro-projection module to modulate the bright field image of the active microbial sample, so that the image presented by the microscope's objective field of view corresponds to the position of the modulated bright field image, thereby facilitating the positioning of the target cells and the extraction of target cell information. This solution aligns the image with the microscope imaging so that the position of the target cells seen in the microscope and the pixel position of the target cells in the image correspond to the actual position of the target cells. This makes it easy to accurately locate and track the target cells, as well as observe changes in the target cells. Therefore, single-cell level optogenetic control can be achieved, which can accurately control optogenetic experiments on active microorganisms with strong motility and rapid growth, such as bacteria.

[0106] Figure 2 It is a schematic structural diagram of another optogenetic control system according to an embodiment of the present application. Figure 2 Can be seen as Figure 1 A specific example of the system shown.

[0107] Figure 2 The dotted lines in the figure represent optical transmission channels, that is, the transmission routes of optical information between different modules or components. Figure 2 The solid lines with arrows in the figure represent data transmission channels and their directions, primarily including image data, synchronization signals, and device parameters. For example, between camera 141 and host computer 131, camera 141 can transmit captured images to host computer 131, host computer 131 can set camera 141 parameters, and can also send synchronization signals to camera 141 to control its operating state. For another example, between host computer 131 and laser light source 111, host computer 131 can send synchronization signals to laser light source 111 to control its operating state. For the sake of brevity, these examples are not listed here.

[0108] In one implementation, Figure 2 As shown, light source module 110 includes a laser light source 111 and a multimode optical fiber 112. The output end of laser light source 111 is connected to the input end of multimode optical fiber 112; laser light source 111 is used to generate laser light of multiple wavelengths; the output end of multimode optical fiber 112 is connected to the input end of micro-projection module 120; multimode optical fiber 112 is used to couple and expand the laser light of multiple wavelengths to form a laser spot; the laser spot is used to irradiate the active microbial sample.

[0109] In one example, the light source module 110 may further include a beam expander 113. The output end of the multimode optical fiber 112 is connected to the input end of the beam expander 113, which is in turn connected to the input end of the micro-projection module 120. After being expanded by the beam expander 113, the laser beam output from the multimode optical fiber 112 forms a laser spot of a certain area.

[0110] In one example, the plurality of wavelengths include wavelengths corresponding to the three color components of red (R), green (G), and blue (B). Alternatively, lasers with wavelengths of 488 nm, 561 nm, and 640 nm, or 445 nm, 515 nm, and 640 nm can be used as the illumination wavelengths corresponding to the three primary colors of RGB.

[0111] In one implementation, Figure 2 As shown, micro-projection module 120 includes a micro-projection chip 121, a lens 122, and a coaxial amplification optical circuit 123. The input of micro-projection chip 121, which also serves as the input of micro-projection module 120, is connected to the output of light source module 110 and the output of data processing module 130. The output of micro-projection chip 121 is connected to the input of coaxial amplification circuit 123. The output of coaxial amplification optical circuit 123 is connected to the input of lens 122; the output of lens 122 is connected to the input of microscope module 150.

[0112] It should be understood that the lens is an accessory that can be used without distinguishing between the two ends, so the input end and the output end are only relative and can be interchanged.

[0113] The micro-projection chip 121 is used to modulate the brightfield image from the data processing module 130. In one example, the micro-projection chip 121 modulates the received pre-processed image to obtain a modulated image; the coaxial amplification optical path 123 is used to project the modulated image into the objective field of view of the microscope module 150.

[0114] In one implementation, the micro-projection chip 121 may be a digital micromirror device (DMD) chip.

[0115] In another implementation, Figure 2 As shown, the micro-projection chip 121 includes a first polarizer 121 - 1 , a spatial light modulator 121 - 2 and a second polarizer 121 - 3 .

[0116] The first polarizer 121 - 1 is disposed before the spatial light modulator 121 - 2 and serves as a polarizer of the spatial light modulator 121 - 2. The function of the polarizer is to ensure that the light entering the spatial light modulator 121 - 2 is in a single polarization direction.

[0117] In one example, the first polarizer 121 - 1 is a polarization beam splitter (PBS) prism.

[0118] In one example, the first polarizer 121 - 1 is configured to receive a laser spot from the light source module 110 and transmit the laser spot to the spatial light modulator 121 - 2 .

[0119] The spatial light modulator 121-2 is used to receive a brightfield image from the data processing module 130 and perform phase modulation on the brightfield image to produce a patterned light spot. The pattern image in the patterned light spot is the modulated brightfield image. In other words, the modulated image is carried by the light spot, or in other words, the modulated image is projected using the light spot as a carrier, and the pattern in the light spot is the modulated image. In one example, the spatial light modulator 121-2 receives a preprocessed image corresponding to the brightfield image from the data processing module 130 and performs phase modulation on the preprocessed image to produce the patterned light spot.

[0120] Optionally, the phase modulation may be a phase modulation of 0 or 180 degrees.

[0121] The light entering the spatial light modulator 121-2 must be in a single polarization direction, which affects the contrast of the projected pattern and is a critical step. Therefore, a polarizer and a spatial light modulator need to be used in combination.

[0122] The micro-projection module 120 is implemented using an RGB color laser projection method. Laser projection of an RGB color image actually uses lasers with three wavelengths, R, G, and B, to project the three color components of the image. The LED signal output by the spatial light modulator 121-2 contains the three color components, R, G, and B, which are then ANDed with the signal controlling the laser light source. To accommodate wide-field fluorescence illumination requirements, lasers with wavelengths of 488 nanometers (nm), 561 nm, and 640 nm, or 445 nm, 515 nm, and 640 nm, are used as the illumination wavelengths for the three primary RGB colors. The R component of the LED is ANDed with the control signal (laser 640 nm), the G component of the LED is ANDed with the control signal (laser 561 nm), and the B component of the LED is ANDed with the control signal (laser 488 nm). Another combination of light can similarly achieve RGB color image projection.

[0123] Optogenetic control microscopes can achieve the same functions as traditional fluorescence microscopes. The pattern of all-white pixels on the spatial light modulator can be used as wide-field fluorescence illumination. Unlike the fluorescence illumination composed of standard multimode optical fibers, due to the working principle of the spatial light modulator, the illumination provided by this method will have a 50% light loss. In the embodiment of the present application, the jitter of the multimode optical fiber can be controlled to make the illumination light field relatively uniform, avoiding the uneven Gaussian beam illumination. In other words, by coiling the multimode optical fiber and using the vibration of the motor to drive the jitter of the multimode optical fiber coil, the illumination field is balanced.

[0124] Optionally, the above synchronization signal can be used to synchronously control the start-up of the motor.

[0125] The following uses a specific example to illustrate the coordinated operation of the light source module 110, the micro-projection module 120, and the microscope module 150. A light source composed of multiple lasers (i.e., lasers of multiple wavelengths provided by the laser light source 111) enters a square-core multimode fiber 112 via a coupled optical path. The optical path passes through the objective lens and a telescope, and the square core of the fiber is imaged at the focal length of the objective lens of the microscope module 150, i.e., the location where the modulated light spot of the spatial light modulator 121-2 is projected. The exiting light spot is slightly larger than the chip size of the spatial light modulator 121-2 to ensure that the light spot is not too large, which would increase optical power loss, but not too small, which would affect optical path adjustment. Optionally, the multimode fiber core size can be 0.2 x 0.2 mm, with an NA of 0.14. Optionally, the objective lens can be a 4x objective lens with an NA of 0.14. Optionally, the telescope can be a 10x telescope. Optionally, the exiting light spot can be approximately 3 mm larger than the chip size of the spatial light modulator 121-2. It should be understood that those skilled in the art may also select other specific values as needed, and there is no numerical limitation. In addition, the core shape of the multimode optical fiber may be the above-mentioned square core or may not be a square core, such as a round core, and there is no limitation.

[0126] The second polarizer 121 - 3 is disposed after the spatial light modulator 121 - 2 and the linear polarization beam splitter cube after the spatial light modulator 121 - 2 ; it is used to transmit the modulated bright field image to the coaxial amplification optical path 123 .

[0127] In one implementation, the second polarizer 121 - 3 may be a PBS prism.

[0128] In one example, the extinction ratio of the PBS prism is 1000:1. Spatial light modulator 121-2 may also utilize other suitable prisms as polarizers. A higher extinction ratio results in higher projection contrast. The pattern modulated by spatial light modulator 121-2 passes through coaxial amplification optical path 123 and enters the back focal plane of the high-power objective lens in microscope module 150, ultimately projecting into the microscope's field of view. It should be understood that those skilled in the art may select other extinction ratios as desired, and there is no numerical limitation.

[0129] like Figure 2 As shown, the data processing module 130 includes a host computer 131. In the embodiment of the present application, the data processing module 130 can utilize any device / equipment capable of performing image processing and generating control signals, such as a computer, a PC, a host computer, a server, an intelligent terminal or a cloud device. Figure 2 The host computer is used as an example for introduction.

[0130] In one implementation, the continuous moments include a first moment and a second moment, the second moment being later than the first moment; the phenotypic information includes static information and process information; and the data processing module 130, when processing the bright field images at the continuous moments, includes the following operations:

[0131] receiving a bright field image at a first moment from an image acquisition module, where the bright field image at the first moment is captured by the image acquisition module under the triggering of a synchronization signal at the first moment;

[0132] performing image processing on the bright field image at the first moment to obtain static information of the target cell at the first moment and obtain a pre-processed image at the first moment;

[0133] Sending the pre-processed image at the first moment to the micro-projection module;

[0134] receiving a bright field image at a second moment from the image acquisition module, where the bright field image at the second moment is captured by the image acquisition module under the triggering of the synchronization signal at the second moment;

[0135] performing image processing on the bright field image at the second moment to obtain static information of the target cell at the second moment and a pre-processed image at the second moment;

[0136] The bright field image at the first moment and the bright field image at the second moment are tracked and analyzed to obtain the process information of the target cells.

[0137] In one example, the host computer 131 performs a series of image processing algorithms such as image background correction and image recognition on the bright field image from the image acquisition module 140. The resulting image includes the solid outline information of the cells, which is stored in the memory of the host computer 131. The obtained solid outline information of the cells is binarized and converted into a binary image, that is, the solid outline area of the illuminated target is 1, and the other dark areas are 0. The binary image is then segmented to obtain the cell mask at this moment, which is a 1-bit image of 2048x2048 pixels. The cell mask is converted into a binary image suitable for the size of the micro-projection chip 121 (1536×2048) after passing through the conversion matrix and is modulated by the micro-projection chip 121 and output to the field of view of the microscope module 150. The micro-projection chip 121 is controlled via a USB or HDMI interface, and the image transmission can be output through Windows API function programming, analogously implementing the transmission control of an 8-bit grayscale image.

[0138] In one example, the host computer 131 is used to generate a synchronization signal, which is specifically used to control the light source module 110 to synchronously illuminate the target cells and control the image acquisition module 140 to capture the active microbial sample to obtain a bright field image.

[0139] In another example, the synchronization signal is used to control the image acquisition module 140 , the light source module 110 and the micro-projection module 120 to work synchronously to obtain a wide-field fluorescence image.

[0140] In another example, the synchronization signal is used to control motor vibration, which in turn causes the coiled multimode fiber to vibrate. By controlling the start and end timing of the motor's vibrations and causing the multimode fiber to vibrate, the synchronization signal can even out the illumination field.

[0141] In another example, the synchronization of the system is controlled by logic signals, which mainly control the synchronization of the camera 141 (an example of the image acquisition module 140), the laser light source 111 and the spatial light modulator 121-2 (an example of the micro-projection chip 121). Figure 3 As shown, the control circuit outputs logic signals for both the 8-bit grayscale image and the 1-bit black-and-white image of the spatial light modulator 121-2. This logic signal serves as an interrupt to output control logic signals for the camera 141 and the laser light source 111 for system synchronization. Camera 141 is triggered by the first rising edge of the spatial light modulator 121-2. When the spatial light modulator 121-2 is high, the laser light source 111 is high; when the spatial light modulator 121-2 is low, the laser light source 111 is low. The specific process is as follows: the host computer 131 sends a capture command to the camera 141, which is triggered by the synchronous logic signal. The laser light source 111 and the spatial light modulator 121-2 are simultaneously high, and the camera 141 receives a rising edge signal and begins operation. After the laser light source 111 and the camera 141 complete their operations, they return to a low state. Figure 3 The motor 114 is used to shake the multimode optical fiber 112 to make the light uniform. The motor 114 can be independent of the light source module 110. Figure 3 The other illumination light sources 115 shown may include, for example, a light source that provides illumination for the microscope module 150. Figure 3 As shown, the host computer 131 controls the camera 141, the motor 114, the spatial light modulator 121-2, the laser light source 111 and the other illumination light sources 115 through the synchronous control center. The synchronous control center can be a control module integrated in the host computer 131, or it can be independent of the host computer 131 in one implementation, such as Figure 2 The image acquisition module 140 is shown as a camera 141. However, it should be understood that the image acquisition module 140 may also be a camera, a video recorder, or an image scanning device, etc., which can capture a sample and obtain an image of the sample.

[0142] Figure 4 This is a schematic flow chart of an optogenetic control method according to an embodiment of the present application. Figure 4The control method can be implemented by using each module in the above-mentioned optogenetic control system. Figure 4 Each step is introduced.

[0143] S401. Acquire multiple bright field images of an active microbial sample at consecutive moments.

[0144] The above-mentioned multiple bright field images are captured when the target cells in the active microbial sample are photostimulated.

[0145] Target cells can be selected at the initial stage or at a certain point during optogenetic control, without any limitation.

[0146] Step S401 may be performed by the image acquisition module 140 , and the light stimulation of the target cells may be performed by the light source module 110 .

[0147] In one implementation, the control method may further include generating a synchronization signal at consecutive moments, the synchronization signal being used to control the light source module, the micro-projection module, and the image processing module to operate synchronously at consecutive moments. In this implementation, step S401 may include, under the triggering of the synchronization signal, controlling the light source module and the micro-projection module to illuminate the active microbial sample while controlling the image processing module to capture a brightfield image of the active microbial sample.

[0148] In this implementation, the synchronization of photography, light stimulation and light modulation is mainly achieved through the synchronous control of the synchronization signal, that is, the synchronous operation of each module in the system.

[0149] In one example, the synchronization signal is also used to control the synchronous operation of the motor at continuous moments. The above control method also includes: controlling the vibration of the motor under the triggering of the synchronization signal, and when the motor vibrates, the winding of the multimode optical fiber in the light source module is driven to shake.

[0150] In this example, during the synchronous operation, the motor vibrates synchronously, which can make the light provided by the light source module 110 better during the synchronous operation.

[0151] S402: Process the multiple bright field images to obtain phenotypic information of the target cells at consecutive moments.

[0152] Step S402 may be executed by the aforementioned data processing module 130 .

[0153] As mentioned above, phenotypic information may include static information and process information. The relevant content can be referred to the above description and will not be repeated here.

[0154] In one implementation, S402 may include:

[0155] S402-1. Perform image preprocessing on a plurality of bright field images to obtain a plurality of preprocessed bright field images, wherein the plurality of preprocessed bright field images contain static information.

[0156] S402-2. Extract static information of the target cells at consecutive moments from the plurality of pre-processed bright field images.

[0157] S402-3. Perform tracking analysis on the multiple pre-processed bright field images to obtain process information of the target cells at consecutive moments.

[0158] S403. Optogenetically control the target cells according to the phenotypic information of the target cells at successive moments.

[0159] Step S403 may be executed by the data processing module 130 .

[0160] Tracking a target cell can be understood as determining the cell's positional changes at each moment. Assuming a continuous sequence of moments includes a first moment and a second moment, with the second moment later than the first, the target cell's position within the active microbial sample at the second moment can be determined based on the process information from the first moment to the second moment. Once this position is determined, the target cell can be located based on this position and optogenetic control can be performed on it, i.e., by continuing to photostimulate the target cell and repeating the steps described above for each of the consecutive moments.

[0161] In one implementation, the control method may further include: modulating the preprocessed image using a micro-projection module to obtain a modulated image, wherein the modulated image is an imaging of a pattern in the light spot; projecting the modulated image into the objective field of view of the microscope module, wherein the imaging of the active microbial sample in the microscope module corresponds to the modulated image, so that at the same moment, the position of the target cell in the preprocessed image is consistent with the position of the target cell in the active microbial sample.

[0162] In this implementation, the micro-projection module 120 modulates the target cell's position in the pre-processed image at the same moment in time, ensuring that the target cell's position in the live microbial sample is consistent. Therefore, in step S403, the target cell's position change can be inferred based on the process information at successive moments, thereby finding the target cell's current position in the image. This position corresponds to the target cell's current position in the sample. When the target cell in the sample is optically stimulated at the next moment in time, the target cell's new position can be found.

[0163] In one implementation, when the micro-projection module 120 is used to modulate the pre-processed image, the process may include performing phase modulation on the pre-processed image to obtain a pattern image in the light spot. For example, 0 or 180 degree phase modulation may be performed.

[0164] Figure 5 This is a schematic flow chart of a control method of an optogenetic control system according to an embodiment of the present application. Figure 5 The optogenetic control system can be any of the above-mentioned optogenetic control systems. Figure 5 Each step is introduced. Figure 5 Where T1 and T2 represent two consecutive moments, and T2 is the next moment of T1.

[0165] S501 . Under the triggering of a synchronization signal, the image acquisition module acquires a bright field image of the active microorganism sample at time T1.

[0166] Step S501 may be performed by the image acquisition module 110 .

[0167] Step S501 can be regarded as an example of acquiring bright field images at consecutive moments under the triggering of the synchronization signal in step S401.

[0168] S502 , the data processing module performs image preprocessing on the bright field image at time T1 to obtain a preprocessed image at time T1 and obtain phenotypic information of target cells in the active microbial sample at time T1.

[0169] The preprocessed image at time T1 is an image obtained after performing the above-mentioned image preprocessing on the bright field image at time T1.

[0170] The phenotypic information of target cells in a live microbial sample at time T1 primarily includes static information such as length, location, size, and fluorescence intensity. One or more target cells can be identified from the sample based on the clarity of the cells in the preprocessed image and the aforementioned phenotypic information.

[0171] Step S502 may be performed by the data processing module 130 .

[0172] Step S502 can be regarded as an example of step S402.

[0173] S503. The micro-projection module receives and modulates the preprocessed bright field image at time T1 to obtain a modulated image at time T1, and projects the modulated image at time T1 into the objective field of view of the microscope module. The imaging of the active microbial sample in the microscope module corresponds to the modulated image at time T1.

[0174] Step S503 may be performed by the pico-projection module 120 .

[0175] Step S503 can be regarded as Figure 4The method shown is an example of the operation of using a micro-projection module to modulate a pre-processed image and projecting the modulated image into the field of view of an objective lens in a microscope module.

[0176] At time T1 S504 , a synchronization signal is generated to control the light source module, the micro-projection module and the camera to work synchronously at the next time.

[0177] Step S504 may be performed by the data processing module 130 .

[0178] Step S504 can be regarded as Figure 4 In the method shown, a synchronization signal is generated at successive moments, and the synchronization signal is used to control the light source module, the micro-projection module, and the image processing module to operate synchronously at successive moments.

[0179] S505 , when the light source module and the micro-projection module synchronously illuminate the target cells, the image acquisition module synchronously acquires the bright field image at time T2.

[0180] It should be noted that it is the target cells that are illuminated during this step, not the entire sample.

[0181] Step S505 may be performed by the image acquisition module 110 .

[0182] Step S505 can be regarded as an example of acquiring bright field images at consecutive moments under the triggering of the synchronization signal in step S401.

[0183] S506. The data processing module modulates the bright field image at time T2 to obtain a preprocessed image at time T2, and obtains the phenotypic information of the target cells at time T2, as well as the process information of the target cells from time T1 to time T2.

[0184] The pre-processed image at time T2 is an image obtained after performing the above-mentioned image processing on the bright field image at time T2.

[0185] The phenotypic information of the target cells at time T2 mainly includes static information such as length, position, size, and fluorescence intensity.

[0186] By comprehensively analyzing the brightfield images at time T1 and time T2, we can obtain information about the target cell's progress from time T1 to time T2. This information can include growth, division, or positional changes. For example, a target cell at time T1 may have become two at time T2. Another example is a target cell at time T1 may have grown larger at time T2. Another example is a target cell at time T1 may have changed position at time T2, and so on.

[0187] Step S506 may be performed by the data processing module 130 .

[0188] Step S506 can be regarded as an example of step S402, specifically an example of steps S402-1 to S402-3.

[0189] Once the above process information is obtained, the operation of step S403 can be performed to determine the position of the target cell at time T2, so as to facilitate optogenetic control at the next moment.

[0190] It should also be understood that the target cells at time T2 can be tracked through the images at time T1 and the images at time T2, while the target cells at time T1 can be tracked through the images at the moment before time T1 and the images at time T1, and so on, which will not be repeated here.

[0191] S507. The micro-projection module receives and modulates the preprocessed bright field image at time T2 to obtain a modulated image at time T2, and projects the modulated image at time T2 into the objective field of view of the microscope module. The imaging of the active microbial sample in the microscope module corresponds to the modulated image at time T2.

[0192] Step S507 may be performed by the pico-projection module 120 .

[0193] Step S507 can be regarded as Figure 4 The method shown is an example of the operation of using a micro-projection module to modulate a pre-processed image and projecting the modulated image into the field of view of an objective lens in a microscope module.

[0194] S508 : Generate a synchronization signal for controlling the light source module, the micro-projection module, and the camera to work synchronously at the next moment.

[0195] At time T2, step S508 may be executed by the data processing module 130.

[0196] Step S508 can be regarded as Figure 4 In the method shown, a synchronization signal is generated at successive moments, and the synchronization signal is used to control the light source module, the micro-projection module, and the image processing module to operate synchronously at successive moments.

[0197] As can be seen, steps S501-S504 and steps S505-S508 correspond to the same operational flow at time T1 and time T2, respectively. Steps S501-S504 acquire an image, modulate, and identify the target cell at time T1, generating a synchronization signal in preparation for the next moment. Steps S506-S508 acquire an image of the target cell at time T2, modulate, and identify the target cell again, generating a synchronization signal in preparation for the next moment. This achieves continuous optogenetic control of the target cell.

[0198] The control method of the embodiment of the present application can achieve continuous optogenetic control at the single-cell level, mainly by modulating the bright field image so that the imaging in the microscope and the captured image correspond to the actual cell position, making it convenient to find the target cells and perform targeted optogenetic control on the target cells.

[0199] exist Figures 4 to 6 In one implementation of the control method shown, the bright field image captured by the microscope is stored in the computer memory in advance. n After the image of the current moment is collected and processed in real time, it is compared with the previous t n-1 The images at each moment are jointly processed, and the image connected region algorithm is used to track the target bacteria or cells in multiple time dimensions in space. After the joint image processing, the computer automatically obtains the phenotypic information of the target bacteria or cells at the current moment. Phenotypic information can include static information of the target bacteria or cells: such as length, position, size, fluorescence intensity, etc.; it can also include process information of the target bacteria or cells, such as growth and division information, position change information, etc. The experimenter performs real-time screening of phenotypic information according to actual experimental needs, and analyzes the screened information in real time on the computer. The experimenter determines the next experimental process based on the real-time screened information. This method has high requirements on the performance of the computer. The image processing capability of the computer determines the time interval for the microscope to collect images and the analysis speed of the screened information. This implementation method can be regarded as an example of the process of image processing by the data processing module. t n The moment corresponds to the second moment, t n-1 The moment corresponds to the first moment.

[0200] Figure 6 It is a schematic flow chart of another control method of the optogenetic control system according to an embodiment of the present application. Figure 6 The control method shown can be seen as Figure 5 A specific example of the control method shown below. Figure 6 Each step is introduced.

[0201] S601. Computer controls the microscope to complete bright field image capture.

[0202] The computer in step S601 can be considered a specific example of data processing module 130. The microscope is a component of microscope module 150. The microscope is integrated with image acquisition module 110, which is camera 111. Therefore, step S601 is an example of data processing module 140 controlling image acquisition module 110 to acquire a brightfield image of a sample. Step S601 can be considered an example of step S501.

[0203] In one implementation, S601 may include setting parameters for the camera and LED brightfield light source via a computer, thereby preparing the camera for shooting. After the preparation is complete, the computer sends an instruction to generate a synchronization control signal. The synchronization control signal is in the form of a pulse sequence, which controls the LED brightfield light source and the camera to synchronize shooting. Alternatively, the synchronization can be achieved by triggering the generation of the synchronization control signal and outputting a pulse sequence using an external pulse signal.

[0204] The LED bright field light source may be considered as an example of a component of the light source module 110 .

[0205] S602 , performing real-time image processing on the bright field image captured in step S601 and buffering the image in a memory.

[0206] In one implementation, S602 may include: returning the image captured at that moment to a computer processor, and undergoing a series of image processing algorithms such as image background correction and image recognition to obtain solid outline information of the cell at that moment, and this information is stored in a computer memory.

[0207] S603. Identify and select target cells (generate cell mask).

[0208] The solid cell outline information obtained in step S602 is binarized and converted into a binary image, where the illuminated solid outline area is set to 1 and the remaining dark areas are set to 0. The binarized image is then segmented to obtain the cell mask at this moment. The image is now a 1-bit image of 2048x2048 pixels.

[0209] S604, mask conversion and transmission.

[0210] The cell mask obtained in step S603 is converted into a binary image suitable for the spatial light modulator chip size (1536×2048) through a conversion matrix. This image is then modulated by the spatial light modulator and output to the microscope's field of view. The spatial light modulator is controlled via a USB or HDMI interface, and image transmission can be programmed using Windows API functions, similar to the transmission control of 8-bit grayscale images.

[0211] S602-S604 can be regarded as a specific example of step S502.

[0212] S605: Synchronous control.

[0213] Specifically, the camera, laser light source, and spatial light modulator are synchronized. A computer generates a synchronization control signal, which synchronizes the camera, laser light source, and spatial light modulator. The laser light source synchronously illuminates the field of view of the microscope optical path, specifically the target cell at that moment.

[0214] S605 can be regarded as a specific example of S504.

[0215] Returning to step S601, the microscope completes the brightfield image capture of the single cell at the next moment. Again through steps S602, S603, S604, and S605, the laser light source synchronously illuminates the target cell at the next moment, achieving continuous optogenetic control of the single cell at the next moment. Repeating steps S601, S602, S603, S604, and S605 achieves continuous optogenetic control of the single cell. This repetitive process can be considered an example of steps S506-S508.

[0216] S606. After achieving continuous single-cell optogenetic control, wide-field fluorescence photography can be performed.

[0217] Step S606 primarily involves conducting optogenetic research based on the optogenetic control of the present embodiment. Therefore, S606 is not a required step for the optogenetic control method of the present invention. After looping through steps S601-S605, in addition to performing fluorescence imaging in step S606, the phenotypic information can also be uploaded to subsequent modules for further processing, or other experiments can be performed, without limitation.

[0218] In one implementation, a method similar to the aforementioned synchronization signal can be used to synchronously control the capture of wide-field fluorescence images. Specifically, a data processing module can be used to generate a synchronization signal to control the light source module, micro-projection module, and image acquisition module to operate synchronously to capture wide-field fluorescence images.

[0219] It should be understood that the synchronization signal is not generated simultaneously with the synchronization signal in the above-mentioned continuous optogenetic control process. For example, the synchronization signal generated at one time may be used to capture a bright field image, while the synchronization signal generated at another time may be used to synchronize the capture of a wide-field fluorescence image.

[0220] also, Figure 6 The modulation process of the micro-projection module is not shown. This step is the step after step S604. S604 transmits the processed image to the micro-projection module, modulates it and then projects it to the microscope module. Then, step S605 synchronously controls the optogenetic operation process at the next moment.

[0221] The widefield fluorescence imaging function demonstrated in this example is the same as that of a standard widefield fluorescence microscope. Simply set the desired time period, fluorescence channel, field of view of interest, and Z-axis to perform the imaging.

[0222] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0223] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0224] An embodiment of the present application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the computer program.

[0225] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented.

[0226] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process of the above-mentioned embodiment method by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can at least include: any entity or device capable of carrying computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, mobile hard drive, magnetic disk, or optical disk. In some jurisdictions, based on legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunication signals.

[0227] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0228] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0229] In the embodiments provided in this application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0230] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0231] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. An optogenetic control system, characterized in that: The system includes a light source module, a micro-projection module, a data processing module, an image acquisition module and a microscope module; The output end of the light source module is connected to the input end of the micro-projection module; the light source module is used to provide light stimulation illumination; The input end of the image acquisition module is connected to the output end of the microscope module; the output end of the image acquisition module is connected to the input end of the data processing module; the image acquisition module is used to obtain bright field images of the active microbial sample at consecutive moments; The output end of the data processing module is connected to the input end of the micro-projection module; the data processing module is used to: performing image preprocessing on the bright field images at consecutive moments to obtain preprocessed images corresponding to the bright field images, wherein the preprocessed images contain phenotypic information of target cells in the active microbial sample at consecutive moments; The output end of the micro-projection module is connected to the input end of the microscope module; the micro-projection module is used to: receiving the pre-processed image from the data processing module; Modulating the preprocessed image to obtain a modulated image, wherein the modulated image is carried by a light spot and images a pattern in the light spot; Projecting the modulated image into the field of view of an objective lens in a microscope module, wherein the imaging of the active microbial sample in the microscope module corresponds to the modulated image; The data processing module is further configured to: A synchronization signal is generated to control the light source module, the image acquisition module and the micro-projection module to work synchronously.

2. The system according to claim 1, wherein The continuous moments include a first moment and a second moment, the second moment being later than the first moment; the phenotypic information includes static information and process information; The data processing module is specifically used for: receiving the bright field image at a first moment from the image acquisition module, where the bright field image at the first moment is captured by the image acquisition module under the triggering of the synchronization signal at the first moment; performing image preprocessing on the bright field image at the first moment to obtain the preprocessed image at the first moment; Extracting static information of the target cell at the first moment from the preprocessed image at the first moment; sending the pre-processed image at the first moment to the micro-projection module, so that the micro-projection module performs a modulation operation; receiving the bright field image at a second moment from the image acquisition module, where the bright field image at the second moment is captured by the image acquisition module under the triggering of the synchronization signal at the second moment; performing image preprocessing on the bright field image at the second moment to obtain the preprocessed image at the second moment; extracting static information of the target cell at the second moment from the preprocessed image at the second moment; Tracking and analyzing the bright field image at the first moment and the bright field image at the second moment are performed to obtain process information of the target cell.

3. The system according to claim 2, wherein: The micro-projection module includes a micro-projection chip, a lens and a coaxial magnification optical path; The input end of the micro-projection chip is connected to the output end of the light source module and the output end of the data processing module; the output end of the micro-projection chip is connected to the input end of the coaxial amplification optical path; the micro-projection chip is used to modulate the pre-processed image from the data processing module to obtain the modulated image; The input end of the lens is connected to the output end of the coaxial amplification optical path; the output end of the lens is connected to the input end of the microscope module; The coaxial amplifying optical path is used to receive the modulated image from the micro-projection chip and project the modulated image into the objective field of view of the microscope module through the lens.

4. The system according to claim 3, wherein: The micro-projection chip is a DMD chip; or the micro-projection chip includes a spatial light modulator, a first polarizer and a second polarizer; The first polarizer is arranged before the spatial light modulator and is the polarizer of the spatial light modulator; the second polarizer is arranged after the spatial light modulator and is the linear polarization beam splitter cube after the spatial light modulator; The spatial light modulator is used to receive the preprocessed image from the data processing module and perform phase modulation on the preprocessed image to obtain a patterned light spot, and the pattern imaging in the patterned light spot is the modulated image.

5. The system according to claim 1, wherein: The light source module includes a laser light source and a multimode optical fiber; The output end of the laser light source is connected to the input end of the multimode optical fiber; the laser light source is used to generate lasers of multiple wavelengths; The output end of the multimode optical fiber is connected to the input end of the micro-projection module; the multimode optical fiber is used to couple and expand the lasers of the multiple wavelengths to form a laser spot; the laser spot is used to irradiate the target cells in the active microbial sample.

6. The system according to claim 5, wherein: The multiple wavelengths include wavelengths corresponding to red, green and blue color components respectively.

7. The system according to claim 5, wherein: The synchronization signal is also used for: The motor is controlled to vibrate, and when the motor vibrates, the coil of the multimode optical fiber is driven to vibrate.

8. The system according to any one of claims 1 to 7, characterized in that The synchronization signal is specifically used to: control the light source module and the micro-projection module to work synchronously to illuminate the target cells; and simultaneously control the image acquisition module to shoot the active microorganism sample to obtain the bright field image.

9. The system according to any one of claims 1 to 7, characterized in that The synchronization signal is also used for: The image acquisition module is controlled so that the light source module and the micro-projection module work synchronously to obtain a wide-field fluorescence image.

10. The system according to any one of claims 2 to 4, characterized in that The static information includes at least one of length, position, size or fluorescence intensity; the process information includes at least one of growth and division information or position change information.

11. An optogenetic control method, characterized in that: include: Acquiring a plurality of bright field images of an active microbial sample at consecutive moments, wherein the plurality of bright field images are captured when target cells in the active microbial sample are photostimulated; processing the plurality of bright field images to obtain phenotypic information of the target cell at the consecutive moments; Optogenetic control is performed on the target cell according to the phenotypic information of the target cell at the continuous moments.

12. The control method according to claim 11, wherein: The phenotypic information includes static information and process information; the processing of the multiple bright field images to obtain the phenotypic information of the target cells in the active microbial sample at the consecutive moments includes: performing image preprocessing on the plurality of bright field images to obtain a plurality of preprocessed bright field images, wherein the plurality of preprocessed bright field images contain the static information; extracting static information of the target cell at the consecutive moments from the plurality of preprocessed bright field images respectively; Tracking analysis is performed on the multiple pre-processed bright field images to obtain process information of the target cells at the consecutive moments.

13. The control method according to claim 12, wherein: The continuous moments include a first moment and a second moment, the second moment is later than the first moment, and the performing optogenetic control on the target cell according to the phenotypic information of the target cell at the continuous moments includes: locating the position of the target cell in the active microbial sample at the second moment according to the process information of the target cell from the first moment to the second moment; Based on the position, the target cell is optogenetically controlled.

14. The control method according to claim 12, wherein: The control method further includes: Modulating the pre-processed image using a micro-projection module to obtain a modulated image, wherein the modulated image is carried by a light spot and images a pattern in the light spot; The modulated image is projected into the objective field of view of a microscope module, and the imaging of the active microbial sample in the microscope module corresponds to the modulated image, so that at the same moment, the position of the target cell in the preprocessed image is consistent with the position of the target cell in the active microbial sample.

15. The control method according to claim 14, wherein: The method of modulating the pre-processed image by using a micro-projection module to obtain a modulated image includes: Phase modulation is performed on the preprocessed image to obtain pattern imaging in the light spot.

16. The control method according to claim 14 or 15, characterized in that: The control method further includes: generating a synchronization signal at the continuous moments, wherein the synchronization signal is used to control the light source module, the micro-projection module and the image processing module to work synchronously at the continuous moments; The step of acquiring a plurality of bright field images of the active microbial sample at consecutive moments comprises: Under the triggering of a synchronization signal, the light source module and the micro-projection module are controlled to illuminate the active microorganism sample, while the image processing module is controlled to collect a bright field image of the active microorganism sample.

17. The control method according to claim 16, wherein: The synchronization signal is further used to control the motor to work synchronously at the consecutive moments, and the control method further includes: Under the triggering of the synchronization signal, the motor is controlled to vibrate. When the motor vibrates, the winding of the multimode optical fiber in the light source module is driven to shake.

18. The control method according to any one of claims 12 to 15, characterized in that: The static information includes at least one of length, position, size or fluorescence intensity; the process information includes at least one of growth and division information or position change information.

Images