Software-assisted laser-scanning imaging and optical manipulation system and methods of use thereof

Abstract

The invention generally relates to a software-assisted laser-scanning imaging and optical manipulation system and methods of use thereof In certain embodiments, the invention provides a software-assisted laser-scanning imaging and optical manipulation system, the system comprising: a laser-scanning apparatus; one or more optic modulators; and a processor operably associated with the laser-scanning apparatus and the one or more optic modulators, wherein the processor is configured to command coupling of one or more laser beams of the laser-scanning apparatus via the one or more optic modulators in order to selectively interact with selected locations on a sample that is being interrogated by the system.

Description

[0001] SOFTWARE-ASSISTED LASER-SCANNING IMAGING AND OPTICAL

[0002] MANIPULATION SYSTEM AND METHODS OF USE THEREOF

[0003] Related Application

[0004] The present application claims the benefit of and priority to U.S. provisional patent application serial number 63 / 546,953, filed November 2, 2023, the content of which is incorporated by reference herein in its entirety.

[0005] Government Support

[0006] This invention was made with government support under R35GM147092 awarded by the National Institutes of Health. The government has certain rights in the invention.

[0007] Field of the Invention

[0008] The invention generally relates to a software-assisted laser-scanning imaging and optical manipulation system and methods of use thereof.

[0009] Background

[0010] Optoelectronics is the field of technology concerned with electronic device applications to the sourcing, detection, and control of light. It encompasses the design, manufacture, and study of electronic hardware devices that, as a result, convert electricity into photon signals for various purposes such as medical equipment, telecommunications, and general science. Optoelectronics, in the context of science, deals with light, its detection, creation, and manipulation for various purposes. This includes X-rays, gamma rays, infrared, ultraviolet, and of course visible light. These devices are basically transducers, devices that convert one form of energy into another form of energy, and can either be electrical-to-optical, which usually means that the machine produces light by expending or using electrical energy, or they can be optical- to-electronic, which means that the device is a detector of light and transforms the detected light signals into equivalent electrical signals for computer processing.

[0011] Summary The invention provides a software-assisted laser-scanning imaging and optical manipulation technology. This technology uses software to coordinate a laser-scanning system and optic modulators that command the coupling of one or more laser beams to selectively interact with selected locations on the sample. Lasers are only activated at the desired areas of interest without affecting unwanted locations. The pixels where lasers are activated are called active pixels (APXs). Chemical information from the sample can be simultaneously monitored for APX selection and cell response measurement. The lasers activated at APXs can be used for precise optical manipulation or region-of-interest (ROI) imaging. Lasers for optical manipulation or ROI imaging can be selected from UV to IR wavelengths, and from continuous wave to pulsed lasers. This technology can selectively photobleach targeted fluorescent molecules, induce reactive oxygen species at a gated area on the sample, apply any input mask for optical manipulation, selectively control activities of molecules, perform ROI imaging to reduce phototoxicity, achieve flexible fluorescence recovery after photobleaching (FRAP) or fluorescence loss in photobleaching (FLIP) to study molecular diffusion at selected locations, and manipulate cell fate and division. This technology also allows for simultaneously controlling multiple laser wavelengths and generating any desired optical pattern for imaging and optical manipulation.

[0012] In certain aspects, the invention provides a software-assisted laser-scanning imaging and optical manipulation system that includes a laser-scanning apparatus; one or more optic modulators; and a processor operably associated with the laser-scanning apparatus and the one or more optic modulators, wherein the processor is configured to command coupling of one or more laser beams of the laser-scanning apparatus via the one or more optic modulators in order to selectively interact with selected locations on a sample that is being interrogated by the system.

[0013] In other aspects, the invention provides a method for analyzing a sample that involves providing a software-assisted laser-scanning imaging and optical manipulation system that comprises a laser-scanning apparatus; one or more optic modulators; and a processor operably associated with the laser-scanning apparatus and the one or more optic modulators, wherein the processor is configured to command coupling of one or more laser beams of the laser-scanning apparatus via the one or more optic modulators; and operating the system to selectively interact with selected locations on a sample that is being interrogated by the system, thereby analyzing the sample. In certain embodiments of the systems and methods, laser beams of the laser-scanning apparatus are only activated at desired areas of interest without affecting unwanted locations. In certain embodiments of the systems and methods, pixels where the laser beams are activated are called APXs, and chemical information from the sample can be simultaneously monitored for APX selection and cell response measurement. In certain embodiments of the systems and methods, the laser beams activated at APXs can be used for optical manipulation or region-of- interest (ROI) imaging.

[0014] In certain embodiments of the systems and methods, the laser beams for optical manipulation or ROI imaging can be selected from UV to IR wavelengths, and from continuous wave to pulsed lasers. In certain embodiments of the systems and methods, the system can be operated to accomplish at least one of the following: selectively photobleach targeted fluorescent molecules; induce reactive oxygen species at a gated area on the sample; apply any input mask for optical manipulation; selectively inhibit photoswitchable inhibitors; perform ROI imaging to reduce phototoxicity; achieve flexible FRAP or FLIP to study molecular diffusion at selected locations; or manipulate cell fate and division.

[0015] In certain embodiments of the systems and methods, the processor is further configured to allow for simultaneously controlling multiple laser wavelengths and generating any desired optical pattern for imaging and optical manipulation. In certain embodiments of the systems and methods, the process is configured to control a subset of the lasers by the one or more optic modulators separately for fast switching from ‘on’ and ‘off states. In certain embodiments of the systems and methods, the one or more optic modulators are controlled by TTL output selected by the processor that allows users to input any mask that determines where desired lasers are turned on the sample. In certain embodiments of the systems and methods, the processor can also be used in combination with a comparator circuit system that allows real-time determination of laser-interacting pixels, such that laser beams that perform optical manipulation or selective imaging are only turned on at desired pixels known, which are selected by the processor or the comparator circuit, or both.

[0016] In certain embodiments of the systems and methods, the processor controls imaging and data acquisition by synchronizing multiple channels of Analog Output (AO), Analog Input (Al), and Digital Output (DO). In certain embodiments of the systems and methods, two AO channels are used to direct illumination sources to specific X / Y positions on a sample via controlling two gal vo mirrors.

[0017] In certain embodiments of the systems and methods, the processor allows users to configure image size and location in an X / Y plain, image resolution (# pixels), measurements acquired and averaged per pixel, dwell time in each pixel, wherein one or more of these parameters are used to calculate precise mirror positions to locate an illumination source on the sample during a scan and thereby synchronize all other AO, Al and DO channels. In certain embodiments of the systems and methods, X and Y parameters are independent of one another, thereby allowing images with any desired dimensions and resolution.

[0018] In certain embodiments of the systems and methods, the system comprises additional AO channels that are available to control other external devices / events to manipulate sample conditions. In certain embodiments of the systems and methods, the system is configured to control the intensity / phase of a laser illumination source or voltage applied to an electrical grid. In certain embodiments of the systems and methods, the system is configured such that a user can create a desired pattern of events / manipulations with an analog mask that is analogous to a digital mask already in use to control the AOMs.

[0019] In certain embodiments of the systems and methods, the system currently comprises at least 16 Al channels such that the system has the capability to acquire 16 channels of data.

[0020] In certain embodiments of the systems and methods, current digital outputs have three predefined functions and five user-definable values. In certain embodiments of the systems and methods, the predefined DO lines indicate when an imaging beam is in-frame, has changed pixels, and has started a new line in an image, wherein these signals are used to synchronize external events / hardware with repeating scan events.

[0021] Brief Description of the Drawings

[0022] FIGS. 1A-D show the S-RPOC technology overview. (FIG. 1A) The configuration of the HMI system designed for precision opto-control. AOM, acousto-optic modulator; APX, active pixel. (FIG. IB) A schematic elucidating the generation of TTL signal for AOM control, determined by optical signals and user input in the software-only mode. (FIG. 1C) The generation of TTL for AOM control in tandem mode involves applying an AND function to the selected ROI created by the software and optical signals determined by the comparator circuit. (FIG. ID) A schematic illustrating the action flow of S-RPOC. The yellow, blue, and green colors represent different fluorescence signals from the sample. The magenta color represents APXs. The red and green loops depict manual selection outlines for different action lasers utilizing the S-RPOC software.

[0023] FIG. 2 panels A-L show flexible control of lasers and determination of APXs. (Panel A) Examples of selecting APXs using different intensity thresholds in an outlined area encompassing a cell nucleus. The circle in the optical signal channel is manually delineated using the S-RPOC software interface. Scale bars: 10 pm. (Panel B) A fluorescence image of untreated 1 pm fluorescent polystyrene microspheres. (Panel C) Manually designated region of interest (ROI) for 405 nm (blue) and 532 nm (green) laser treatment, with the overlapped area shown in cyan and magnified for clarity. (Panel D) The 405 nm laser treatment induces photobleaching in the selected area. (Panels E-G) Photobleaching of a Purdue logo on fluorescent microspheres using the mask presented in panel (Panel F). (Panel H) The mask of the ‘Mona Lisa’ used for photobleaching. (Panels I and J) Photobleaching of fluorescent microspheres using the mask in Panel H and its contrast-inversed counterpart. (Panel K) In a conventional confocal fluorescence microscope, the manually selected ROI for laser treatment is performed before imaging. (Panel L) In RPOC, the APXs are automatically selected and guided by user input. Laser scanning, treatment, and imaging are performed simultaneously.

[0024] FIG. 3 panels A-0 show simultaneous creation of various optical manipulation conditions in the same FOV. (Panel A) A fluorescent image of HeLa EGFP-Alpha-Tubulin Histone-H2B- mCherry cells before treatment. The mCherry signals are displayed in red and EGFP signals are displayed in green. (Panel B) APXs of 405 nm laser give different laser dosages to different nuclei. For each nucleus, manual delineation is performed followed by separate adjustment of the intensity threshold in each selected region. (Panel C) The mCherry and EGFP signals after 260 s treatment. (Panel D) The mCherry-H2B signal decay as a function of treatment time. (Panel E) The correlation between laser dosage and the mCherry-H2B signals after treatment. (Panels F-I) Similar to Panels A-C but simultaneously treated one nucleus with a 405 nm laser and another with 532 nm lasers. The 405 nm laser control is performed in tandem with the comparator circuit box, resulting in an automatic stop of laser interaction after the signal decreases to the preset threshold. The 532 nm laser is solely operated by the software and thus maintains a consistent dosage during treatment. (Panel J) The mCherry-H2B signal changes during treatment for the control (untreated), 405 nm (blue), and 532 nm (green) treated cells in panel (Panel F). Note that during treatment, signals from treated nuclei are enhanced by action lasers. (Panel K) The first- order derivative of the signals in panel (Panel J). The red dash line across j and k indicates the approximate stop point of the blue light treatment, which is around 250 s. (Panels L-M) The mCherry-H2B signals before and after treatment using solely the software. The movement of the cell nucleus during treatment results in an untreated edge, as marked by the yellow arrow. (Panels N-O) The mCherry-H2B signals before and after treatment using the software + comparator circuit tandem model. Object motion-induced photobleaching artifact is prevented. Scale bars: 10 pm.

[0025] FIG. 4 panels A-K show S-RPOC enables adaptive FRAP and FLIP. (Panel A) mCherry- H2B signals (red) in live HeLa cells before, during, and after treatment, and APXs for 405 nm (blue) and 532 nm (green) lasers. (Panel B) Changes in mCherry-H2B signals at treated and untreated nucleus sections during treatment. (Panel C) Normalized mCherry-H2B signal changes at treated and untreated nucleus sections measured immediately post-treatment. (Panel D) mCherry-H2B signal alterations at treated and untreated nucleus segments from 0 to 5 hours post-treatment. The magenta bar indicates the treatment time period. (Panel E) The EGFP- tubulin signals in the same FOV of Panel A before, during, and after treatment. (Panels F-H) Similar to Panels A-D, exhibiting tubulin signals outside the nuclei of the treated cells. (Panel I) EGFP -tubulin signals (green), APXs for 405 nm laser treatment (magenta), and the selected untreated region (yellow) for signal analysis. (Panel J) EGFP-tubulin signal changes from the treated and untreated areas in panel (Panel I) during treatment. (Panel K) EGFP-tubulin signal changes from the same areas immediately after treatment. Scale bars: 10 pm.

[0026] FIG. 5 panels A-F show inducing ROS selectively using a 405 nm laser at the mitochondria of a chosen cell. (Panel A) EB3-EGFP (green) and MitoTracker (yellow) signals before, during, and after treatment of mitochondria by a 405 nm laser in a chosen cell. The APXs (magenta) are selected on mitochondria within the selected cell. The red outline represents the cell delineated using the S-RPOC software. (Panel B) Fluorescence signals of EB3-EGFP and MitoTracker before (I), during (II), and after (III) treatment. (Panel C) Intensity changes of microtubule and MitoTracker as a function of time before and after RPOC for both the treated and untreated cells. (Panel D) MitoTracker signal changes post-treatment within and outside the mitochondria of the treated cell. (Panel E) Time-lapse images of MitoTracker (yellow) and APX (magenta) during treatment when solely relying on the comparator circuit box. The disruption of APXs over time can be visualized. (Panel F) An illustration of APX disruption induced by the combined effect of signal enhancement by the 405 nm laser and MitoTracker leakage outside mitochondria. This disruption occurs during mitochondria treatment when only the comparator circuit is applied for real-time decision-making. Scale bars: 10 pm.

[0027] FIG. 6 panels A-H show regulating the short-term and long-term behaviors of specific cells. (Panel A) EB3-EGFP signals from HeLa cells before PST-1 activation. The cell for treatment is outlined. (Panel B) APXs selected for activating PST-1. (Panel C) EB3-EGFP signals from HeLa cells after PST-1 activation. (Panel D) Fluorescence signals of mCherry-H2B (red) and EGFP -tubulin (green) from HeLa cells at different time points and their corresponding bright field images. One centrosome is treated with 240 pW 405 nm laser (blue pixels, purple arrow), while the other is treated with 240 pW 532 nm laser (yellow pixels, green arrow). (Panel E) Time-lapse images of HeLa cells expressing EB3-EGFP before, during, and 6 hours after 405 nm laser treatment of the circled area in the nucleus. The treatment time is 560 s and the laser dosage is 4 mJ. (Panel F) Similar to Panel E but the nucleus is treated for 280 s and a laser dosage of 2 mJ. (Panel G) EB3-EGFP signal changes during treatment for cells in Panels E-F. (Panel H) Bright-field images collected 10 hours after treatment from the same FOVs in Panels E-F. Scale bars: 10 pm.

[0028] FIG. 7 panel A shows the optical and electronic configurations of the S-RPOC setup. AOM, acousto-optic modulator; PD, photodiode; NDF, neutral density filter; DM, dichroic mirror; QWP, quarter- wave plate; PMT, photomultiplier tube; PBS, polarizing beam splitter. FIG. 7 panel B shows a workflow depicting the decision-making process across various S-RPOC operational modes. APXs, active pixels; TTL, transistor-transistor logic; ROI, region of interest.

[0029] FIG. 8 shows the input / output and functionalities of the dual-channel comparator circuit box.

[0030] FIG. 9A shows one configuration of the comparator circuit box when employed in tandem with the S-RPOC software to perform real-time APX determination for mobile targets within the selected regions of interest. FIG. 9B shows another configuration of the tandem mode allows to select APXs on mobile targets outside of the outlined regions of interest. When the bottom ‘invert’ switch is switched to non-invert, this configuration gives the same function as FIG. 9A. FIG. 10 shows the summary of the distinctive functions and features of S-RPOC in the software-only, comparator-circuit-only, and software + comparator-circuit tandem modes. The conventional RPOC only utilizes the comparator circuit.

[0031] FIG. 11 shows signals from EB3-EGFP in HeLa cells and the chosen APXs based on the EB3-EGFP signals for concurrent 532 nm and 405 nm treatment. The composite images superimpose EB3-EGFP signals with all APXs. Scale bars: 10 pm.

[0032] FIG. 12 panels A-H show examples of HMI functionalities on selecting APXs. (Panels A-D) Selection of APXs through manual outlining of an ROI containing a nucleus expressing mCherry-H2B and applying different intensity thresholds. A reduced threshold value enlarges the selected nucleus area. (Panel E) Setting the threshold to 0 V or below allows the selection of APXs that cover the entire designated ROI. (Panel F) Employing the inverse function within a specific area enables the selection of APXs below the intensity threshold within the chosen ROI. (Panels G-H) Utilizing the Tn Range' function facilitates the selection of APXs within a specific mCherry intensity range of the nucleus.

[0033] FIG. 13 panels A-J show examples of HMI functionalities in selecting APXs. (Panel A) Using the ‘Multi -sei ection’ function to designate multiple ROIs. (Panel B) Defining the APXs by adjusting the intensity threshold following the multi-area selection. (Panels C-E) Sequentially adding APXs in different ROIs for the 405 nm laser interaction using the 'Add to line' function. The intensity threshold of each ROI can be individually adjusted. (Panel F) Employing the 'Invert' function to reverse the selected APXs for the entire FOV after selection. (Panel G) Designating an ROI for 405 nm laser treatment. (Panel H) Designating an ROI for 532 nm laser treatment. (Panel I) Displaying chosen APXs for both lasers in Panels G-H. (Panel I) Using the ‘Invert’ function exclusively for the 532 nm laser to reverse its APXs within the selected ROI.

[0034] FIG. 14 shows the tubulin-EGFP signal changes during RPOC from cells shown in FIG. 3 panels A-C.

[0035] FIG. 15 panel A shows the image of APXs defined only by the S-RPOC software (green) and in tandem with the software and comparator circuit (blue). FIG. 15 panel B shows the corresponding intensity profiles of the white dash lines in the APX image. The scale bar is 10 pm.

[0036] FIG. 16 shows absolute mCherry -H2B signal changes in the treated and untreated areas in the nuclei shown in FIG. 4 panels A-C. FIG. 17 panel A shows EB3-EGFP signals for HeLa cells continuously excited by 25 pW 473 nm laser in different time windows. FIG. 17 panel B shows time-lapse EB3-EGFP signal changes under continuous 473 nm laser scanning for 400 seconds. Scale bars: 10 pm.

[0037] FIG. 18 panel A shows EB3-EGFP (green) and MitoTracker (yellow) signals from HeLa cells continuously excited by 25 pW 473 nm laser and 10 pW 589 nm laser in two-time windows. FIG. 18 panel B shows the APX (magenta), EB3-EGFP (green), and MitoTracker (yellow) signals during 405 nm laser treatment targeting mitochondria. Here, only the comparator circuit is used to determine APXs. The S-RPOC software is not applied. Scale bars: 10 pm.

[0038] FIG. 19 panels A-D show the control of cell division using S-RPOC. (Panel A) shows a HeLa cell expressing mCherry-H2B (red) and EGFP-tubulin (green) in the mitotic phase visualized using the S-RPOC fluorescence imaging channels and the bright-field transmission mode. (Panel B) Treatment of centrosomes using 240 pW 405 nm (blue) and 240 pW 532 nm (yellow) lasers for 275 seconds. (Panel C) Time-lapse images of the treated cell at different time points after treatment. (Panel D) After 15 hours, the cell transitions into a multinucleated form. Scale bars: 10 pm.

[0039] FIG. 20 panels A-D show replicating the modulation of cell division using S-RPOC via the treatment of centrosomes. (Panel A) A HeLa cell expressing mCherry-H2B (red) and EGFP- tubulin (green) in the mitotic phase visualized using the S-RPOC fluorescence imaging channel. (Panel B) Treatment of centrosomes using 240 pW 405 nm (blue) and 240 pW 532 nm (yellow) lasers. (Panel C) Time-lapse images of the treated cell at different time points post-treatment. (Panel D) Time-lapse fluorescence and bright-field images of the treated cell at different time points after 5 hours. Scale bars: 10 pm.

[0040] FIG. 21 panels A-D show the control of cell viability using RPOC. (Panel A) shows EB3- EGFP signals from HeLa cells before, during, and after treatment of the nucleus of a specific cell by using a 240 pW 405 nm laser. The unlabeled nucleus is chosen based on the contrast of the EB3-EGFP signals. The treatment is performed for 560 seconds. (Panel B) Bright-field images of the same FOV in Panel A at different time points after treatment. (Panel C) Similar to Panel A, exhibiting EB3-EGFP signals, but treated for only 280 seconds. (Panel D) Similar to Panel B, illustrating the same FOV and cells featured in Panel C using bright field imaging. Scale bars: 10 pm. FIG. 22 panels A-D show the killing of a cancer cell in a cancer and fibroblast co-culture system. (Panel A) shows time-lapse fluorescence images of Pane. 10.05 cells (red) and CAF 19 cells (green) before, during, and after treatment with a 240 pW 405 nm laser targeting the selected APX highlighted in magenta. (Panel B) Temporal changes in fluorescence signals for the CAF 19 cell (red), the untreated Pane. 10.05 cell (black), and the treated Pane. 10.05 cell (blue) during treatment and the subsequent 400 seconds. (Panel C) Post-treatment, the fluorescence signals from CAF 19 cells (red columns) notably decrease but gradually recover in a 6-hour duration. Both untreated (black columns) and treated Pane. 10.05 cells exhibit decreased fluorescence signals, with a more pronounced reduction observed in the treated cell. (Panel D) B right-fi eld images captured before and after treatment within the same FOV at different time points. The detachment of the treated cell from neighboring cells and the condensation of the nucleus in the treated cell are detected. Scale bars: 10 pm.

[0041] Detailed Description

[0042] The invention generally relates to a software-assisted laser-scanning imaging and optical manipulation system and methods of use thereof. In certain embodiments herein, the optoelectronic system includes several lasers for real-time imaging and optical manipulation. A subset of the lasers can be controlled by acousto-optic modulators (AOMs) separately for fast switching from the ‘on’ and ‘off states. The AOMs are controlled by TTL output selected by software (Lab VIEW is used as an example) that allows users to input any mask that determines where the desired lasers are turned on the sample. The software can also be used in combination with a comparator circuit system that allows real-time determination of the laser-interacting pixels. Lasers that perform optical manipulation or selective imaging are only turned on at the desired pixels known as active pixels (APXs), which are selected by the software or the comparator circuit, or both.

[0043] The imaging program / processor controls all aspects of the imaging and data acquisition process by precisely synchronizing multiple channels of Analog Output (AO), Analog Input (Al) and Digital Output (DO). In certain embodiments, the systems of the invention use two AO channels to direct illumination sources to specific X / Y positions on a sample via controlling two galvo mirrors. While at a given location, the program / processor uses additional AO, Al and DO channels to control data acquisition and external influences / manipulations by any device compatible with TTL and analog control signals. The examples cited herein demonstrate the use of analog Inputs to read signals from photodetectors. More generally, signals can be recorded from any sensor capable of generating analog voltage signals, for example, an ultrasound transducer used in photoacoustic spectroscopy. The AO and DO lines can control any device accepting these control signals. In the embodiments herein, the DO lines may be used to control the AOMs thereby controlling the secondary illumination of the sample during an image scan. These additional outputs could just as easily be used to control a device generating electrochemical pulses applied to the sample at various points in the scan.

[0044] The program / controller interface allows users to configure image size and location in the X / Y plain, image resolution (# pixels), measurements acquired and averaged per pixel, and dwell time in each pixel. These parameters may be used to calculate precise mirror positions to locate the illumination source on the sample during the scan and synchronize all other AO, Al, and DO channels. The X and Y parameters may be completely independent of one another thereby allowing images with any desired dimensions and resolution.

[0045] The current system hardware has additional AO channels available to control other external devices / events to manipulate sample conditions. An example could be controlling the intensity / phase of a laser illumination source or voltage applied to an electrical grid. Users will create the desired pattern of events / manipulations with an analog mask analogous to the digital mask already in use to control the AOMs.

[0046] The current processor / program hardware has 16 Al channels. Imaging needs have typically used 2-4 channels of data collection. The current application has the capability to acquire 8 channels of data. Implementing additional channels would be very straightforward to one of skill in the art.

[0047] The current digital outputs have three predefined functions and five user-definable values. The predefined DO lines indicate when the imaging beam is in-frame, has changed pixels, and has started a new line in an image. These signals can be used to synchronize external events / hardware with repeating scan events such as starting a new line. One of the user- definable lines is used to provide a mask repeated in each pixel. The four remaining user- definable lines allow for control of four devices when applying a mask to the entire image.

[0048] If this technology is used for optical manipulation, it is known as the software-assisted real-time precision opto-control (RPOC) system. As an example, FIG. 7 panel A shows a typical RPOC optical configuration with five continuous wave (CW) lasers available. Here, the 633 nm, 589 nm, or 473 nm laser, or in any combination, are used as chemical selection lasers to identify chemicals of interest from the sample. These lasers can also be used to excite fluorophores that are readout of cell responses. The 405 nm and 532 nm lasers are used as optical manipulation lasers. AOMs are used to control the coupling of the 405 nm and 532 nm lasers to the sample at selected APXs. Four laser beams are combined and scanned by a pair of galvo mirrors on the sample. Optical signals (e.g. fluorescence) generated from the sample are acquired by photomultiplier tubes (PMTs) placed in the epi-directi on and in confocal configurations. Pixels at the regions of interest (ROI) selected by the software output a ‘ 1 ’ TTL signal during laser scanning to turn on the optical manipulation laser only at the ROIs. A ‘0’ TTL output is applied for all areas outside of the ROI. The ROIs can be manually selected in the software by gating or fed in by uploading an image. A thresholding function is available in the software to convert the image to a mask for the TTL output. The TTL output is used to command the AOM for coupling the optical manipulation laser to the sample. The TTL signals for AOM control, image acquisition, and galvo mirror position are all under software control to ensure precise synchronization and timing. The workflow of this software-assisted opto-control is illustrated in FIG. 7 panel B. Using this technology, active pixels can be selected at any location on the sample, regardless of having optical signals or not (FIG. 1 panel B). A manual drawing tool is included in the software to the selection of ROI. This platform can also be used in tandem with the RPOC system based on a comparator circuit box to selectively track and manipulate dynamic chemical targets in the sample (FIG. 1 panel C).

[0049] Note that lasers for chemical detection, optical manipulation, and readout are not limited to the options shown in FIG. 7 panel A. They can be selected from CW lasers of any wavelength (from UV to IR) and pulsed lasers of any pulse duration (ms to fs). The selection of lasers allows to detect chemical targets using different modalities and manipulate biomolecular targets using various mechanisms such as blue-light generated reactive oxygen species (ROS), light-activated photoswitchable inhibitor, pulsed-laser-induced low-density plasma, IR-laser-induced selective heating, etc.

[0050] FIG. 2 panel B shows a typical image of fluorescent micro-particles (1 pm in size) detected in the chemical selection channel. FIG. 2 panels C and D show APX selected using the software for controlling the blue and green optical manipulation lasers, respectively. FIG. 2 panel D shows that after 30 frames of optical manipulation, the blue-laser-activated area shows bleached fluorescence signals from the beads while the green-laser-activated area does not have photobleaching. Aside from selecting a gated mask from the software, the mask of any image input allows one to turn on the optical manipulation laser at selected areas and photobleach fluorophores of any arbitrary patterns (FIG. 2 panels E through J). An inversion function can be applied to any mask to create a reversed photobleaching or manipulation effect (FIG. 2 panels H- J).

[0051] This technology can also be applied for flexible fluorescence recovery after photobleaching (FRAP) or fluorescence loss in photobleaching (FLIP). In FIG. 4 panel A, fluorescence signals from nuclei histone-2B (H2B) are detected in HeLa cells. Software-assisted RPOC allows to select areas within two nuclei for interaction with 405 or 532 nm lasers. The lasers induce fluorescent signal loss of H2B in the laser-radiated areas (FIG. 4 panel B). After RPOC, signal recovery was detected (FIG. 4 panel C and panel D) which was attributed to both dynamics of H2B in the nuclei detected as FRAP and the synthesis of H2B by the cells. Quantitative analysis by monitoring fluorescent signal changes over time can study protein dynamics and production in cells (FIG. 4 panel D).

[0052] Another example is to pertrub cell functions precisely using the 405 nm laser. The 405 nm laser is known to generate ROS which might easily damage the endoplasmic reticulum (ER) and nuclei. ROLLSM can be selected at the mitochondria or other subcellular locations to monitor cell responses to localized laser perturbation without affecting the functions of ER or nuclei. FIG. 5 panel A shows APX selected from a manually gated area targeting mitochondria in live cells. The mitochondria are labeled using MitoTracker Red (FIG. 5 panel A). Blue laser at 405 nm was activated at the APX for mitochondria blue light interaction. This approach gives fixed laser doses for mitochondria ROS generation. Such a fixed laser dose cannot be achieved using the comparator circuit since MitoTracker signals experience photobleaching and signal crosstalk during RPOC. By using the 473 nm laser and the readout channel at 509 / 22 nm, we can monitor the loss of mitochondria membrane potential (FIG. 4 panel A) and the decrease of microtubule polymerization (FIG. 4 panels B-D) caused by mitochondria-associated ROS.

[0053] This technology allows to activate drugs only in selected cells or subcellular areas. FIG 6 panel A shows the APX selected in HeLa cells transfected with EB3-EGFP. The cells are treated with PST-1, a photoswitchable microtubule polymerization inhibitor. The activated PST-1 disrupted microtubule polymerization only in the selected cell while not affecting other cells, as show in FIG. 6 panel B and panel C.

[0054] This technology allows to control of cell fate. In FIG. 6 panel D, a HeLa cell in the mitotic phase is detected. DNAs in the cell are shown in red while microtubules are shown in green. A 405 nm laser is used to interact with one of the centrosomes of the cell. One hour after the RPOC treatment, the 405 nm laser treatment induces microtubule and chromosome disruption (FIG. panel 6D). The cell is monitored for 18 hours after the RPOC and multi-nuclei formation is found (FIG. panel 6D). This result shows that RPOC is able to control the cell fate by selectively interacting with subcellular molecular targets with lasers.

[0055] FIG. 6 panel E shows the manually selected APX area inside the nuclei of a HeLa cell. Here, the nuclei of the cells do not have optical signals but have inverted contrast in the EB3 fluorescence channel. Blue light at 405 nm is activated only at these APXs at a fixed intensity of 4 mW. ROS generated inside these areas for HeLa cells can induce changes in microtubule polymerization and cellular fluorescent protein signals which can be monitored in the readout channels (FIG. 6 panels E and F). Using this method, we can quantify changes in EB3 signals and dynamics in HeLa cells induced by ROS generated by blue light in nuclei.

[0056] This technology can be potentially used for the selective activation of neuronal activities for different neurons using the same or different laser wavelengths. Different from wide-field structured illumination methods, the laser scanning methods offer several advantages including:

[0057] (1) The masks generated by the software-assisted RPOC technology are free of laser speckles.

[0058] (2) It can simultaneously and separately control several laser wavelengths for precise optical manipulation. (3) It can perform two-photon optical manipulation and imaging that requires tight laser focusing and fast laser scanning.

[0059] This technology can be used for the region of interest laser scanning microscopy (ROL LSM). As illustrated in FIG. 11, the sample (e.g. a cell) can be imaged with a low dose of laser to reveal the morphology or chemical distributions for ROI selection. The lasers for initial imaging to determine the ROIs are called probing lasers. Then, an ROI can be manually or automatically selected using the software or via image processing for ROI-LSM. A higher dose of laser can be applied to the ROIs for functional imaging to improve the sensitivity at the ROIs. Here, the lasers that are only turned on at the ROIs for functional imaging are called tracing lasers. Since the tracing lasers are only activated at the APXs in the ROIs, it reduces the phototoxicity for the sample and can be applied at a higher dose.

[0060] As shown herein, the developed technology can be used for optical manipulation, ROI- LSM imaging, and flexible FRAP or FLIP. It is expected to have wide applications in fundamental cell biology and medical science.

[0061] Incorporation by Reference

[0062] References and citations to other documents, such as patents, patent applications, patent publications, journals, books, papers, web contents, have been made throughout this disclosure, including to the Supplementary. The Supplementary, and all other such documents are hereby incorporated herein by reference in their entirety for all purposes.

[0063] Equivalents

[0064] The invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The foregoing embodiments are therefore to be considered in all respects illustrative rather than limiting to the invention described herein.

[0065] EXAMPLES

[0066] Example 1 : Precise Optical Manipulation of Cell Behaviors via Advanced Human-Machine Interaction

[0067] The traditional method in biological science to regulate cell functions often employs chemical interventions, which commonly lack precision in space and time. While optical manipulation offers superior spatial precision, existing technologies are constrained by limitations in flexibility, accuracy, and response time. Here, we present precise optical manipulation of molecular activities and cell behaviors via innovations in human-machine interaction (HMI). A software-assisted real-time precision opto-control (S-RPOC) was developed by integrating adaptive target selection and flexible decision-making. The advanced human-machine interface facilitates automatic target selection driven by optical signals while permitting user-defined delineation. It allows the creation of various optical manipulation conditions in the same field of view and simultaneous monitoring of short-term and long-term cell responses. Specifically, S-RPOC showcases versatile capabilities including adaptive photobleaching, comprehensive quantification of protein dynamics, selective organelle perturbation, control of cell division, and manipulation of individual cell behaviors within a population. By bringing advanced HMI to optical manipulation, S-RPOC holds the promise to advance our knowledge in site-specific biomolecular activities and bring about new approaches to control behaviors of biological samples.

[0068] Introduction

[0069] Optical imaging serves as a powerful tool for unraveling intricate biological processes. However, it cannot actively manipulate molecular targets and cellular behaviors. Active control of intracellular targets is crucial for deciphering correlations between biomolecules and cellular responses. Conventional approaches to manipulating molecular activities, such as chemical treatments or genetic interventions, lack spatiotemporal precision, potentially resulting in unintended side effects on non-targeted sites. Hence, to comprehensively understand dynamic molecular mechanisms and the causalities underlying cell behaviors, it is imperative to manipulate cellular targets with high spatiotemporal accuracy.

[0070] Laser-based optical manipulation methods leverage various light-matter interactions to modulate chemical processes at high spatiotemporal precision. These techniques harness lasers to apply mechanical forces, induce thermal perturbation, regulate chemical processes, activate proteins, generate reactive oxygen species (ROS), and control photoswitchable compounds that interrupt molecular processes. One paradigm of optical manipulation techniques relies on tightly focused laser beams that precisely trigger desired effects at the focal point. For example, optical tweezers use optical gradient forces to manipulate specific targets. Focused pulsed lasers facilitate tasks such as cutting cell membranes for drug delivery, disrupting mitochondria functions, and inducing cell fusion. However, existing manipulation platforms rely on manually delineating targets based on prior knowledge from the sample, lacking the ability for chemicalbased automatic target selection, and are unsuitable for highly mobile compositions. Alternatively, another approach of optical manipulation involves patterned illumination, utilizing widefield illumination and spatial light modulation techniques. This method enables the simultaneous regulation of multiple neurons with optogenetics. However, it also necessitates prior sample information followed by the computation of a mask for spatial light modulation. The resulting light patterns often exhibit speckles, particularly in scenarios involving random or intricate patterns. Furthermore, it cannot simultaneously produce precise light patterns across diverse wavelengths. Compared with the focused laser approach, wide-field patterned illumination generally has diminished spatial accuracy and is incompatible with modalities that rely on tightly focused laser beams such as two-photon excitation fluorescence.

[0071] To address the limitations of existing optical manipulation technologies, we have developed a real-time precision opto-control (RPOC) technology that harnesses chemical signals to command lasers for optical manipulation. RPOC utilizes a real-time optoelectronic feedback system to enable automated selection of the laser interaction locus during scanning. A priori knowledge of chemical distributions in the sample is not required. However, in the current RPOC framework, all chemical targets within the field of view (FOV) are chosen using a single discriminator and treated with a singular laser wavelength. The RPOC that solely relies on the comparator circuit for decision-making does not permit effective human-machine interaction (HMI) functionalities such as flexible target delineation and assigning different treatment conditions in the same FOV. Moreover, the presence and stability of optical signals are required for identifying the manipulation targets.

[0072] Here, we developed an advanced HMI system, termed software-assisted RPOC (S- RPOC), for optical manipulation. By integrating user-defined spatial discriminators with signal- defined intensity discriminators, S-RPOC offers unprecedented flexibility, automation, and precision across chemical target selection, laser control, and real-time monitoring of target responses. It facilitates the concurrent creation of various optical treatment conditions within the same FOV while also permitting comprehensive monitoring of both short-term and long-term cell responses. Applying S-RPOC, we demonstrate various new capabilities in biological science, including adaptable photobleaching to enhance understanding of protein dynamics, sitespecific perturbation of organelle functions, selective activation of inhibitors only in designated cells, manipulation of cell division via centrosomes perturbation, and label-free blue-light perturbation of cell nucleus. S-RPOC represents a significant advancement in HMI for optical manipulation, overcoming existing limitations and paving the way for new possibilities in biological science. Advanced HMI System For Optical Manipidation

[0073] The HMI configuration developed for advanced optical manipulation is shown in FIG. 1A. This system dynamically integrates an interactive RPOC software with a multifunction Input / Output (Multi-I / O), the galvo scanner, and acousto-optic modulators (AOMs). The AOMs control action lasers that facilitate precise modulation of chemical processes in the FOV. A multichannel comparator circuit can be used separately or in tandem with the S-RPOC software for the selection of active pixels (APXs). The HMI system is integrated into a lab-built laserscanning confocal microscope, as illustrated in FIG. 7 panel A. This platform is equipped with five continuous-wave (CW) solid-state lasers, ensuring coverage across excitation spectra of common fluorescent dyes and proteins. Specifically, the 405 and 532 nm lasers, commanded by AOMs, serve as the action lasers controlling cellular chemical processes. Other laser wavelengths are available for the excitation of fluorophores for chemical target selection or readout of cell responses. A stage-top incubator maintains the ideal culture conditions for cells and allows continuous monitoring of cell responses in the same FOV for up to 72 hours.

[0074] S-RPOC offers diverse HMI functionalities for APX selection, allowing manual, automatic, or combined approaches. The HMI decision-making flowchart is visualized in FIG. 7 panel B. Through the software subprogram, users can manually define the region of interest (ROI), followed by automatic target selection within the outlined areas. To perform this function, obtaining a sample image assists in selecting ROIs within the FOV. Within the ROI, intensity thresholding automatically identifies molecular targets for optical control. Different ROIs and molecular targets can be assigned to different action lasers. Furthermore, the software enables direct input of an image mask or a manually selected portion of the mask to choose APXs. These designated APXs can then be assigned to the action lasers. Different action lasers can work simultaneously during laser scanning. Notably, the manual ROI selection function facilitated by the software interface can be used in tandem with a comparator circuit (FIG. 8 and FIGS. 9A-B). The circuit allows real-time comparison of the optical signals with a predetermined intensity threshold for APX selection. This integration offers an advantage for simultaneously tracking and precise control of moving targets within any ROI in an unsupervised manner. The response time of the active feedback loop was measured to be less than 600 ns, primarily due to the response of the AOM driver. The response time of the software is about 67 ns, majorly determined by the 1.5 MHz sampling rate of the data acquisition card. This makes the total response time of S-RPOC approximately 600-700 ns, still much faster than the pixel dwell time.

[0075] FIG. IB explains how the transistor-to-transistor logic (TTL) command is determined from optical signals to control the AOM in the software-only mode. User input, such as ROI delineation, adjusting the intensity threshold, or uploading a mask, specifies the APXs that are synchronized with galvo scanning to send TTL T to AOM only at APXs. In tandem mode, as illustrated in FIG. 1C, the software-selected ROI and the comparator circuit-selected APXs are computed with an AND function to only address mobile targets within the ROI. FIG. ID demonstrates the action flowchart employing S-RPOC for target selection and simultaneous monitoring of both APXs and cell responses. A comprehensive discussion of the functionalities of the platform, as well as the advantages and limitations associated with different modes, can be found in the Examples 8-9 and FIG. 10). In short, the software-only mode offers fixed laser dosage throughout treatment and allows mask input, albeit requiring prior knowledge of the sample and being applicable primarily for stationary targets. Conversely, the comparator + software tandem mode enables the tracking and control of highly mobile targets (FIG. 11), yet the laser dosage might be affected by alterations in optical signals. Both modes facilitate the manual outlining of ROIs in any shapes and sizes, as well as automatic target selection based on optical signals. S-RPOC facilitates the creation of diverse optical manipulation conditions within the same FOV, which represents a significant advantage over conventional methodologies.

[0076] Examples of basic HMI functionalities for selecting APXs based on a pre-acquired image are shown in FIG. 2 panel A, FIG. 12 panels A-H, and FIG. 13 panels A-J. Each manual delineation action selects APXs within the delineated area with adjustable APX thresholds. APXs selected from each action can be continuously added to different laser channels. FIG. 2 panels B-D illustrate a manual outlining ROIs on fluorescent microspheres for treatment with 405 nm and 532 nm lasers using a digital sketch board connected to the S-RPOC interface. The photobleaching effect, induced solely by the 405 nm laser, can be monitored both during and after laser treatment. The system also facilitates APX selection using a predetermined mask. FIG. 2 panels E-G show the photobleaching of fluorescent microspheres using a Purdue logo as the input mask. Additionally, FIG. 2 panels H-I display the bleaching of ‘The Mona Lisa’ and the inverted pattern on fluorescent microspheres using the painting mask and its contrast inversion. The manual delineation and mask input can be used in tandem. These results highlight the adaptability of S-RPOC in controlling lasers for optical manipulation across any patterns and laser combinations.

[0077] FIG. 2 panels K-L describe key distinctions between S-RPOC and conventional laser beam control. Different from a conventional confocal fluorescence microscope, where lasers scan through various ROIs before imaging, S-RPOC selectively activates or deactivates the action lasers at chosen APXs during imaging. The executions of image acquisition and optocontrol occur simultaneously without interruption. Consequently, S-RPOC maintains a consistent speed of optical manipulation and imaging regardless of the complexity of the treatment pattern. It also enables the simultaneous treatment of diverse areas in any pattern or distribution using different action lasers. Furthermore, as shown in FIG. 2 panel A, FIG. 12 panels A-H, and FIG. 13 panels A-J in contrast to manual ROI selection in a standard confocal microscope, APXs in S- RPOC can be automatically chosen based on the intensity thresholding of optical signals from the sample. These APXs perfectly align with desired molecular targets forming any distribution pattern and can be flexibly adjusted using the manual selection and intensity thresholding functions.

[0078] Concurrently Creating Diverse Optical Manipulation Conditions

[0079] One of S-RPOC’ s major strengths lies in its capability to simultaneously create diverse optical manipulation conditions within the same FOV, enabling simultaneous comparisons of multiple parameters and enhancing manipulation throughput. As exemplified in FIG. 3 panels A- C, different nuclei of HeLa Kyoto Hi stone-H2B-m Cherry EGFP- Alpha-Tubulin cells were treated at different laser dosages using a 405 nm laser simultaneously. The cell responses to different treatment conditions are monitored concurrently during treatment using the mCherry or EGFP signals. The mCherry signal decrease observed in nuclei (FIG. 3 panels D-E) is majorly resulted from the generation of ROS induced by the 405 nm laser. The photobleaching effect of the 405 nm laser is weak, as this wavelength does not match the absorption band of mCherry. A greater laser dosage leads to increased ROS generation, giving a faster fluorescence signal decay. The ROS generated in the nuclei also causes a decrease in EGFP -tubulin signals, which can be correlated with laser dosage (FIG. 14). It is important to note that cells treated with 0.77 mJ and 1.54 mJ in the nuclei are highly overlapped in both the cytoplasm and nuclei. Consequently, the treatment of one cell can impact the other, resulting in a more pronounced decay of the nuclei and tubulin fluorescence signals compared to the ideal condition. For well-separated cells, such as those were treated with 0.23 mJ, 1.51 mJ, and 2.47 mJ, higher laser dosages lead to a faster decay of both mCherry and EGFP signals (FIG. 3 panels D-E, and FIG. 14). Furthermore, S- RPOC permits simultaneous treatment of nuclei using 405 nm and 532 nm lasers within the same FOV (FIG. 3 panels F-I). In contrast to ROS, the 532 nm laser primarily induces photobleaching of mCherry -H2B signals, resulting in a distinct signal decay pattern (FIG. 3 panel J). The 405 nm laser does not induce much photobleaching since mCherry does not have an absorption band at 405 nm. Compared to the 532 nm laser, the 405 nm ROS-induced mCherry signal decrease is less significant.

[0080] In FIG. 3 panels F-I, the combined use of the comparator circuit with manual nuclei selection is applied for the 405 nm treated nuclei. This approach enables the automatic cessation of optical treatment once the optical signal decreases to a predetermined level. This automatic halting of the action laser is facilitated through the dynamic determination of the APX through the comparator circuit. FIG. 3 panels J-K illustrate the automatic stop of 405 nm ROS generation when the signal decreases below the intensity threshold. In FIG. 3 panels F-I, the intensity threshold for the 405 nm treated nucleus is set at 50% of the initial fluorescence signal level. In the absence of the 405 nm action laser, a slow decline in fluorescence signal observed in the control group is caused by photobleaching induced by the 473 nm and 589 nm excitation lasers. The exertion of the 405 nm laser triggers a significantly accelerated signal decay owing to the generation of ROS. By comparing the first derivative of the signal decays, we can evaluate that a total laser dosage of 0.24 mJ over 250 seconds induces about 50% photodamage to mCherry molecules before the automatic stop of the treatment. Conversely, in FIG. 3 panels F-I, the treatment of nuclei with the 532 nm laser is solely managed by the software, employing a consistent laser dosage throughout the treatment duration without automated laser suspension (Example 11 and FIG. 18 panels A-B).

[0081] Furthermore, the combined use of the comparator circuit and software ensures precise treatment of moving targets within the designated area. Relying solely on the software maintains constant APXs and laser dosages in the selected regions throughout the treatment. However, if the target moves, it can shift away from the fixed APX areas, leading to untreated mismatches, as shown in FIG. 3 panels L-M. This disparity is resolved when the comparator circuit is used alongside the software, eliminating untreated nucleus boundaries (FIG. 3 panels N-O). Facilitating Adaptable FRAP And FLIP For Improved Understanding Of Intracellular Protein Dynamics

[0082] Fluorescence recovery after photobleaching (FRAP) and fluorescence loss in photobleaching (FLIP) are established techniques to investigate the dynamics of fluorophores in live cells. However, existing confocal fluorescence platforms used for FRAP or FLIP are constrained by limitations related to ROI selection and laser control. As illustrated in FIG. 2 panels K-L, conventional methods direct lasers to scan through manually chosen ROIs before imaging. This hinders capturing the information on fluorophore dynamics during the photobleaching process. Additionally, this traditional photobleaching approach is impractical in tracking complex patterns created by random fluorophore distributions. Furthermore, the separation of photobleaching and imaging processes does not allow precise laser interaction only with mobile fluorescent species. S-RPOC overcomes these limitations by employing optical- signal-driven real-time target selection and laser activation during laser scanning.

[0083] Using S-RPOC, we investigated protein dynamics in HeLa cells expressing Histone- H2B-mCherry and EGFP-Alpha-Tubulin. By outlining portions of two nuclei and adjusting intensity thresholds (FIG. 4 panel A), we selected APXs for 405 nm (240 pW) and 532 nm (320 pW) lasers. We tracked H2B-mCherry signals before, during, and after a 260-second optical treatment, observing distinct responses in treated and untreated areas within nuclei. The 532 nm laser induced photobleaching of H2B-mCherry, while the 405 nm laser generated ROS, resulting in different signal decrease patterns (FIG. 4 panels A-B) similar to FIG. 3 panels G-K. Immediately after treatment, the nucleus area treated with the 532 nm laser exhibited FRAP while the 405 nm treated area, due to the disturbance from ROS, continued to show a gradual signal decline similar to the control (FIG. 4 panel C). Further discussion is available in Example 12 and FIGS. 15A-B and FIG. 16). Monitoring mCherry-H2B signal changes over 5 hours (FIG. 4 panel D) also revealed an initial decrease in untreated ROIs, contrasted by continuous signal recovery in treated ROIs. This suggests that immediately after treatment, H2B loss due to protein mobility in the untreated ROIs exceeds cellular H2B synthesis. Two hours post-treatment, cellular production of H2B overtakes the migration-induced signal loss, resulting in a gradual increase in fluorescence signal in all areas.

[0084] During the treatment of nuclei, we simultaneously monitored tubulin EGFP signals (FIG. 4 panel E). Our findings revealed that the 405 nm treatment of a partial nucleus led to a faster decrease in tubulin EGFP signals compared to the case of the 532 nm treatment (FIG. 4 panel F). This difference further supports the fact that the alterations induced by the 405 nm laser are attributed to ROS that oxidizes EGFP. After the treatment, the accelerated tubulin signal loss induced by ROS prominently slowed down (FIG. 4 panel G). Long-term monitoring displayed a two-hour continuous intensity decrease in tubulin-EGFP signals before recovery (FIG. 4 panel H).

[0085] The H2B protein exhibits a recovery half-time significantly longer than 5 hours (FIG. 4 panel D). To investigate proteins with a shorter diffusion time scale, we conducted a FRAP study of tubulin. APXs are selected at a subcellular location and the 405 nm laser was used for photobleaching (FIG. 4 panel I). The APX area demonstrates a faster EGFP signal decrease compared to the untreated ROI during RPOC (FIG. 4 panel J). After bleaching, the treated ROI shows a signal recovery within 12 seconds (FIG. 4 panel K).

[0086] These results emphasize the power of S-RPOC in adaptive ROI selection and automated APX determination for FRAP and FLIP. In addition, optical imaging in RPOC is uninterrupted by the treatment process, allowing to capture important cell responses during treatment. Collectively, these capabilities enhance the FRAP and FLIP studies, offering improved insights into protein dynamics across various treatment conditions in a single FOV.

[0087] Precisely Perturbing Mitochondria Within Specific Cells In A Population

[0088] Mitochondria act as cellular powerhouses. Light-induced ROS generated within mitochondria can disrupt mitochondrial functions, causing significant functional damage to cells. S-RPOC enables the selective disruption of mitochondria inside selected cells within a population and the simultaneous monitoring of cell responses.

[0089] In FIG. 5 panel A, we label the mitochondria of HeLa Kyto EB3-EGFP cells using MitoTracker Red and outline a specific cell for exclusive 405 nm laser interaction with its mitochondria. Using the MitoTracker signals and adjusting the intensity threshold allows the automatic selection of mitochondria within the outlined cell (FIG. 5 panel A). The application of a 240 pW 405 nm laser induces the MitoTracker signals to leak into the EB3-EGFP (FIG. 5 panel A). After treatment, the 405 nm laser interaction with mitochondria induces a significant loss in EGFP signals within the treated cell due to ROS generation. Such an EGFP signal decrease induced by light-mitochondria interaction is more pronounced than by the light-nuclei interaction as shown in prior figures. This suggests diverse light-induced ROS disruptions to cells through different organelles. On the contrary, the MitoTracker exhibits much less signal decrease (FIG. 5 panels A-C). This indicates the higher stability of MitoTracker dye upon the 405 nm laser interaction and ROS perturbation. However, the leakage of MitoTracker into the cytosol, evidenced by increased MitoTracker intensity outside mitochondria (FIG. 5 panel D), signifies mitochondrial damage and the loss of mitochondrial membrane potential caused by ROS. Additional discussion on the fluorescence signal changes of the EB3-EGFP HeLa cells can be found in Example 13.

[0090] S-RPOC allows monitoring of critical changes in cellular energy flux and mitochondria function through EGFP and MitoTracker signals during and after blue light treatment of mitochondria using a consistent laser dosage. This accomplishment cannot be achieved with conventional RPOC solely with the comparator circuit. This limitation arises from the MitoTracker signal enhancement by the 405 nm laser and dye leakage into the cytosol, leading to a disruptive effect on APXs and laser dosage coordination as illustrated in FIG. 5 panels E-F, FIGS. 18 panels A-B. S-RPOC addresses the challenge posed by the alteration of optical signals during optical treatment.

[0091] Regulating The Behaviors Of Specific Cells In Both Short-Term And Long-Term Scenarios

[0092] Apart from its ability to execute adaptive FRAP and FLIP and perturb mitochondria, S- RPOC holds the capability to govern various short-term and long-term behaviors for cells within a population through precise optical manipulation.

[0093] Together with photosensitive compounds, S-RPOC allows the inhibition of biochemical processes exclusively in selected cells within a population. For example, PST-1 is a photoswitchable microtubule inhibitor that can be activated by a blue laser and inactivated by a green laser. Utilizing HeLa Kyoto EB3-EGFP cells for visualizing microtubule polymerizations, where the EB3 protein binds to the plus end of microtubules during polymerization, we employed S-RPOC to delineate a single cell for PST-1 activation (FIG. 6 panels A-B). The utilization of a 405 nm laser at 6 pW exclusively illuminated the outlined single cell resulting in the inhibition of microtubule polymerization only for this cell within the population, as shown in FIG. 6 panels B-C. S-RPOC demonstrates the capability to regulate cell division by employing blue-light perturbation specifically targeting the centrosome during the mitotic phase. The H2B-mCherry EGFP -tubulin HeLa cells are used to visualize both chromosomes and microtubules. The 532 nm and 405 nm lasers, both at 240 pW, are used to interact with two centrosomes of the cell (FIG. 6 panel D). Using S-RPOC, the APXs are selected only at the centrosomes by thresholding tubulin EGFP signals. The exposure with 0.38 mJ 405 nm laser at the centrosome disrupts the microtubule network in the anaphase, while the exposure with 0.41 mJ 532 nm does not change the spindle morphology (FIG. 6 panel D). Such a targeted optical perturbation of centrosomes by the 405 nm laser alters cell mitosis and results in the formation of multi-nuclei cells (FIG. 6 panel D, FIG. 19 panels A-B, FIG. 20 panels A-D, and Example 14. The entire process, starting from the treatment to the development of multinucleation, spans 15 hours and is continuously monitored within the same FOV aided by a stage-top incubator.

[0094] Utilizing the stage-top incubator facilitates the maintenance of the cellular microenvironment, enabling the study of long-term cell responses. We conducted continuous monitoring of blue-light-induced cell perturbation, specifically targeting the nuclei, across different treatment durations and laser dosages. S-RPOC permits the selection of molecular targets or cellular compartments even without optical signals. By using HeLa Kyoto EB3-EGFP cells, we outline nuclei areas in cells that show as voids in the EB3-EGFP signals. The results reveal that 405 nm laser treatment with a 2.0 mJ dosage directed at the nuclei causes a reversible perturbation to the cells, whereas a 4.0 mJ laser dosage results in irreversible apoptosis of the cells, as depicted in FIG. 6 panels E-H, FIG. 21 panels A-D, and Example 15. This study assessed the safe laser dosage for illuminating nuclei and the laser dosage required to induce apoptosis specifically targeting nuclei. Moreover, targeted treatment of cancer cells within a fibroblast coculture system is conducted using a 405 nm laser (FIG. 22 panels A-D and Example 15), demonstrating the ability to selectively damage cancer cells while preserving fibroblasts.

[0095] Discussion

[0096] We developed an advanced HMI system dedicated to precise optical manipulation. It offers unparalleled freedom in the selection of molecular targets for opto-control. It allows the flexible selection of individual or multiple entities in cells, irrespective of their complexity and distribution. In addition, it offers simultaneous monitoring of cell responses during and after the manipulation, capturing both short-term and long-term changes using a broad array of readouts. S-RPOC stands out for its unparalleled versatility and capacity to manipulate solely the intended targets within live cells, leaving undesired locations unaffected. It can also concurrently establish multiple treatment conditions within the same FOV, thereby significantly reducing the overall experimental time. Being able to compare the different treatment conditions in a single FOV minimizes variations induced by different cell populations. With S-RPOC, we demonstrated adaptive photobleaching of fluorophores, a better understanding of protein dynamics, inhibition of tubulin polymerization in selected cells, ROS generation only in the mitochondria of the cell of interest, and the control of cell division by centrosome stimulation. These diverse applications exemplify the potential and broad impact of S-RPOC technology in biological sciences.

[0097] The spatial resolution and opto-control precision of S-RPOC are currently diffractionlimited, operating at a scale of about 300 nm. The laser scanning system has a fast-axis scan rate of 1 kHz, typically allowing a 0.2 to 1 Hz frame rate used in this study. The choice of optical manipulation lasers extends beyond the current 405 and 532 nm CW lasers. A broad range of lasers such as ultraviolet, near-infrared, mid-infrared, and ultrafast lasers can be applied in the future, allowing for a diverse range of opto-control applications. Beyond generating ROS and altering chemical states, these lasers can selectively induce heating, generate localized plasma, or perturb samples through various light-matter interaction mechanisms, enabling the study of cellular responses to a broad spectrum of stimuli directed at specific molecular targets.

[0098] Beyond live mammalian cells, this advanced HMI system has the potential to extend its applications to precise manipulation of chemical targets in multicellular systems and larger model organisms such as cultured organoids, brain slices, and Caenorhabditis elegans. In microbiology, S-RPOC can be applied to track and manipulate highly mobile bacteria cells of interest without affecting unwanted locations. The site-specific control of chemical targets, facilitated by S-RPOC, also opens new possibilities in directing stem cell differentiation and modulating embryo development. Furthermore, S-RPOC could have significant value in optogenetic systems by enabling precise control of synaptic firing in synapses associated with specific chemical species or states.

[0099] Example 2: The S-RPOC platform

[0100] The schematic of the S-RPOC optical configuration is illustrated in FIG. 1 panel A. This system incorporates five continuous-wave (CW) solid-state lasers (405 nm, 473 nm, 532 nm, 589 nm, 643 nm, CNI Laser). The output power of the lasers can be adjusted via variable neutral density filters (NDFs, 54-081, Edmund Optics) in their respective beam paths. The 405 nm and 532 nm lasers, designated for optical manipulation, are controlled separately by two acoustooptic modulators (AOMs, M1205-T80L-1 with 552F-2 driver, Isomet). After the AOMs, the first-order diffractions of these lasers are combined collinearly with other lasers using long-pass dichroic mirrors (DM, 69-887, 69-888, 69-889, 69-890, Edmund Optics). A TTL ‘ 1’ command deflects the action lasers to the target while a TTL ‘0’ command blocks the action laser. Microscope coverslips are placed after the AOMs to reflect a small portion of the first-order beams into photodiodes (PDA10A2, Thorlabs) for direct visualization of the APXs of the 405 nm and 532 nm lasers. The combined laser beams then pass through a polarizing beamsplitter (PBS251, Thorlabs) and a quarter-wave plate (10RP44-1, Newport) before entering a 2D galvo scanner (Satum-5, ScannerMax). This galvo scanner is integrated into an inverted microscope frame (1X73, Olympus) along with a 3D translational stage (Hl 17 with Motor Focus Drive and ProScan III system, Prior Technology) to accommodate the sample. A stage-top incubator (OTH- STXF-WSKMXCO2O2, Tokai Hit) is employed to maintain the CO2 level, temperature, and humidity, allowing prolonged opto-control and monitoring of cell responses within the same FOV. A camera (18MP, OMAX) is set up in transmission mode to capture bright-field images of cells. A water-dipping objective lens (UPlanSApo-S, 60X, NA = 1.20, Olympus) is utilized to focus laser beams onto the sample for RPOC and fluorescence imaging.

[0101] Example 3: The fluorescence signal detection operates in an epi-confocal mode

[0102] Fluorescence signals are directed towards three photomultiplier tubes (PMTs, H7422-40, Hamamatsu) via a polarizing beamsplitter and are separated by two long-pass dichroic beam splitters sequentially (FF552-DiO2, FF648-Di01, Semrock). Pinholes (P300HK, Thorlabs) are positioned at the sample conjugate plane, aligned with the focal position of the lenses before the PMTs. Three bandpass filters are used for the three PMTs for signal detection (FF01-509 / 22, Semrock, ET642 / 80m, Chroma Technology Corporation, and FF01-680 / 42, Semrock). The PMT output currents from all channels are converted to voltage and amplified using three preamplifiers (PMT4V3, Advanced Research Instruments Corporation). The amplified signals are then routed to the opto-control units, which play a pivotal role in determining APXs for optical manipulation. The control units consist of a Multi-I / O system (PCIe-6363 paired with BNC-2120, National Instruments), a comparator circuit, and interactive Lab VIEW software. Manual delineation of desired areas can be accomplished through an electronic sketch pad or mouse using the interactive software interface. The threshold for the TTL command can be adjusted both via the software and comparator circuit. The ultimate decision to activate the APXs can be made solely through the software, using the comparator circuit, or via logical calculations derived from both the software and the circuit. The lasers, optical detectors, and laser scanners are housed within a light-tight enclosure, allowing the system to operate in ambient room light.

[0103] To enable APX selection using the software, the synchronization between the Digital Output (DO) transmitting TTL commands and the Analog Output (AO) governing the galvo scanner is critical. In the software-only mode, an acquired fluorescence image from the sample or a provided mask input is transformed into a TTL time series. This series is directed in realtime to the AOM of either the 405 nm, the 532 nm laser, or both, during laser scanning. A TTL T command signifies an APX, precisely activating the laser at the targeted pixel, while a TTL ‘0’ command blocks the action laser. In the software + comparator circuit tandem mode, the TTL command selecting the area by the software is directed to one input of the comparator circuit. Simultaneously, the fluorescence signal from mobile targets is directed to another input of the comparator circuit to compare with the intensity threshold. This result is logically computed with the area selection TTL in real-time via AND, OR, or NOT functions (FIGS. 9A-B). The final outputs after logic computation control the AOMs. The digital AND function determines APXs on mobile targets within the selected area, while the digital NOT(software output)+AND allows users to assign APXs on mobile targets outside the selected area.

[0104] Example 4: The laser dosage calculation

[0105] When the comparator circuit is applied, the laser dosage is impacted by the changes in optical signals from the sample. To quantify the laser dosage received by a single cell, two photodiodes are applied to detect a small portion of 405 nm or 532 nm lasers deflected from the first-order output of the AOMs. Such detections generate APX images. To calculate laser dosage received by a single cell, manual delineation of an area encircling all the APXs in the selected cell is performed in Imaged. The laser dosage is computed as where A represents the size of a delineated area encompassing all APXs for a single cell, li is the average intensity within the delineated area for frame i, P denotes the laser power on the sample, T is the pixel dwell time, and N stands for the total number of frames for laser treatment. The unit of the laser dosage is joule (J). Lnax can be achieved by reducing the intensity threshold to zero using the comparator circuit. The image processing, region outlining, and quantifications are conducted using the built-in functions of Image-J (Fiji).

[0106] Alternatively, when only applying the RPOC software, both the APXs and laser dosage remain consistent throughout the treatment. Hence, the APX information can be obtained directly from the Multi-I / O. Except the above equation, another method to calculate the laser dose is using the equation where AAPX and Atotai are the total areas of the APXs and the entire image, respectively, calculated using the built-in functions of ImageJ. P denotes the laser power on the sample, T is the total time for each frame, and N is the total number of frames. Under the software-only condition, the laser dosage quantified using these two methods agrees with each other.

[0107] Example 5: Cell preparation

[0108] HeLa Kyoto EB3-EGFP cells and HeLa Kyoto EGFP-alpha-tubulin / H2B-mCherry cells are purchased from Biohippo. Pane.10.05 and CAF19 cells are obtained from Dr. Bumsoo Han’s research group at Purdue University. Cells are cultured in Dulbecco’s Modified Eagle Medium (DMEM, ATCC) with 10% fetal bovine serum (FBS, ATCC) and 1% penicillin / streptomycin (Thermofisher Scientific). The cells are seeded in 35 mm glass-bottom dishes (MatTek Life Sciences) with 2 mL culture medium and then incubated in a CO incubator set at 37 °C and 5% CO2 concentration. Upon reaching approximately 50-70% confluency, the cells are used for treatment or live-cell optical manipulation in the stage-top incubator. Example 6: Fluorescent microspheres and MitoTracker labeling of cells

[0109] The 1 pm fluorescence polystyrene microspheres (Lot No. 2051) are purchased from Phosphorex Inc. Their excitation and emission maximums are 460 nm and 500 nm, respectively. The microsphere solutions are deposited onto a coverslip and air-dried to form an evenly distributed single-layer microsphere fdm for photobleaching experiments.

[0110] HeLa Kyoto EB3-EGFP cells are first seeded in 35 mm glass-bottom dishes and cultured overnight to reach a confluency of around 50-70%. Mitotracker Red CMXRos is added to the culture medium at a final concentration of 200 nM. The cells are then incubated for 30 min at 37 °C and 5% CO2 concentration before S-RPOC.

[0111] Example 7: PST-1 preparation and treatment

[0112] PST-1, synthesized as described in Clark et al. (“Real-time precision opto-control of chemical processes in live cells”. Nat. Commun. 13, 4343 (2022)), is dissolved in dimethyl sulfoxide at a concentration of 2 mM to create the stock solution. Before cell treatment, the PST-1 stock solution undergoes exposure to a 532 nm laser (CNI laser) for 5 seconds to convert PST-1 into its trans-inactivated form. Following this, cells are treated with the inactivated PST-1 at a final concentration of 4 pM for 15 minutes before performing RPOC. The 405 nm laser in the S-RPOC system is utilized to selectively activate PST-1 exclusively at specified cells. The 405 nm laser power for PST-1 activation used in S-RPOC is 6 pW on the sample.

[0113] Example 8: The Comparator Circuit Box And Its Functions

[0114] The two-channel comparator circuit box used in this research is assembled with commonly available electronic components. It can operate independently or in tandem with the S-RPOC software, enabling real-time tracking and exclusive optical manipulation of mobile targets within the sample. The comparator circuit allows the system to work in an unsupervised configuration.

[0115] The ports and function switches of the comparator circuit box are shown in FIG. 8. It integrates two identical comparator circuits that can function separately or in combination. Each circuit includes an ‘analog input’ for receiving signals from an optical detector. The signals are compared with a manually adjustable preselected threshold via an adjustment knob or a digital threshold input. The resultant transistor-transistor logic (TTL) signal after comparison can output directly from the ‘direct comparator output’ ports or perform logic computation with the TTL signal generated from the other circuit. Switching between the two modes is facilitated by signal path switches. In the digital logic mode, the ‘direct comparator output’ can also serve as a signal input. In addition, a buffered analog signal output is provided for each comparator circuit to display the optical image. Digital logic functions such as AND, OR, and NOT are available for computations between the two channels. Digital NOT functions are positioned before the ‘direct comparator output’ and ‘digital logic output’, enabling inversions of active pixels (APXs) if needed. Two ‘digital logic output’ copies are available post-digital logic computation, allowing simultaneously commanding acousto-optic modulators (AOMs) and displaying APXs. Before each digital logic output, an ‘on’ and ‘off switch is available to facilitate sending TTL ‘ 1’ and ‘0’ signals to constantly turn on and off the connected AOMs.

[0116] When optical signals from two separate detection channels are linked to two analog inputs, the comparator circuit allows for selecting APXs based on criteria established separately by both comparators. Alternatively, when the optical signals from a single detection channel are divided and connected to both analog inputs, it permits the selection of APXs from the cellular compositions exhibiting optical signals within an intensity passband. Either comparator circuit can also function independently without interference from the other. Regardless of the conditions, outputs from the two comparator circuits can independently control two AOMs, commanding different action lasers.

[0117] In the scenario where the comparator circuit is used in tandem with the S-RPOC software, its connection is illustrated in FIG. 9A. One comparator circuit operates as previously described. The software TTL signal output can feed as the input from the ‘direct output channel’ of the other unconnected comparator circuit. The ‘signal path switch’ from the disconnected comparator circuit must be directed to the digital logic function. By selecting the AND function of any ‘digital logic output’, the comparator circuit will target mobile chemical entities in the area delineated by the S-RPOC software. Note that this tandem mode enables the treatment of a specific subset of targets with the treatment of another subset selected solely using the software (as shown in FIG. 3 panel G). To select mobile targets outside the delineated region, the connection is shown in FIG. 9B. Note that if the ‘invert’ function in the bottom comparator circuit is disabled (set to non-invert), the connection in FIG. 9B offers the same function as FIG. 9 A.

[0118] Example 9: S-RPOC Function Modes

[0119] S-RPOC can operate in two configurations: The software-only mode and the software + comparator circuit in tandem mode. The primary functional characteristics, as well as the advantages and disadvantages, are elucidated In FIG. 10.

[0120] In the former configuration, the comparator circuit box is bypassed, and the digital output from the Multi I / O system is directly used to command AOMs. This modality is ideal for controlling less mobile molecular targets or when the optical signals from the sample are significantly affected by the treatment. Typically, an optical image is acquired to aid in selecting regions of interest (ROI). Laser dosage within the region of interest (ROI), selected by the software based on the acquired image, remains consistent throughout the treatment process. Furthermore, it permits the input of a digital mask to guide laser interactions, as shown in FIG. 2 panels E-J. The software provides four digital outputs for users, allowing manual selection, digital mask input, or partial selection of the input mask separately for up to four action laser sources. Each channel includes independent intensity threshold functions aiding in the automatic selection of chemical targets based on optical signals. The mask input enables laser manipulation based on complex post-image analysis beyond intensity thresholding. For manual delineation of targets using a digital sketch pad or mouse, the selected area can be continuously added to each channel with distinct intensity thresholds for individual actions. These unique functionalities enable flexible user input, automated target selection based on chemical signals, laser dose control through intensity thresholding, and simultaneous control of multiple laser wavelengths. Various treatment conditions, including differences in area, laser dosage, and wavelength, can be simultaneously generated within a single field of view (FOV). This capability facilitates effective comparison of diverse treatment conditions, significantly enhancing RPOC throughput. Furthermore, it permits the study of interactions among different cells treated in different manners.

[0121] The tandem configuration is particularly suitable for manipulating highly mobile targets in cells. In contrast to conventional RPOC, the S-RPOC tandem mode allows for precise control of moving targets within an ROI with different lasers and comparisons of different treatment conditions in the same FOV. It additionally prevents imprecise laser treatments caused by slow target drifting, exemplified by the nucleus movement in FIG. 3 panels L-O. However, in this mode, the laser dosage can be affected by the changes in optical signal during treatment. One notable disadvantage is the difficulty in ensuring a constant laser dose over time. However, this adaptive and automatic APX adjustment permits dynamic photo interactions. The photobleaching or light-induced ROS generation can automatically stop when the optical signals fall below the intensity threshold established by the comparator circuit. For example, as demonstrated in FIG. 3 panels F-K, the interaction of the 405 nm laser with the nucleus automatically stops when the nucleus fluorescence signals drop to 50% of the initial signals. The tandem configuration is particularly advantageous for instantaneously tracking and modulating laser interactions for highly mobile targets. In FIG. 11, the EB3-EGFP signals from HeLa cells and the identified APXs utilizing the EB3 comets are presented for 532 nm and 405 nm lasers. Through the S-RPOC software, individual cells can be delineated, allowing lasers to exclusively engage with EB3 comets in the respective cells in real time. In FIG. 11, the low power of the action lasers in both treatment conditions prevents noticeable photobleaching or disruption of EB3 comets through targeted actions. This figure highlights the ability to selectively choose APXs on highly dynamic molecular targets using different lasers in tandem mode.

[0122] Example 10: S-RPQC Target Selection Using The Software And Intensity Thresholding S-RPOC permits adaptable APX selection through both manual delineation and automated target selection based on optical signals. FIG. 12 panels A-H give examples of delineating a single ROI involving the molecular targets. Encircling a cell nucleus expressing mCherry-H2B signals and adjusting the intensity threshold within the software allows for the selection of APX covering different areas of the nucleus. Lowering the threshold level enables the selection of a larger area within the nucleus. Furthermore, reducing the threshold levels to the minimum allows for the selection of APX covering the entire selected area. In addition, using the invert function, APXs can be selected on pixels having optical intensity below the threshold within the delineated area. The ‘In Range’ function permits the selection of APX for entities falling within any intensity range.

[0123] FIG. 13 panels A-B demonstrate the selection of multiple sub-FOV areas and the simultaneous adjusting of the intensity threshold of all these areas using the ‘Multi Selection’ function. FIG. 13 panels C-E illustrate the sequential addition of APXs to the same treatment channel by outlining different areas within the image. The intensity threshold for the sub-FOVs in each action is individually adjustable. FIG. 13 panel F shows selecting the APXs using the invert function after sequentially adding all the sub-FOVs. FIG. 13 panels G-H show choosing APXs using manual delineation and intensity thresholding for treatment using different laser lines (Line 4: 405 nm laser; Line 5: 532 nm laser). FIG. 13 panel I displays the treated areas by blue and green laser lines within the same FOV. FIG. 13 panel I shows switching the green laser treatment APXs using the ‘invert’ function.

[0124] These examples exemplify the flexibility of the human-machine interaction (HMI) system in selecting APXs for optical manipulation and adaptive control of laser doses at any target. The laser dosage can be controlled by adjusting the output power of the action lasers.

[0125] Example 11 : Laser Dosage Calculation For Software-Only And In Tandem With Comparator Circuit

[0126] The laser dose calculations differ when employing only the software as compared to incorporating the comparator circuit. Detailed equations and explanations can be found in the Methods section. When using only the software, the TTL command remains T throughout the entire APX, while outside APXs, the TTL command is consistently 'O'. Consequently, the APX intensity has only two values. FIG. 15 panels A-B illustrate the control of the green laser using this software-only mode for nucleus treatment.

[0127] When the comparator circuit is applied, either independently or in tandem mode, the optical signal is continuously compared with the threshold in real time. The fluctuations in optical signals may surpass the threshold for varying durations, resulting in diverse APX signals. In FIG. 15 panels A-B, the blue laser operates in tandem mode for nucleus treatment.

[0128] Example 12: mCherry-H2B Signal Changes During And After Treatment

[0129] In FIG. 4 panels A-H, S-RPOC facilitates comprehensive monitoring of the entire photointeraction process with 405 nm and 532 nm lasers, followed by FLIP and FRAP assessments. During treatment, distinct signal decay patterns are observed in response to the blue and green lasers, induced by ROS and photobleaching, respectively. Immediately after treatment, the region treated with the green laser starts signal recovery, attributed to FRAP, while the area treated with the blue laser demonstrates continuous signal decay similar to the untreated case. The normalized signal decay curves shown in FIG. 4 panel C exhibit similar decay rates for all untreated areas and the blue laser-treated area. However, in absolute values, the signal decay in the untreated areas is more pronounced compared to both the decay in the blue-treated area and the rise in the green-treated area (FIG. 16). The signal rise induced by the FRAP of mCherry in the treated region surpasses the impact of imaging laser photobleaching, whereas the combined ROS+FRAP process does not surpass the imaging laser photobleaching effect, leading to a slow signal decline. Strategies to mitigate the imaging laser photobleaching effect include reducing the excitation laser power, decreasing the pixel dwell time, and enlarging the FOV.

[0130] Example 13: Fluorescence Signal Changes Of EB3-EGFP Hela Cells

[0131] End-binding protein 3 (EB3) attaches to the plus end of microtubules. Therefore, EB3- EGFP signals, typically appearing as comets, allow us to visualize microtubule polymerization in live cells. The reduction in EB3-EGFP signals serves as a metric for quantifying the generation of ROS in cells. In FIG. 17 panels A-B, EB3-EGFP signals excited with 25 pW 473 nm laser and a pixel dwell time of 10 microseconds show a gradual signal decrease due to the photobleaching of EGFP molecules by the 473 nm laser.

[0132] When MitoTracker Red is utilized as a label, mitochondria can be simultaneously visualized with EB3-EGFP signals from separate PMT channels. As shown in FIG. 18 panel A, the fluorescence signals from both molecular entities are excited by 25 pW 473 nm and 10 pW 589 nm lasers. Negligible EB3-EGFP signal decay is detected after 90 seconds of laser illumination.

[0133] When RPOC is performed targeting mitochondria using the conventional approach by employing only the comparator circuit box, the leakage of MitoTracker fluorescence into the EGFP channel due to 405 nm laser interaction and the diffusion of MitoTracker outside mitochondria are detected after 20 seconds of treatment using the 25 pW 405 nm laser (FIG. 18 panel B). The MitoTracker leakage and signal enhancement disrupt APXs, as explained in FIG. 5 panel F. In comparison, when S-RPOC is applied to maintain a consistent laser dose specifically at the mitochondria, as demonstrated in FIG. 5 panel A, such APX disruptions do not occur. Example 14: Controlling Cell Division By The Treatment Of The Centrosomes Using A 405 Nm Laser

[0134] For regulating cell division by targeting centrosomes, S-RPOC is employed to select a confined area encompassing the centrosomes, indicated by the conjunctions of microtubule spindles. Fine adjustments to intensity thresholds are made to exclusively select APXs at the convergence points of the spindles. Employing an oversampling condition in which the laser spot size is larger than the pixel size, the treatment of centrosomes by lasers can be performed. In FIG. 6 panel D, FIG. 19 panels A-D, two centrosomes are designated for laser treatment, one with the 405 nm laser and the other with the 532 nm laser. The total laser dosage for the treatment is 0.38 mJ for the 405 nm laser and 0.41 mJ for the 532 nm laser. One hour after the treatment, a structural disruption is observed in microtubule spindles and chromosomes associated with the centrosome treated by the 405 nm laser. Following cell culture for 15 hours post-treatment (FIG. 6 panel D, FIG. 19 panels A-D), the formation of multi-nuclei was detected in the treated cells. This experiment is replicated as depicted in FIG. 20 panels A-D. The fluorescence signals and bright-field transmission illumination are used to monitor the cells at different time points. Future studies will include extended time-lapse studies to understand the prolonged viability and productivity of the treated cells.

[0135] Example 15: Time-Lapse Imaging Of EB3-EGFP Cells Treated With 405 Nm Lasers Solely In The Nuclei

[0136] FIG. 21 panels A-D display the time-lapse EB3-EGFP signals of HeLa cells when their nuclei are exclusively illuminated by a 405 nm laser. A longer treatment time results in a higher laser dosage within the nucleus, triggering apoptosis characterized by cell shrinkage and condensation (FIG. 21 panels A-B). Conversely, the treatment with a lower laser dosage does not prompt a comparable apoptotic response (FIG. 21 panels C-D). This investigation not only exemplifies the capability of S-RPOC in monitoring prolonged cellular reactions following precise label-free organelle perturbation but also assesses the 405 nm laser dosage associated with nuclear treatment capable of inducing apoptosis in HeLa cells.

[0137] Example 16: Treatment Of A Single Cancer Cell In A Co-Culture System

[0138] RPOC allows for the selective treatment of cancer cells in co-culture. In this work, pancreatic cancer cells (Pane 10.05) expressing tetramethylrhodamine (TRITC) and cancer- associated fibroblasts (CAF 19) expressing fluorescein isothiocyanate (FITC) are co-cultured for selective treatment. Based on different fluorescence signals of cancer cells and fibroblasts, we outlined one cancer cell and selectively directed a 405 nm laser to interact with it without affecting a neighboring fibroblast (FIG. 22 panel A). The laser power for treatment is 600 pW. A substantial decrease in fluorescence signals is observed during treatment in the targeted cancer cell (FIG. 22 panel B). The adjacent cancer and fibroblast cells exhibit a less significant reduction in signals. Long-term imaging of cell responses reveals a recovery of fluorescent signals in the fibroblast but an irreversible loss of signals in the treated cancer cell (FIG. 22 panel C). Moreover, exposure to blue light prompts detachment of the treated cancer cell from its adjacent cancer cells, with less disruption for the CAF 19 (FIG. 22 panels A and D). These findings demonstrate distinct responses of various neighboring cells within a co-culture system to the blue-light-treated cancer cells. Further investigations into cell interactions in co-culture systems after RPOC treatment will be conducted in the future.

Claims

What is claimed is:

1. A software-assisted laser- scanning imaging and optical manipulation system, the system comprising: a laser- scanning apparatus; one or more optic modulators; and a processor operably associated with the laser-scanning apparatus and the one or more optic modulators, wherein the processor is configured to command coupling of one or more laser beams of the laser-scanning apparatus via either one or more optic modulators or analog signals; and operating the system to selectively interact with selected locations on a sample that is being interrogated by the system.

2. The system of claim 1, wherein laser beams of the laser-scanning apparatus are only activated at desired areas of interest without affecting unwanted locations.

3. The system of claim 2, wherein pixels where the laser beams are activated are called active pixels (APXs) and chemical information from the sample can be simultaneously monitored for APX selection and cell response measurement.

4. The system of claim 3, where the laser beams activated at APXs can be used for optical manipulation or region-of-interest (ROI) imaging.

5. The system of claim 4, wherein the laser beams for optical manipulation or ROI imaging can be selected from UV to IR wavelengths, and from continuous wave to pulsed lasers.

6. The system of claim 1, wherein the system can be operated to accomplish at least one of the following: selectively photobleach targeted fluorescent molecules; induce reactive oxygen species at a gated area on the sample; apply any input mask for optical manipulation; selectively inhibit photoswitchable inhibitors; perform ROI imaging to reduce phototoxicity; or achieve flexible FRAP or FLIP to study molecular diffusion at selected locations.

7. The system of claim 1, wherein the processor is further configured to allow for simultaneously controlling multiple laser wavelengths and generating any desired optical pattern for imaging and optical manipulation.

8. The system of claim 1, wherein the process is configured to control a subset of the lasers by the one or more optic modulators separately for fast switching from ‘on’ and ‘off states.

9. The system of claim 8, wherein the one or more optic modulators are controlled by TTL output selected by the processor that allows users to input any mask that determines where desired lasers are turned on the sample.

10. The system of claim 8, wherein the processor can also be used in combination with a comparator circuit system that allows real-time determination of laser-interacting pixels, such that laser beams that perform optical manipulation or selective imaging are only turned on at desired pixels known, which are selected by the processor or the comparator circuit, or both.

11. The system of claim 1, wherein the processor controls imaging and data acquisition by synchronizing multiple channels of Analog Output (AO), Analog Input (Al) and Digital Output (DO).

12. The system of claim 11, wherein two AO channels are used to direct illumination sources to specific X / Y positions on a sample via controlling two galvo mirrors.

13. The system of claim 12, wherein processor allows users to configure image size and location in an X / Y plain, image resolution (# pixels), measurements acquired and averaged per pixel, dwell time in each pixel, wherein one or more of these parameters are used to calculate precise mirror positions to locate an illumination source on the sample during a scan and thereby synchronize all other AO, Al and DO channels.

14. The system of claim 13, wherein X and Y parameters are independent of one another, thereby allowing images with any desired dimensions and resolution.

15. The system of claim 14, wherein the system comprises additional AO channels that are available to control other external devices / events to manipulate sample conditions.

16. The system of claim 14, wherein the system is configured to control intensity / phase of a laser illumination source or voltage applied to an electrical grid.

17. The system of claim 16, wherein the system is configured such that a user can create a desired pattern of events / manipulations with an analog mask that is analogous to a digital mask already in use to control the AOMs.

18. The system of claim 13, wherein the system comprises at least 16 Al channels such that the system has the capability to acquire 8 channels of data.

19. The system of claim 13, wherein current digital outputs have three predefined functions and five user-definable values.

20. The system of claim 19, wherein the predefined DO lines indicate when an imaging beam is in-frame, has changed pixels, and has started a new line in an image, wherein these signals are used to synchronize external events / hardware with repeating scan events.

21. A method for analyzing a sample, the method comprising: providing a software-assisted laser-scanning imaging and optical manipulation system that comprises a laser-scanning apparatus; one or more optic modulators; and a processor operably associated with the laser-scanning apparatus and the one or more optic modulators, wherein the processor is configured to command coupling of one or more laser beams of the laser-scanning apparatus via the one or more optic modulators; and operating the system to selectively interact with selected locations on a sample that is being interrogated by the system, thereby analyzing the sample.

22. The method of claim 21, wherein laser beams of the laser-scanning apparatus are only activated at desired areas of interest without affecting unwanted locations.

23. The method of claim 22, wherein pixels where the laser beams are activated are called active pixels (APXs) and chemical information from the sample can be simultaneously monitored for APX selection and cell response measurement.

24. The method of claim 23, where the laser beams activated at APXs can be used for optical manipulation or region-of-interest (ROI) imaging.

25. The method of claim 24, wherein the laser beams for optical manipulation or ROI imaging can be selected from UV to IR wavelengths, and from continuous wave to pulsed lasers.

26. The method of claim 21, wherein the system can be operated to accomplish at least one of the following: selectively photobleach targeted fluorescent molecules; induce reactive oxygen species at a gated area on the sample; apply any input mask for optical manipulation; selectively inhibit photoswitchable inhibitors; perform ROI imaging to reduce phototoxicity; achieve flexible FRAP or FLIP to study molecular diffusion at selected locations, or control cell fate.

27. The method of claim 21, wherein the processor is further configured to allow for simultaneously controlling multiple laser wavelengths and generating any desired optical pattern for imaging and optical manipulation.

28. The method of claim 21, wherein the process is configured to control a subset of the lasers by the one or more optic modulators separately for fast switching from ‘on’ and ‘off states.

29. The method of claim 28, wherein the one or more optic modulators are controlled by TTL output selected by the processor that allows users to input any mask that determines where desired lasers are turned on the sample.

30. The method of claim 28, wherein the processor can also be used in combination with a comparator circuit system that allows real-time determination of laser-interacting pixels, such that laser beams that perform optical manipulation or selective imaging are only turned on at desired pixels known, which are selected by the processor or the comparator circuit, or both.

Images