A colony selection system and method based on light manipulation technology

Abstract

This invention discloses a colony selection system and method based on optical manipulation technology, comprising: a chip platform with a chip placement section for placing array-type chamber chips; a well plate platform with a well plate placement section for placing well plates; a photothermal module for emitting photothermal light and irradiating photosensitive heating elements on one or more chambers of the array-type chamber chips; an imaging module for identifying the state of colonies within the chambers of the array-type chamber chips within the current field of view; and an automatic acquisition and control module for identifying and calibrating colonies within the chambers of the array-type chamber chips through the identification by the imaging module, thereby controlling the photothermal module to perform photothermal heating on the photosensitive heating elements on the identified and calibrated chambers, and adjusting the position of the chip platform and / or the well plate platform to transport target colonies from the array-type chamber chips to the collection chambers of the well plate. This system allows for target colony selection in both synchronous and asynchronous modes, thereby greatly improving the selection efficiency of target colonies.

Description

Technical Field

[0001] This invention belongs to the field of colony selection technology, specifically relating to a colony selection system and method based on light manipulation technology. Background Technology

[0002] Currently, in biological experiments such as bioengineering and microbial culture, the selection of microbial colonies is mainly done manually. With the increasing applications and rapid development of synthetic biology, microbial colony selection experiments often require the selection of thousands or even tens of thousands of cloned colonies. Microbial colony selection faces the following challenges:

[0003] 1. The workload is large, and manual selection is inefficient;

[0004] 2. Subjective judgment in selecting clonal colonies can easily lead to the omission of target clones.

[0005] Therefore, manually picking microbial colonies is gradually becoming insufficient to meet the requirements of high-throughput experiments.

[0006] With the development of biotechnology, traditional manual picking of microbial colonies has gradually been replaced. Microbial colony picking instruments have replaced manual picking with automated methods, significantly improving both efficiency and accuracy. Currently, most microbial colony picking instruments on the market use a 96-needle picking head that moves along the X, Y, and Z axes. First, based on machine vision technology, a camera identifies and locates the microbial colonies in the petri dish. The 96-needle picking head picks up each colony individually and then inoculates them into the inoculation plate in one go. Finally, the 96 needles are cleaned and sterilized. For details, please refer to Chinese invention patents CN113604348A, CN108641903A, and CN113667588A.

[0007] The main drawbacks of the above scheme are:

[0008] 1. Contamination issue: The metal picking needle is reused during the work process, and even if there is a sterilization process, there is still a risk of incomplete sterilization, which reduces the reliability and scientific validity of the experimental results.

[0009] 2. The structure is complex and the cost is high. The 96-pin picking head is a key component. It needs to achieve high-precision movement in the X and Y axes, and each of the 96 pins needs to move independently in the Z (up and down) direction. In addition, the 96 pins need to be designed to withstand high temperature and have high rigidity. The structure design is complex and the cost is high.

[0010] 3. Increased energy consumption, mainly due to the high-temperature sterilization of the 96 needles and other sterilization processes, which consume a lot of energy; the 96-needle picking method uses a common culture dish as the source plate, but achieving high-throughput picking involves multiple culture dishes, or even more than a dozen, to complete one picking experiment, resulting in a large demand for consumables and increased energy consumption. Summary of the Invention

[0011] Details of one or more embodiments of the present invention are set forth in the following drawings and description to make other features, objects and advantages of the present application more readily apparent.

[0012] Based on existing array-type chamber chips, this invention designs a fully automatic colony selection system and method based on light manipulation technology, which fundamentally avoids the drawbacks of traditional manual colony selectors and can greatly improve sorting efficiency.

[0013] This invention discloses a colony selection system based on light manipulation technology, comprising:

[0014] The chip platform is equipped with a chip placement section for placing array-type chamber chips;

[0015] The orifice plate platform is equipped with an orifice plate placement section for placing orifice plates;

[0016] Photothermal module, used to emit photothermal light;

[0017] The optical-thermal path consists of one unit and illuminates the photosensitive heating element on one of the corresponding array-type cavity chips.

[0018] or

[0019] The number of photothermal optical paths is multiple and they irradiate in parallel onto the photosensitive heating elements in multiple corresponding array-type cavity chips;

[0020] The imaging module is used to identify the state of bacterial colonies within the chambers of the array-type chamber chip within the current field of view;

[0021] The automatic acquisition and control module identifies and calibrates the colonies in the chambers of the array-type chamber chip through the recognition of the imaging module, so as to control the photothermal module to perform photothermal heating on the photosensitive heating part on the identification and calibration chamber, and adjust the position of the chip platform and / or the well plate platform so that the target colonies are transported from the array-type chamber chip to the collection chamber of the well plate.

[0022] In some embodiments, the horizontal height of the chip placement portion is higher than the horizontal height of the perforated plate placement portion.

[0023] In some implementations, it also includes:

[0024] A fluorescence detection module is used to emit fluorescence light; the emitting ends of the photothermal module, the imaging module, and the fluorescence detection module share the same microscope objective and are fixedly installed above the chip placement part.

[0025] In some embodiments, the automatic data acquisition control module includes:

[0026] The detection and identification module is used to detect bacterial colonies within the chamber; the detection and identification module is one or more of the following: fluorescence detection module, Raman detection module, laser confocal detection module, and mass spectrometry detection module.

[0027] In some embodiments, the photothermal module includes a colony picking light source and a beam shaping module; the colony picking light source is a laser light source or an LED light source; the beam shaping module is used to shape the photothermal optical path irradiating the photosensitive heating part into a single beam or multiple beams and to reduce the beam size.

[0028] In some embodiments, the photothermal module includes: a laser, an optical beam expander, and a collimating lens;

[0029] The imaging module includes: a relay lens and an image acquisition unit;

[0030] The laser is used to emit photothermal light, and the photothermal light is expanded by the optical beam expanding system, then parallel light is obtained by the collimating lens, and finally reaches the microscope objective by reflection by the dichroic mirror.

[0031] The imaging light path acquired by the microscope objective is transmitted to the relay lens through the dichroic mirror and then to the image acquisition device.

[0032] The fluorescence detection module is positioned between the dichroic mirror and the relay lens.

[0033] In some embodiments, the photothermal module includes: a laser, a half-glass slide, an optical beam expander, a collimating lens, a spatial light modulator, a quarter-glass slide, and a polarizing beam splitter;

[0034] The imaging module includes: a relay lens and an image acquisition unit;

[0035] The laser is used to emit photothermal light, and the polarization direction of the photothermal light is changed by adjusting the rotation angle of the half-glass slide. The photothermal light is then expanded by the optical beam expander system, and parallel light is obtained by the collimating lens and transmitted to the polarizing beam splitter. The reflection of the polarizing beam splitter causes the light to enter the quarter-glass slide and the spatial light modulator in sequence. The spatial light modulator loads a hologram formed by the position coordinates of multiple target colonies in the chamber, and then modulates multiple photothermal light paths according to the hologram. The modulated multiple photothermal light paths are then transmitted through the quarter-glass slide and the polarizing beam splitter in sequence, reflected by the dichroic mirror, and then enter the microscope objective to form an array of multiple photothermal light paths.

[0036] The imaging light path acquired by the microscope objective is transmitted to the relay lens through the dichroic mirror and then to the image acquisition device.

[0037] The fluorescence detection module is positioned between the dichroic mirror and the relay lens.

[0038] In some implementations, it also includes:

[0039] The detection module is used to identify whether the droplets discharged from the array chamber chip contain target colonies, and to determine whether the end of the droplets discharged from the array chamber chip is aligned with the collection hole corresponding to the well plate.

[0040] In some embodiments, the detection module includes:

[0041] A vision camera is located below the perforated plate platform;

[0042] A beam splitter is positioned above the vision camera;

[0043] An illumination source and a reflector are respectively disposed on both sides of the beam splitter; the illumination source emits a light beam and the beam is split into reflected light and transmitted light by the beam splitter; the reflected light is used to illuminate the collection hole corresponding to the orifice plate to determine the alignment status of the end of the droplet discharged from the array chamber chip with the corresponding collection hole of the orifice plate; the transmitted light is used by the reflector to illuminate the array chamber chip for imaging.

[0044] In some embodiments, the chip platform and the well plate platform move horizontally along the XY axis and move relative to each other vertically; the vision camera is used to identify whether the end of the discharged droplet contains a target colony, and the target colony is collected by controlling the relative vertical movement of the chip platform and the well plate platform.

[0045] In some implementations, it also includes:

[0046] A chip export tube is used to connect to the export port in an array-type chamber chip; the chip export tube is vertically disposed at the export port or fixedly disposed between the chip platform and the orifice plate platform and connected to the export port through a flexible connecting tube.

[0047] The present invention also discloses a selection method for a colony selection system based on light manipulation technology according to any one of the above embodiments, wherein the method for collecting target colonies includes: synchronous mode and asynchronous mode;

[0048] The synchronous mode acquires information about target colonies in the array-type chamber chip within the current field of view, plans the picking path of the target colonies according to a greedy algorithm, and collects them according to the planned path until the picking of target colonies in the entire array-type chamber chip is completed.

[0049] In asynchronous mode, image acquisition of the array-type chamber chip is completed, and colonies are identified and labeled based on the image acquisition. All labeled target colonies are then selected sequentially using a greedy algorithm for path planning.

[0050] In some implementations, a chip tilt correction method is also included: acquiring image information of the array-type chamber chip after placement and identifying each chamber; forming a data point series by using the center points of each chamber located on the same horizontal baseline, and obtaining the corresponding two-dimensional array coordinates; solving for the average slope S_lope based on the connection fitting between adjacent arrays on each horizontal baseline, where S_lope has positive and negative values; solving for K based on S_lope, where K is the distance of the overall tilt of each chamber; and moving the chip platform according to the K value, where a positive K value moves it upwards and a negative K value moves it downwards.

[0051] In some embodiments, the method by which the photothermal module irradiates the photosensitive heating element on the cavity of the array-type cavity chip is as follows:

[0052] By adjusting the position of the chip platform, a single photothermal optical path formed by the laser is irradiated onto the chamber containing the target colony, and the target colony is extracted by photothermal heating.

[0053] or

[0054] By expanding and aligning the laser beam, it enters a spatial light modulator. The spatial light modulator forms multiple photothermal paths within the current field of view based on the image of the chamber positions containing the target colonies. These paths are then irradiated onto the respective chambers containing the target colonies. Since the distances from the exit to each chamber within the current field of view are different, all target colonies are sequentially extracted by photothermal heating.

[0055] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0056] 1. This device changes the traditional needle-picking method for selecting colonies. It utilizes the structure of an array-type chamber chip and is based on the photothermal effect to achieve contactless extraction of target colonies through photothermal means. At the same time, it can also utilize the chamber characteristics within the structure of the array-type chamber chip itself to achieve online culture of colonies, thus ensuring the success rate and efficiency of colony selection.

[0057] 2. To ensure the accuracy of the picking process, a corresponding detection module has been added. It can not only detect whether the well positions are aligned, but also detect whether the exported droplets contain target colonies, thereby ensuring the accuracy of the picking process.

[0058] 3. Based on the above device structure, two extraction methods, synchronous and asynchronous, are formed to adapt to different working conditions and thus improve the efficiency of colony picking.

[0059] 4. The selected photothermal modules can be selected sequentially in a single operation or in multiple operations, depending on whether they can achieve a single photothermal path or multiple photothermal paths. Attached Figure Description

[0060] The accompanying drawings, which are provided to further illustrate the invention and constitute a part of this invention, are illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention.

[0061] Figure 1 This is a schematic diagram of the structure of the present invention.

[0062] Figure 2 This is a schematic diagram of the structure of the multi-beam optical path module of the present invention.

[0063] Figure 3 This is a schematic diagram of the detection module of the present invention.

[0064] Figure 4 This is a schematic diagram of the structure of the array-type chamber chip of the present invention.

[0065] Figure 5 This is a schematic diagram of the alignment imaging structure of the machine vision camera of the present invention.

[0066] Figure 6 This is a schematic diagram of the structure of the machine vision camera for droplet detection according to the present invention.

[0067] Figure Descriptions: Optical Path Module 1; Fluorescence Detection Module 11; Photothermal Module 12; Laser 1201; Optical Shutter 1202; Half-glass Slide 1203; Mirror 1 1204; Optical Beam Expander System 1205; Collimating Lens 1206; Mirror 2 1207; Spatial Light Modulator 1208; Quarter-glass Slide 1209; Polarizing Beam Splitter 1210; Mirror 3 1211; Lens 1 1212; Mirror 3 1213; Lens 2 1214 Imaging module 13, reflector 5 1301, microscope objective 1302, relay lens 1303, image acquisition unit 1304; dichroic mirror 14; detection module 2, illumination source 21, collimation and homogenization module 22, machine vision camera 23, telecentric lens 24, beam splitter prism 25, reflector 26, orifice plate 27, collection hole 28, droplet 29; chip platform 3, flexible connecting tube 30, chip fixture 31; orifice plate platform 4, orifice plate fixture 41; pumping module 5. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments provided by this invention without inventive effort are within the scope of protection of this invention.

[0069] Obviously, the accompanying drawings described below are merely some examples or embodiments of the present invention. Those skilled in the art can apply the present invention to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this invention, modifications to design, manufacturing, or production based on the technical content disclosed in this invention are merely conventional technical means and should not be construed as insufficient disclosure of the present invention.

[0070] In this invention, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this invention may be combined with other embodiments without conflict.

[0071] This invention discloses a colony selection system based on light manipulation technology, comprising:

[0072] Chip platform 3 is provided with a chip placement section for placing array-type chamber chips;

[0073] The orifice plate platform 4 is provided with an orifice plate placement section for placing the orifice plate 27;

[0074] The photothermal module 12 is used to emit a photothermal optical path; the number of the photothermal optical path is one and it irradiates the photosensitive heating part on one of the corresponding multiple array-type cavity chips, or the number of the photothermal optical path is multiple and they irradiate the photosensitive heating parts on multiple chambers of the corresponding multiple array-type cavity chips in parallel.

[0075] Imaging module 13 is used to identify the state of bacterial colonies within the chambers of the array-type chamber chip within the current field of view;

[0076] The automatic acquisition and control module identifies and calibrates the colonies in the chamber of the array-type chamber chip through the identification of the imaging module 13, so as to control the photothermal module 12 to perform photothermal heating on the photosensitive heating part on the identification and calibration chamber, and adjust the position of the chip platform 3 and / or the well plate platform 4 so that the target colonies are transported from the array-type chamber chip to the collection chamber of the well plate.

[0077] Correspondingly, a pumping module 5 can also be integrated for sample / liquid injection into array-type chamber chips.

[0078] Among them, the structure of the array-type chamber chip is as follows: Figure 4 As shown, the chip structure is the existing chip structure CN112871227B, so it will not be described in detail further. This device is a supporting system developed and designed based on this chip structure. This invention innovatively breaks the needle-picking colony selection mode. By injecting the colony sample into the chamber of the array-type chamber chip, contactless picking is achieved through photothermal stimulation, thereby completing the picking of the target colony.

[0079] In some embodiments, the horizontal height of the chip placement section is higher than that of the orifice plate placement section. The chip placement section and the orifice plate placement section are respectively provided with corresponding chip clamps 31 and orifice plate clamps 41 for clamping and fixing the array-type chamber chip and the orifice plate 27. The design of the vertical height difference facilitates droplet discharge while ensuring the compactness of the device structure.

[0080] In some embodiments, it also includes:

[0081] The fluorescence detection module 11 is used to emit fluorescence light; the emitting ends of the photothermal module 12, the imaging module 13, and the fluorescence detection module 11 share the same microscope objective 1302 and are fixedly installed above the chip placement part.

[0082] Multiple fluorescent light cubes can be set within the fluorescence detection module 11 for different switching.

[0083] In some embodiments, the automatic data acquisition and control module includes:

[0084] The detection and identification module is used to detect bacterial colonies within the chamber; the detection and identification module is one or more of the following: fluorescence detection module, Raman detection module, laser confocal detection module, and mass spectrometry detection module.

[0085] In some embodiments, the photothermal module includes a colony picking light source and a beam shaping module; the colony picking light source is a laser light source or an LED light source; the beam shaping module is used to shape the photothermal optical path irradiating the photosensitive heating part into a single beam or multiple beams and to reduce the beam size.

[0086] There are two structural forms for the photothermal module 12:

[0087] The first type is a more conventional pipeline setup, which uses a laser to generate a laser beam to form a single-beam photothermal pipeline, so it will not be described in detail here.

[0088] The second type is as follows Figure 2 As shown, laser 1201 emits laser light with a wavelength in the near-infrared band. The laser light passes through and is blocked by an optical shutter 1202. The laser light continues to propagate to a half-glass slide 1203. By rotating the angle of the half-glass slide 1203, the polarization direction of the laser light can be changed. The laser light with a changed polarization direction reaches the reflector 1204 and then the optical beam expander 1205, which expands the laser beam. The expanded laser light passes through the collimating lens 1206 to obtain parallel light. Thus, parallel light with a changed polarization direction and expanded by 5-10 times is obtained. This magnification is determined by the combination of the optical beam expander 1205 and the collimating lens 1206. The expanded parallel light is reflected by mirror 1207 and enters polarizing beam splitter 1210. Polarizing beam splitter 1210 and half-glass slide 1203 work together to achieve the ratio of laser transmission and reflection. Rotating and adjusting the angle of half-glass slide 1203 can maximize the reflection of laser light in polarizing beam splitter 1210. For example, if a phase image of a multi-spot pattern is loaded into spatial light modulator 1208, then the laser emitted from spatial light modulator 1208 is the phase light field of the multi-spot pattern. The phase light field of the multi-spot pattern corresponds to the multiple photothermal optical paths that need to be defined. The phase light field of the multi-spot pattern passes through polarizing beam splitter 1210 again, and the transmitted beam is the desired beam. The transmitted beam enters the 4f system through mirror 1211. The 4f system consists of lens 1212 and lens 1214, which realizes beam contraction. The beam contraction magnification is determined by the entrance pupil of the microscope objective. The contracted beam is introduced into the imaging optical path through dichroic mirror 14.

[0089] The imaging module 13 is mainly used for microscopic imaging, specifically bright-field imaging and fluorescence imaging. Fluorescence imaging requires the cooperation of a fluorescence detection module, which mainly includes a reflector 1301, a microscope objective 1302, a relay lens 1303, and an image acquisition unit 1304. The fluorescence detection module 11 is positioned between a dichroic mirror 14 and a relay lens 1303. The dichroic mirror 14 is a high-reflection, low-pass mirror, which can reflect long-wavelength light beams while allowing short-wavelength light to pass through. Therefore, the long-wavelength light beam in the photothermal optical path is reflected into the reflector 1301 and then into the microscope objective 1302. The microscope objective 1302 focuses the photothermal optical path onto the array-type cavity chip.

[0090] In some embodiments, it also includes:

[0091] The detection module 2 is used to identify whether the droplets 29 discharged from the array chamber chip contain target colonies, and to determine whether the end of the droplets discharged from the array chamber chip is aligned with the collection hole corresponding to the orifice plate 27.

[0092] Specifically, the detection module 2 includes:

[0093] A vision camera 23 is located below the perforated plate platform 4;

[0094] The beam splitter 25 is positioned above the visual camera 23;

[0095] An illumination source 21 and a reflector 26 are respectively disposed on both sides of the beam splitter 25. The illumination source 21 emits a light beam, which is split into reflected light and transmitted light by the beam splitter 25. The reflected light is used to illuminate the collection hole corresponding to the orifice plate 27 to determine the alignment status of the end of the droplet discharged from the array chamber chip with the corresponding collection hole of the orifice plate 27. The transmitted light is used by the reflector 26 to illuminate the array chamber chip for imaging.

[0096] The chip platform 3 and the well plate platform 4 move horizontally along the XY axis and move relative to each other vertically. The vision camera 23 is used to identify whether the end of the discharged droplet contains a target colony. The target colony is collected by controlling the relative vertical movement of the chip platform 3 and the well plate platform 4.

[0097] A chip export tube is used to connect to the export port in the array-type chamber chip; the chip export tube is vertically disposed at the export port or fixedly disposed between the chip platform 3 and the orifice plate platform 4 and connected to the export port through a flexible connecting tube 30.

[0098] The vertically fixed method means that the chip outlet tube and the array-type chamber chip are set to move together. Therefore, it is necessary to continuously adjust the collection hole 28 on the corresponding orifice plate 27 according to the position of the chip outlet tube. The fixed setting method keeps the position of the chip outlet tube fixed, and the chip platform 3 can be compensated by the displacement of the flexible connecting tube 30 during the movement, which makes the adjustment more convenient.

[0099] The main functions of detection module 2 are as follows:

[0100] 1. Detect the coaxiality between the end of the chip outlet tube and any initial hole on the via plate 27;

[0101] 2. Detect whether the target colonies are encapsulated in the oil phase droplets;

[0102] 3. Integrated design: This part not only has detection function, but also provides lighting function, providing illumination for consumable chips and detection targets.

[0103] The illumination source 21 is a white LED lamp that emits white light. After passing through the collimation and homogenization module 22, the light source is transformed into a parallel beam with uniform light energy distribution. The beam continues to propagate and reaches the beam splitter prism, which splits the beam into reflected and transmitted light. The transmission-to-reflection ratio of the beam splitter prism is 8:2 (or another ratio). The transmitted light is reflected by the reflector and enters the illumination of the consumable chip, coaxial with the microscope objective 1302, and the microscope objective performs bright-field imaging. The reflected light is used to illuminate the collection hole 28 (such as the A1 hole) of the orifice plate 27, the droplet 29, and the end of the chip outlet tube. The orifice plate 27 is mounted on a three-axis displacement platform (orifice plate platform 4), and the orifice plate can be moved up, down, left, right, forward, and backward by the movement of the three-axis displacement platform. The image is then formed by the telecentric lens 24, which features high resolution, ultra-wide depth of field, ultra-low distortion, and a unique parallel light design. The image is then displayed on a machine vision camera for inspection purposes.

[0104] The imaging effect of the chip outlet tube end and the A1 hole of the perforated plate 27 on the machine vision camera is as follows: Figure 5 As shown, the black dot represents the center position of hole A1, and the gray dot represents the position of the end of the chip outlet tube. When the target position is found to be inconsistent with the actual position, the position difference can be calculated. Specifically, let the black dot position be (x0, y0) and the actual position be (xy). Then Δx = x - x0; Δy = y - y0. The XY axes of the hole plate 27 then move according to the difference, making the target position coincide with the actual position. This completes the coaxial alignment of hole A1 and the end of the chip outlet tube.

[0105] The state of droplet 29 includes two types: uncoated target droplet and coated target droplet. Uncoated target droplet is oil phase buffer solution, while coated target droplet is oil phase buffer solution containing sample droplet. The imaging of the two types of droplets in machine vision camera 23 is also different. When the imaging signal is coated target droplet, the recognition algorithm gives a feedback signal to start the droplet collection command, that is, the well plate 27 performs up and down movement.

[0106] Based on the system in the above embodiments, a corresponding selection method is also set, specifically including two modes: synchronous mode and asynchronous mode.

[0107] The synchronous mode acquires information about target colonies in the array-type chamber chip within the current field of view, plans the picking path of the target colonies according to a greedy algorithm, and collects them according to the planned path until the picking of target colonies in the entire array-type chamber chip is completed.

[0108] In asynchronous mode, image acquisition of the array-type chamber chip is completed, and colonies are identified and labeled based on the image acquisition. All labeled target colonies are then selected sequentially using a greedy algorithm for path planning.

[0109] In the above process, one step requires calibrating the position of the array-type chamber chips. Since they cannot be guaranteed to be parallel each time they are placed, they will be tilted in the microscopic field. Therefore, a corresponding chip tilt correction method is set up. Specifically, image information of the array-type chamber chips after placement is acquired and each chamber is identified. Data points are formed by using the center points of each chamber located on the same horizontal baseline, and the corresponding two-dimensional array coordinates are obtained. The average slope S_lope is solved by fitting the connection between adjacent arrays on each horizontal baseline. S_lope can be positive or negative. K is calculated based on S_lope. K is the distance of the overall tilt of each chamber. The chip platform 3 is moved according to the K value. A positive K value moves it upwards, and a negative K value moves it downwards.

[0110] Combining the aforementioned two different structural forms of the photothermal module 12, there are two corresponding heating methods.

[0111] Single photothermal optical path: By adjusting the position of the chip platform 3, the single photothermal optical path formed by the laser 1201 is irradiated onto the chamber containing the target colony, and the target colony is extracted by photothermal heating.

[0112] Multiple photothermal optical paths: The beam formed by the laser 1201 is expanded and parallelized, and then enters the spatial light modulator 1208. The spatial light modulator 1208 forms multiple photothermal optical paths within the current field of view based on the image of the location of the chambers containing the target colonies, and illuminates each corresponding chamber containing the target colonies. Since the distance of each chamber from the outlet is different within the current field of view, all target colonies are sequentially extracted by photothermal heating.

[0113] Its working principle is as follows:

[0114] Photothermal effects, also known as light-matter interaction, refer to the phenomenon where a light beam irradiates the surface of an object, causing localized heat generation on that surface. When focused near-infrared light irradiates a microcavity, localized heat is generated, causing the liquid inside the cavity to expand. This expansion forces droplets out of the cavity through a channel containing flowing liquid, which in turn propels droplets containing bacterial colonies to a collection point. The collection device is then activated to collect the bacterial droplets.

[0115] The following is a complete workflow introduction:

[0116] I. Sample Pretreatment

[0117] 1. Sample introduction process

[0118] It mainly includes two steps: sample introduction and culture.

[0119] This process includes sample spotting, inspection, and sealing, which mainly enables the sample to enter the microfluidic chip and be individually distributed into each chamber.

[0120] 2. Training process

[0121] Place the chip in the incubator, taking care to maintain chip humidity and prevent liquid evaporation. The incubator temperature should be determined based on the type of sample.

[0122] II. Preparations before selection

[0123] The main steps are as follows:

[0124] 1.1 Placing the chip: Connect the tubing, place the chip in the chip fixture, and put it on the stage;

[0125] 1.2 Oil Inlet: Turn on the chip power source and introduce the oil phase. The oil phase introduction time is about 10-60 minutes.

[0126] 1.3 Positioning: Manually / automatically move the chip to the upper left corner of the image, manually / automatically adjust the focus until the image is clear, and set the chip's three-axis platform to its zero point position. Adjust the three-axis position of the moving aperture plate so that the chip capillary is coaxial with the initial aperture of the aperture plate as observed by the human eye, and set the aperture plate's three-axis to its zero point position. During coaxial setting, the aforementioned detection module 2 can also be used for automatic identification and adjustment to achieve automated coaxial adjustment.

[0127] III. Detection / Selection

[0128] The detection can be performed using two methods: bright field and fluorescence. The appropriate method can be selected according to the specific requirements, and the selection method is the same as that of the fluorescence detection module 11 and imaging module 13 mentioned above.

[0129] Due to the limited field of view of microscopic imaging, the chip needs to be imaged multiple times to complete the image acquisition of all chambers. Each image acquisition is a screen, and the switching of each screen is achieved by moving the chip platform 3. Therefore, the detection / picking step is divided into two modes: synchronous mode and asynchronous mode.

[0130] (I) Synchronization Mode

[0131] Image acquisition, target recognition, and picking are all completed within the same image acquisition screen. That is, image acquisition and image recognition are performed simultaneously, and target colonies are picked up at the same time. The specific steps are as follows:

[0132] 1. Bright field

[0133] 1.1 Identify the initial position

[0134] Initial screen recognition, based on pre-selection preparation -> positioning -> initial screen settings, uses AI technology to identify the initial screen. The specific recognition algorithm is as follows (taking the upper left corner of the image as an example):

[0135] S1: Image acquisition;

[0136] S2: Image input, which sends the acquired image into the model (this model is a model that has been trained and established in the previous stage);

[0137] S3: Image discrimination, based on the model, determines whether an image contains a cavity;

[0138] S4-1: If so, continue moving the chip to the left to the next screen, and repeat steps S1-S3;

[0139] S4-2: If not, return to the previous screen;

[0140] S5: Boundary correction. Set the Padding value so that the leftmost chamber edge is equal to the left boundary of the image, making the chambers symmetrically distributed within the image.

[0141] Similarly, the up and down position recognition is completed, and finally the initial screen recognition is achieved, and the platform moves to the initial screen position.

[0142] 1.2 Autofocus / Correction

[0143] Perform image autofocus and chip tilt correction, and record relevant data of the current screen, including position information, focus information, and correction parameters. The specific focusing / correction methods are as follows:

[0144] Autofocus (hill-climbing algorithm)

[0145] S1: Image acquisition, set the Z-axis position to the zero point at this time;

[0146] S2: Image input, which sends the acquired image to the OpenCvSharp algorithm;

[0147] S3: Image gradient value calculation, using the Tenengrad gradient method: Sobe l operator to calculate image gradient values;

[0148] S4: Image set acquisition. Move the image set up and down at the zero point of the Z-axis with a step interval to acquire the image set, calculate the image gradient value, draw the curve of Z-axis position and gradient value, and find the peak position (as shown in the figure).

[0149] S5: Image focusing. The platform moves to the highest point of the peak. At this time, the Z-axis position is the image in focus, thus achieving image focusing.

[0150] Chip tilt correction:

[0151] S1: Image acquisition, acquiring the focused image;

[0152] S2: Image input, the image is fed into the trained model;

[0153] S3: Data point generation, identification of chambers, and then identification of the center point of the chambers to form a data point column. Each point has a coordinate (X, Y) to form a two-dimensional array.

[0154] S4: K value calculation, each row of array is connected and fitted, the average slope S_lope is calculated, S_lope has positive and negative values, K is calculated based on the slope S_lope, K value is the overall tilt distance of the chamber.

[0155] S5: Correct the image. Based on K obtained in step S4, move the chip platform. A positive K value moves the platform upward; a negative K value moves it downward.

[0156] Complete autofocus and chip tilt correction, and record the current screen position information, focus information, and correction parameters.

[0157] 1.3 Image Acquisition

[0158] The image acquisition device activates its image capture function to capture the current screen image.

[0159] 1.4 Image Recognition

[0160] 1.4.1 ID information allocation: the outer contour of each cavity in the image is identified, the identified cavities are bounded and labeled, and the location information of each cavity is recorded.

[0161] S1: Image acquisition, acquiring the focused image;

[0162] S2: Image input, the image is fed into the trained model;

[0163] S3: Data point generation, identification of chambers, and then identification of the center point of the chambers to form a data point column, each point having a coordinate (X, Y);

[0164] S4: Minimum point traversal, traversing the center point with the minimum Y value Mi n.

[0165] S5: Drawing cutting lines, drawing cutting lines using Min±Step respectively;

[0166] S6: Same row determination. Traverse all center points. If Y is within the range of Min±Step, then they are in the same row and sorted by X value from smallest to largest, and set as the first row (odd row).

[0167] S7: Repeat the next row traversal, setting it as the second row (even-numbered row), sorting by X values ​​from largest to smallest following the previous row:

[0168] S8: The data point sorting information is assigned to each corresponding chamber, completing the chamber ID allocation.

[0169] At this point, the ID information for each chamber was assigned, including pixel coordinates, platform coordinates, and sequence information.

[0170] 1.4.2 Colony identification: For each identified chamber, the presence or absence of colonies is determined by marking the chamber as 0 or 1, where 0 represents the absence of colonies and 1 represents the presence of colonies.

[0171] 1.4.3 Path planning: For all chambers in the image with a value of 1, the shortest path is planned based on a greedy algorithm, and the chambers are labeled with serial numbers 1-N (N≤50).

[0172] 1.5 Picking

[0173] Based on the image recognition information from step 1.4, colonies are selected as follows:

[0174] 1.5.1 Chamber positioning: According to step 1.4.3, the chamber with serial number 1 is moved to the photothermal heating point by moving the chip platform;

[0175] 1.5.2 Turn on the laser 1201 and let its optical-thermal optical path irradiate the cavity 1;

[0176] 1.5.3 Time setting: Based on T=t0+Δt*(m-1) (where T is the start time for collecting bacterial droplets, t0 is the base time, Δt is the time interval, and m is the number of screens), the collection time of bacterial droplets is dynamically set;

[0177] 1.5.4 Start the collection: Start the receiver (taking a 96-well plate as an example) to collect bacterial droplets. The specific process is to switch the well positions of the 96-well plate and move it up and down.

[0178] 1.5.5 Repeat steps 1.5.1-1.5.4 to complete the picking of all colony chambers.

[0179] 1.6 Switch screen

[0180] 1.7 Repeat steps 1.2-1.6 until all images have been acquired.

[0181] (II) Asynchronous Mode

[0182] First, all images are acquired, then identified and selected. The specific steps are as follows:

[0183] Bright Field

[0184] 1. Panoramic image shooting

[0185] 1.1 Initial Screen Recognition: Based on the pre-selection preparation -> positioning -> initial screen setting, AI technology is used to recognize the initial screen. The specific recognition algorithm is as follows (taking the upper left corner of the image as an example):

[0186] S1: Image acquisition;

[0187] S2: Image input, which sends the acquired image into the model (this model is a model that has been trained and established in the previous stage);

[0188] S3: Image discrimination, based on the model, determines whether an image contains a cavity;

[0189] S4-1: If so, continue moving the chip to the left to the next screen, and repeat steps S1-S3;

[0190] S4-2: If not, return to the previous screen;

[0191] S5: Boundary correction. Set the Padding value so that the leftmost chamber edge is equal to the left boundary of the image, making the chambers symmetrically distributed within the image.

[0192] Similarly, the up and down position recognition is completed, and finally the initial screen recognition is achieved, and the platform moves to the initial screen position.

[0193] Once the initial screen recognition is complete, the platform moves to the initial screen position.

[0194] 1.2 Perform image autofocus and chip tilt correction, and record relevant data of the current screen, including position information, focus information, and correction parameters. The specific focusing / correction method is as follows:

[0195] Autofocus (hill-climbing algorithm)

[0196] S1: Image acquisition, set the Z-axis position to the zero point at this time;

[0197] S2: Image input, which sends the acquired image to the OpenCvSharp algorithm;

[0198] S3: Image gradient value calculation, using the Tenengrad gradient method: Sobe l operator to calculate image gradient values;

[0199] S4: Image set acquisition. Move the image set up and down at the zero point of the Z-axis with a step interval to acquire the image set, calculate the image gradient value, draw the curve of Z-axis position and gradient value, and find the peak position (as shown in the figure).

[0200] S5: Image focusing. The platform moves to the highest point of the peak. At this time, the Z-axis position is the image in focus, thus achieving image focusing.

[0201] S1: Image acquisition, acquiring the focused image;

[0202] S2: Image input, the image is fed into the trained model;

[0203] S3: Data point generation, identification of chambers, and then identification of the center point of the chambers to form a data point column. Each point has a coordinate (X, Y) to form a two-dimensional array.

[0204] S4: K value calculation, each row of array is connected and fitted, the average slope S_lope is calculated, S_lope has positive and negative values, K is calculated based on the slope S_lope, K value is the overall tilt distance of the chamber.

[0205] S5: Correct the image. Based on K obtained in step S4, move the chip platform. A positive K value moves the platform upward; a negative K value moves it downward.

[0206] Complete autofocus and chip tilt correction, and record the current screen position information, focus information, and correction parameters.

[0207] 1.3 Image Acquisition - The image acquisition device acquires images and displays them in the panoramic frame. The panoramic image is set to N*M pairs according to the number of chip chambers, where N is the number of rows and M is the number of columns. The image N1 of the first row and first column is acquired.

[0208] 1.4 Chamber Identification / Data Analysis: Image recognition and data analysis are performed on the acquired images.

[0209] ID information is assigned, the outer contour of each cavity in the image is identified, the identified cavities are bounded and labeled, and the location information of each cavity is recorded.

[0210] S1: Image acquisition, acquiring the focused image;

[0211] S2: Image input, the image is fed into the trained model;

[0212] S3: Data point generation, identification of chambers, and then identification of the center point of the chambers to form a data point column, each point having a coordinate (X, Y);

[0213] S4: Minimum point traversal, traversing the center point with the minimum Y value Mi n.

[0214] S5: Drawing cutting lines, drawing cutting lines using Min±Step respectively;

[0215] S6: Same row determination. Traverse all center points. If Y is within the range of Min±Step, then they are in the same row and sorted by X value from smallest to largest, and set as the first row (odd row).

[0216] S7: Repeat the next row traversal, setting it as the second row (even-numbered row), sorting by X values ​​from largest to smallest following the previous row:

[0217] S8: The data point sorting information is assigned to each corresponding chamber, completing the chamber ID allocation.

[0218] At this point, ID information allocation for each chamber was implemented, including pixel coordinates, platform coordinates, and sequence information. Colony identification involved determining the presence or absence of colonies in each identified chamber, assigning a 0 / 1 identifier to each chamber, where 0 represents the absence of colonies and 1 represents the presence of colonies.

[0219] Colony identification: If a chamber is marked with 1, further identification is performed on that chamber. Based on a semantic segmentation algorithm, the colony region (ROI) is identified, and its contour is obtained. The contour is then calculated using OpenCV, which calculates the area, average gray level, axis center, and other relevant data of the region containing the contour. The colony area and gray level are labeled on the chamber, and a scatter plot is constructed using the area and gray level values, along with the range of area and gray level values. This step can be performed in a separate thread, concurrently with the next step.

[0220] 1.5 Switch to the next screen and repeat steps 1.2-1.4 until all images have been captured, completing the panoramic image capture.

[0221] 2. Set the selection parameters

[0222] Set the selection criteria, which include the area of ​​the chamber, the gray level, and the product of the two values ​​(concentration). For example, the numerical range is 0-1 (or 0-255). Select the desired numerical range based on the displayed value.

[0223] 3. Pick

[0224] 3.1 Return to the initial screen, returning to the initial position where the panoramic image was first captured;

[0225] 3.2 Path planning: Based on the selection parameters set in step 2, the shortest path is planned for each screen image and the sequence number is marked from NM-1 to NM-P, where N represents the row, M represents the column, and P represents the number of chambers in the screen image, P≤50. The images are selected according to the marked sequence number.

[0226] 3.3 Chamber positioning: According to step 3.2, the chamber with serial number 1 is moved to the photothermal heating point by moving the chip platform;

[0227] 3.4 Open the optical shutter 1202 to allow the photothermal optical path to irradiate the photothermal heating point of the chamber 1;

[0228] 3.5 Time setting: Based on T=t0+Δt*(m-1) (where T is the start time for collecting bacterial droplets, t0 is the base time, Δt is the time interval, and m is the number of screens, i.e. the number of columns), the collection time of bacterial droplets is dynamically set;

[0229] 3.6 Collect droplets; start collecting droplets according to the set time.

[0230] 3.7 Continue with steps 3.2-3.6 until all colonies in the current chamber have been picked;

[0231] 3.8 Switch to the next screen and repeat steps 3.2-3.7 until all screens are completed.

[0232] It should be noted that: if a photothermal module 12 with a spatial light modulator 1208 is used, when multiple photothermal optical paths are used to irradiate a chamber containing target colonies, it is not necessary to conduct experiments through the moving chip platform 3. The image information of the chamber containing multiple target colonies is directly obtained, converted into a corresponding phase map, and then synchronous irradiation of multiple photothermal optical paths can be achieved through the corresponding holographic calculation formula. Furthermore, since the distance between each chamber and the outlet of the array-type chamber chip is different, the target colonies can be quickly exported in the corresponding order, thereby greatly improving the selection efficiency.

[0233] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A colony selection system based on light manipulation technology, characterized in that, include: The chip platform is equipped with a chip placement section for placing array-type chamber chips; The orifice plate platform is equipped with an orifice plate placement section for placing orifice plates; Photothermal module, used to emit photothermal light; The optical-thermal path consists of one unit and illuminates the photosensitive heating element on one of the corresponding array-type cavity chips. or The number of photothermal optical paths is multiple and they irradiate in parallel onto the photosensitive heating elements in multiple corresponding array-type cavity chips; The imaging module is used to identify the state of bacterial colonies within the chambers of the array-type chamber chip within the current field of view; The automatic acquisition and control module identifies and calibrates the colonies in the chambers of the array-type chamber chip through the recognition of the imaging module, so as to control the photothermal module to perform photothermal heating on the photosensitive heating part on the identification and calibration chamber, and adjust the position of the chip platform and / or the well plate platform so that the target colonies are transported from the array-type chamber chip to the collection chamber of the well plate. The detection module is used to identify whether the droplets discharged from the array chamber chip contain target colonies, and to determine whether the end of the droplets discharged from the array chamber chip is aligned with the collection hole corresponding to the well plate.

2. The colony selection system according to claim 1, characterized in that, The horizontal height of the chip placement section is higher than the horizontal height of the perforated plate placement section.

3. The colony selection system according to claim 1, characterized in that, Also includes: A fluorescence detection module is used to emit fluorescence light; the emitting ends of the photothermal module, the imaging module, and the fluorescence detection module share the same microscope objective and are fixedly installed above the chip placement part.

4. The colony selection system according to claim 1, characterized in that, The automatic data acquisition and control module includes: The detection and identification module is used to detect bacterial colonies within the chamber; the detection and identification module is one or any combination of fluorescence detection module, Raman detection module, laser confocal detection module, and mass spectrometry detection module.

5. The colony selection system according to claim 1, characterized in that, The photothermal module includes a colony picking light source and a beam shaping module; the colony picking light source is a laser light source or an LED light source; the beam shaping module is used to shape the photothermal optical path irradiating the photosensitive heating part into a single beam or multiple beams and to reduce the beam size.

6. The colony selection system according to claim 1, characterized in that, The detection module includes: A vision camera is located below the perforated plate platform; A beam splitter is positioned above the vision camera; An illumination source and a reflector are respectively disposed on both sides of the beam splitter; the illumination source emits a light beam and the beam is split into reflected light and transmitted light by the beam splitter; the reflected light is used to illuminate the collection hole corresponding to the orifice plate to determine the alignment status of the end of the droplet discharged from the array chamber chip with the corresponding collection hole of the orifice plate; the transmitted light is used by the reflector to illuminate the array chamber chip for imaging.

7. The colony selection system according to claim 6, characterized in that, The chip platform and the well plate platform move horizontally along the XY axis and move relative to each other vertically. The vision camera is used to identify whether the end of the discharged droplet contains a target colony. The target colony is collected by controlling the relative vertical movement of the chip platform and the well plate platform.

8. The colony selection system according to claim 7, characterized in that, Also includes: A chip export tube is used to connect to the export port in an array-type chamber chip; the chip export tube is vertically disposed at the export port or fixedly disposed between the chip platform and the orifice plate platform and connected to the export port through a flexible connecting tube.

9. A selection method based on the colony selection system based on light manipulation technology according to any one of claims 1-8, characterized in that, Methods for collecting target colonies include: synchronous mode and asynchronous mode; The synchronous mode acquires information about target colonies in the array-type chamber chip within the current field of view, plans the picking path of the target colonies according to a greedy algorithm, and collects them according to the planned path until the picking of target colonies in the entire array-type chamber chip is completed. In asynchronous mode, image acquisition of the array-type chamber chip is completed, and colonies are identified and labeled based on the image acquisition. All labeled target colonies are then selected sequentially using a greedy algorithm for path planning.

10. The selection method according to claim 9, characterized in that, It also includes a chip tilt correction method: acquiring image information of the array-type chamber chip after placement and identifying each chamber, forming a data point column by the center points of each chamber located on the same horizontal baseline, and obtaining the corresponding two-dimensional array coordinates; solving the average slope Slope by fitting the connection between adjacent arrays on each horizontal baseline, with Slope having positive and negative values, solving K based on Slope, where K is the distance of the overall tilt of each chamber, and moving the chip platform according to the K value, with a positive K value indicating upward movement; The K value shifts downwards to a negative value.

11. The selection method according to claim 9, characterized in that, The method by which the photothermal module irradiates the photosensitive heating element on the cavity of the array-type cavity chip is as follows: By adjusting the position of the chip platform, a single photothermal optical path formed by the laser is irradiated onto the chamber containing the target bacterial colony, and the target bacterial colony is extracted by photothermal heating. or By expanding and aligning the laser beam, it enters a spatial light modulator. The spatial light modulator forms multiple photothermal paths within the current field of view based on the image of the chamber positions containing the target colonies. These paths are then irradiated onto the respective chambers containing the target colonies. Since the distances from the exit to each chamber within the current field of view are different, all target colonies are sequentially extracted by photothermal heating.

Images