Time-difference incubator integrating two-dimensional and three-dimensional imaging and control method thereof

Abstract

The present application relates to the technical field of incubator imaging, in particular to a time difference incubator integrated with two-dimensional and three-dimensional imaging and a control method thereof, the incubator comprising: a cabinet, a culture environment module, a turntable, a culture dish, a two-dimensional imaging module, a three-dimensional imaging module, a driving module and a control module arranged in the cabinet; the two-dimensional imaging module is used for collecting two-dimensional images of the culture target; the control module obtains position information and three-dimensional scanning information of the culture target according to the two-dimensional images, controls the three-dimensional imaging module to collect three-dimensional images of the culture target based on the position information and the three-dimensional scanning information; and the control module processes the two-dimensional images and the three-dimensional images to generate a time difference sequence. The present application performs three-dimensional scanning based on two-dimensional image feedback information, processes the two-dimensional images and the three-dimensional images, and generates a time difference sequence, thereby providing comprehensive data support for cell morphology and growth and improving the precision and efficiency of culture target monitoring.

Description

Technical Field

[0001] This invention relates to the field of incubator imaging technology, specifically to a time-difference incubator integrating two-dimensional and three-dimensional imaging and its control method. Background Technology

[0002] With the continuous development of biomedical and cell culture technologies, incubators play an important role in laboratory research. Traditional incubators mainly provide basic culture conditions such as constant temperature and humidity, while time-lapse incubators have built-in imaging systems that enable long-term, non-invasive observation of live cells, recording key parameters such as cell division time points, fragmentation rate, and morphological changes.

[0003] Currently, monitoring of cell culture processes largely relies on two-dimensional imaging technology, using cameras or microscopes to observe targets in culture dishes statically or dynamically. While two-dimensional imaging technology offers high spatial resolution, it has limitations in dynamically observing and accurately analyzing sample morphology and structural changes. Therefore, three-dimensional imaging technology is increasingly being introduced into culture monitoring to provide more comprehensive stereoscopic images, thereby enabling efficient monitoring of dynamic characteristics such as cell morphology, location, and activity. Three-dimensional imaging technology can acquire spatial structural information of the target more accurately, but it requires lengthy scanning times and complex equipment configurations, resulting in cumbersome operation and low imaging efficiency. Summary of the Invention

[0004] (a) Purpose of the invention The purpose of this invention is to provide a time difference incubator that integrates two-dimensional and three-dimensional imaging and its control method. It performs three-dimensional scanning based on two-dimensional image feedback information, processes the two-dimensional and three-dimensional images to generate time difference sequences, provides comprehensive data support for cell morphology and growth, and improves the accuracy and efficiency of monitoring culture targets.

[0005] (II) Technical Solution To address the above problems, this invention provides a time-difference incubator integrating two-dimensional and three-dimensional imaging, comprising: The enclosure and the culture environment module, turntable, culture dish, two-dimensional imaging module, three-dimensional imaging module, drive module and control module disposed within the enclosure; The culture environment module is used to provide culture conditions for the culture target; the culture dish is placed on the turntable to support the culture target; The two-dimensional imaging module, the three-dimensional imaging module, and the drive module are connected to the control module, and the turntable is connected to the drive module via a transmission connection. The two-dimensional imaging module is used to acquire two-dimensional images of the culture target. The control module obtains the position information and three-dimensional scanning information of the culture target based on the two-dimensional images, and controls the three-dimensional imaging module to acquire three-dimensional images of the culture target based on the position information and three-dimensional scanning information. The control module processes the two-dimensional and three-dimensional images to generate a time difference sequence.

[0006] In another aspect of the present invention, preferably, the culture dish includes a plurality of through holes, each through hole corresponding to a culture target, and the center of each through hole has the same radius and angular position relative to the axis of the turntable.

[0007] In another aspect of the present invention, preferably, the two-dimensional imaging module and the three-dimensional imaging module adopt a movable structure, and the two-dimensional imaging module and the three-dimensional imaging module are respectively arranged at the first station and the second station on the turntable motion trajectory; The two-dimensional imaging module is used to perform two-dimensional imaging of the culture target at the first station, and the three-dimensional imaging module is used to perform three-dimensional imaging of the culture target at the second station. The control module controls the drive module to switch the turntable between the first workstation and the second workstation.

[0008] In another aspect of the present invention, preferably, the two-dimensional imaging module and the three-dimensional imaging module adopt a coaxial integrated structure; The coaxial integrated structure includes a dichroic beam splitter and a common objective lens. The dichroic beam splitter is used to combine or split the two-dimensional imaging optical path and the three-dimensional imaging optical path, so that the two imaging modes can share the common objective lens.

[0009] In another aspect of the present invention, preferably, the two-dimensional imaging module includes a modulation component and a removal component, wherein the removal component is connected to the modulation component; The modulation component is used to modulate the imaging light to form illumination conditions for two-dimensional imaging. The removal component is used to position the modulation component in the working position when the two-dimensional imaging module is working, and to remove the modulation component from the working position when the three-dimensional imaging module is working.

[0010] In another aspect, preferably, a control method for a time difference incubator integrating two-dimensional and three-dimensional imaging is provided. The control method is applicable to the time difference incubator described above, and includes: The two-dimensional imaging module is controlled to acquire two-dimensional images of the culture target. The two-dimensional image is processed to obtain the location information and three-dimensional scanning information of the culture target; Based on the location information and three-dimensional scanning information, the three-dimensional imaging module is controlled to acquire three-dimensional images of the culture target to obtain three-dimensional images. The two-dimensional and three-dimensional images are subjected to time correlation and fusion processing to obtain the time difference sequence of the training target.

[0011] In another aspect of the present invention, preferably, the image processing of the two-dimensional image to obtain the location information and three-dimensional scan information of the culture target includes: The two-dimensional image is preprocessed to obtain a preprocessed two-dimensional image; Using a preset contour extraction method, the contour region of the culture target in the preprocessed two-dimensional image is extracted; Using a preset position algorithm and the contour region, the centroid coordinates of the culture target are calculated, and the position information of the culture target in the two-dimensional image is determined based on the centroid coordinates. Based on the pre-established coordinate calibration relationship, the position information is converted from two-dimensional image coordinates to the physical coordinates of the culture target, and the physical coordinates are converted to the scanning coordinates of the three-dimensional imaging module to obtain the corresponding three-dimensional scanning information.

[0012] In another aspect of the present invention, preferably, the step of controlling the three-dimensional imaging module to acquire three-dimensional images of the culture target based on the position information and the three-dimensional scanning information, and obtaining a three-dimensional image, includes: Based on the location information and the preset calibration transformation matrix, the galvanometer scanning coordinates of the three-dimensional imaging module are obtained; Based on the response relationship between the galvanometer scanning coordinates and the galvanometer deflection angle and the driving voltage, the galvanometer control voltage is obtained; The starting point and scanning range of the three-dimensional scan are determined based on the three-dimensional scan information; Based on the galvanometer control voltage, the three-dimensional scanning start point, and the scanning range, the three-dimensional imaging module is controlled to acquire three-dimensional images of the culture target and obtain three-dimensional images.

[0013] In another aspect of the present invention, preferably, the step of performing time correlation and fusion processing on the two-dimensional image and the three-dimensional image to obtain the time difference sequence of the training target includes: The two-dimensional and three-dimensional images are processed according to a preset indexing rule to obtain a first sequence based on the training objective; Based on the pre-established coordinate calibration relationship, the two-dimensional and three-dimensional images in the first sequence are spatially registered so that the same cultivation target is aligned in the two-dimensional image coordinates and the three-dimensional image coordinates to obtain the second sequence; Based on the second sequence, the two-dimensional texture information in the two-dimensional image is mapped to the corresponding spatial position in the three-dimensional image to obtain a three-dimensional model that integrates the two-dimensional texture information; By associating the two-dimensional image with the three-dimensional model in chronological order, the time difference sequence of the training target is obtained.

[0014] In another aspect, preferably, the invention further includes, before the two-dimensional imaging module acquires two-dimensional images of the culture target, performing visual servo centering and hierarchical adaptive focusing processing on the culture dish to obtain the spatial alignment state of the culture dish and the two-dimensional imaging module and the imaging focal plane parameters. The visual servo centering and hierarchical adaptive focusing processing obtains the spatial alignment state and imaging focal plane parameters between the culture dish and the two-dimensional imaging module, including: The geometric center of the through-hole is determined based on the boundary information of the through-hole in the culture dish in the two-dimensional image, and compared with the current field of view center of the two-dimensional imaging module to obtain the positional deviation. The position of the culture dish is corrected based on the position deviation to align the geometric center of the through hole with the current field of view, thereby obtaining the spatial alignment state between the through hole of the culture dish and the two-dimensional imaging module. Determine the reference through hole, perform coarse focusing on the reference through hole, determine the optimal focal plane position within a preset travel range, and determine it as the reference focal plane; Based on the reference focal plane and the preset adaptive interval algorithm, a narrow band interval centered on the reference focal plane is constructed; Based on the narrow band range, fine focusing is performed on the remaining through holes to obtain the imaging focal plane parameters corresponding to each through hole.

[0015] (III) Beneficial Effects The above-described technical solution of the present invention has the following beneficial technical effects: This invention provides real-time image feedback through a two-dimensional imaging module. The control module analyzes the two-dimensional image to obtain the location information and three-dimensional scanning information of the culture target, and controls the three-dimensional imaging module accordingly to achieve precise three-dimensional image acquisition. This avoids the time waste of traditional imaging methods and significantly improves the accuracy and efficiency of imaging. Furthermore, the two-dimensional and three-dimensional images are processed to generate time-varying sequences, providing comprehensive data support for cell morphology and growth. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall structure of one embodiment of the present invention; Figure 2 This is a schematic diagram of the box structure according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a petri dish structure according to an embodiment of the present invention; Figure 4 This is a side view of the movable structure of a two-dimensional imaging module and a three-dimensional imaging module according to an embodiment of the present invention; Figure 5This is a top view of the movable structure of a two-dimensional imaging module and a three-dimensional imaging module according to an embodiment of the present invention; Figure 6 This is a side view of the coaxial integrated structure of a two-dimensional imaging module and a three-dimensional imaging module according to an embodiment of the present invention; Figure 7 This is a top view of a coaxial integrated structure of a two-dimensional imaging module and a three-dimensional imaging module according to an embodiment of the present invention; Figure 8 This is a schematic diagram of image processing according to an embodiment of the present invention; Figure 9 This is a schematic diagram of visual servo centering and hierarchical adaptive focusing according to an embodiment of the present invention; Figure label: 1: Box body, 2: Turntable, 3: Petri dish, 31: Through hole, 4: Two-dimensional imaging module, 5: Three-dimensional imaging module, 6: Drive module, 7: Control module. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0018] The accompanying drawings show structural schematic diagrams according to embodiments of the present invention. These drawings are not drawn to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0019] Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0020] In the description of this invention, it should be noted that the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0021] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0022] The invention will now be described in more detail with reference to the accompanying drawings. In the various drawings, the same elements are indicated by similar reference numerals. For clarity, the various parts in the drawings are not drawn to scale.

[0023] Example 1 A time-difference incubator integrating two-dimensional and three-dimensional imaging. Figure 1 A schematic diagram of the overall structure of an embodiment of the present invention is shown, as follows. Figure 1 As shown, it includes: The enclosure 1 and the culture environment module, turntable 2, culture dish 3, two-dimensional imaging module 4, three-dimensional imaging module 5, drive module 6 and control module 7 disposed within the enclosure 1; Figure 2 A schematic diagram of a box structure according to an embodiment of the present invention is shown, as follows: Figure 2 As shown, the chamber 1 has an internal cavity. The chamber 1 can be designed with heat insulation, waterproofing, and corrosion resistance to ensure stable operation of the equipment and meet the environmental requirements of the cultured target. Externally, heat dissipation holes can be provided to ensure uniform temperature and humidity distribution within the chamber and to prevent overheating or humidity fluctuations caused by equipment operation. Various modules are housed within the cavity of the chamber 1. In this embodiment, the chamber 1 includes a culture environment module, a turntable 2, a culture dish 3, a two-dimensional imaging module 4, a three-dimensional imaging module 5, a drive module 6, and a control module 7. The culture environment module provides culture conditions for the culture target, ensuring its growth or reaction in an ideal environment. The culture environment module may include a temperature unit, a humidity unit, a gas composition unit, etc. For example, the temperature can be adjusted by built-in heating or cooling components, the humidity can be controlled by humidification or dehumidification components, and the gas composition can be equipped with a gas regulation device, such as a concentration adjustment component for oxygen, carbon dioxide, or other gases. The temperature unit, humidity unit, and gas composition unit can be controlled by the control module 7 to provide for different experimental needs and specific growth requirements of the culture target.

[0024] The culture dish 3 is mounted on the turntable 2 to support the culture target. The turntable 2 is connected to the drive module 6 via a transmission structure. The drive module 6 is connected to the control module 7. The drive module 6 provides power for the rotation or movement of the turntable 2 to adjust the position or angle of the culture dish 3. The control module 7 issues commands to adjust the movement of the turntable 2 driven by the drive module 6, thereby controlling the position of the culture dish 3. The culture dish 3 is made of glass or transparent plastic material, with good optical transmittance, supporting the imaging requirements of the two-dimensional imaging module 4 and the three-dimensional imaging module 5. The turntable 2 may have a groove structure adapted to the shape of the culture dish 3 to limit the movement of the culture dish 3 and prevent unnecessary movement of the culture dish 3 during the movement of the turntable 2. Figure 3A schematic diagram of a petri dish structure according to an embodiment of the present invention is shown, as follows: Figure 3 As shown, the culture dish 3 includes several through holes 31, each corresponding to a culture target. The center of each through hole 31 has the same radius and angular position relative to the axis of the turntable 2. A hole position index is established based on the encoder feedback of the turntable 2, and each through hole is defined as a set of polar coordinate parameters, namely, hole position index i, rotation radius R, and corresponding angular position θ. i By using polar coordinates to position the through-holes in the culture dish, precise control and imaging of the cultured target at different locations can be achieved. The radius represents the distance of the through-hole 31 from the axis of the turntable 2, while the angle represents the rotational angle of the through-hole 31 relative to the axis of the turntable 2. In this embodiment, the radius and angle positions of all through-holes 31 are fixed and symmetrical in the culture dish 3, ensuring that the position of the cultured target in different through-holes 31 is consistent, ensuring that the alignment of the cultured target with the two-dimensional and three-dimensional imaging modules is maintained during rotation, and ensuring the accuracy and repeatability of the imaging data.

[0025] The two-dimensional imaging module 4 and the three-dimensional imaging module 5 are connected to the control module 7. The two-dimensional imaging module 4 can be a bright-field microscope imaging module, and the three-dimensional imaging module 5 can be an OCT imaging module. The two-dimensional imaging module 4 and the three-dimensional imaging module 5 are installed in an inverted manner below the turntable 2, with the optical axis of the objective lens facing upwards, aligned with the circumferential trajectory of the culture target on the turntable 2.

[0026] The objective lenses of the two-dimensional imaging module 4 and the three-dimensional imaging module 5 are aligned with the upward optical axis of the target being cultured; furthermore, in this embodiment, Figure 4 A side view of the movable structure of a two-dimensional imaging module and a three-dimensional imaging module according to an embodiment of the present invention is shown. Figure 5 A top view of the movable structure of a two-dimensional imaging module and a three-dimensional imaging module according to an embodiment of the present invention is shown, as follows: Figure 4 and Figure 5As shown, the two-dimensional imaging module 4 and the three-dimensional imaging module 5 adopt a movable structure. The two-dimensional imaging module 4 and the three-dimensional imaging module 5 are respectively arranged at the first station and the second station on the motion trajectory of the turntable 2. The first station and the second station are arranged at intervals along the circumference of the turntable 2, so that the turntable can pass through the two imaging areas in sequence during rotation. The two-dimensional imaging module 4 is used to perform two-dimensional imaging of the culture target at the first station, and the three-dimensional imaging module 5 is used to perform three-dimensional imaging of the culture target at the second station. The control module 7 controls the drive module 6 to drive the turntable 2 to switch between the first station and the second station. The first station is set as the two-dimensional imaging area, and the second station is set as the three-dimensional imaging area. The two-dimensional imaging module 4 acquires two-dimensional images of the culture target at the first station. The control module 7 obtains the position information and three-dimensional scanning information of the culture target based on the two-dimensional images. Based on the position information, the control module 7 controls the drive module 6 to drive the turntable 2 to rotate so that the culture target is in the second station. During the switch, the coordinates can be converted according to the preset angle and encoder angle feedback. Furthermore, the control module 7 controls the three-dimensional imaging module 5 to acquire three-dimensional images of the culture target according to the three-dimensional scanning information. The two-dimensional imaging module 4 and the three-dimensional imaging module 5 are fixed at two different positions on the turntable track. Because the two systems are physically separated, the difficulty of optical path coupling is reduced, the structure is simple, maintenance is convenient, and it is suitable for high-throughput inspection scenarios with multiple apertures.

[0027] Furthermore, in another embodiment, Figure 6 A side view of a coaxial integrated structure of a two-dimensional imaging module and a three-dimensional imaging module according to an embodiment of the present invention is shown. Figure 7 A top view of a coaxial integrated structure of a two-dimensional imaging module and a three-dimensional imaging module according to an embodiment of the present invention is shown, as follows: Figure 6 and Figure 7As shown, the two-dimensional imaging module 4 and the three-dimensional imaging module 5 adopt a coaxial integrated structure. This coaxial integrated structure includes a dichroic beam splitter and a common objective lens. The dichroic beam splitter is used to combine or split the two-dimensional imaging optical path and the three-dimensional imaging optical path, allowing both imaging modes to share the common objective lens. The common objective lens is located at the front end of the imaging system and is used for unified optical imaging and signal acquisition of the culture target. The dichroic beam splitter is located at the branching position of the optical path and is used to split or combine the optical path according to the characteristics of optical signals in different wavelength bands, enabling the two-dimensional imaging optical path and the three-dimensional imaging optical path to achieve coaxial transmission in space. The dichroic beam splitter can selectively transmit or reflect excitation light, reflected light, or fluorescence signals based on the optical characteristics of different wavelengths. In the two-dimensional imaging mode, the illumination light is modulated and illuminates the culture target. The reflected light signal enters the optical path through the common objective lens and is separated by the dichroic beam splitter before entering the two-dimensional imaging sensor, thereby obtaining a planar structural image. In three-dimensional imaging mode, structured light or scanning beams act on the culture target through a shared objective lens, and combined with reflection or fluorescence signals, the beams are split by a dichroic beam splitter and then enter the three-dimensional imaging detection unit to acquire depth information or three-dimensional reconstruction data. The two-dimensional imaging module 4 includes a modulation component and a removal component, the removal component being connected to the modulation component. The modulation component is used to modulate the imaging light to form illumination conditions for two-dimensional imaging. The modulation component may include a light source modulation unit, an aperture structure, or a filter structure to adjust the light intensity distribution, illumination uniformity, and spectral range, thereby enabling the two-dimensional imaging to obtain stable and high-contrast image information. The modulation component may be a liquid crystal modulation element.

[0028] The removal component is used to position the modulation component in the working position during the operation of the two-dimensional imaging module 4, and to remove the modulation component from the working position during the operation of the three-dimensional imaging module 5. The removal component enables spatial switching of the modulation component in different imaging modes. When the two-dimensional imaging module 4 is operating, the removal component drives the modulation component to the working position, placing it in the optical path, thereby forming standardized two-dimensional illumination and imaging conditions. When the three-dimensional imaging module 5 is operating, the removal component drives the modulation component to move out of the main optical path, restoring the optical path to an imaging state suitable for three-dimensional structured light or confocal scanning, thus avoiding interference or attenuation of the three-dimensional imaging optical path by the modulation component. The removal component can also employ an electrically controlled switchable structure, such as an electrically driven sliding mechanism, to improve the switching speed.

[0029] The two-dimensional imaging module 4 includes a bright-field microscope light source, an objective lens, a CCD camera, and a Z-axis actuator. The Z-axis actuator is used to drive the objective lens to move along the optical axis to achieve focusing. The three-dimensional imaging module 5 is the sample arm of the OCT system, including a galvanometer, a galvanometer controller, and a lens group. The galvanometer controller is used to drive the galvanometer to deflect the angle, so as to achieve the lateral and longitudinal scanning of the culture target by the beam.

[0030] The two-dimensional imaging module 4 is used to acquire two-dimensional images of the culture target. The control module 7 obtains the position information and three-dimensional scanning information of the culture target based on the two-dimensional images, and controls the three-dimensional imaging module 5 to acquire three-dimensional images of the culture target based on the position information and three-dimensional scanning information. In this embodiment, a movable or coaxial integrated structure is used to achieve a compatible arrangement of the bright-field microscope optical path and the OCT optical path within the limited space and constant temperature and gas environment of the incubator. This allows for dual-mode switching or collaborative operation without significantly sacrificing imaging performance. Because the coaxial integrated structure eliminates the need for prior movement via the turntable 2, the two imaging modules work by sharing objectives and optical axes, enabling compatible operation of the two imaging modes within a more compact space and facilitating rapid switching between them. Furthermore, the 3D image is scanned based on the positional information of the 2D image and the 3D scanning information, avoiding blind scanning of invalid areas within the through-hole by the 3D imaging module and improving the acquisition efficiency of the 3D scan.

[0031] The control module 7 processes the two-dimensional and three-dimensional images to generate a time-difference sequence. The control module 7 includes a data acquisition unit, a storage unit, an image processing unit, and a time scheduling unit. These units work collaboratively to achieve temporal organization and analysis of multimodal images. During the data acquisition phase, the control module 7 receives two-dimensional image data and three-dimensional volume data at corresponding time points from the two-dimensional imaging module 4 and the three-dimensional imaging module 5, respectively, according to a preset sampling period or triggering conditions, and assigns a unified timestamp to each frame of data. The timestamp can be generated based on the system clock or an external synchronization signal, thereby ensuring the consistency of different modal data in the time dimension. In the preprocessing phase, the image processing unit performs basic processing on the acquired two-dimensional and three-dimensional images, including denoising, brightness and contrast normalization, distortion correction, and background correction, to improve the stability and accuracy of subsequent analysis. For three-dimensional data, volume data reconstruction, slice rearrangement, or depth information extraction processing can also be performed. In the registration and fusion phase, the control module 7 performs spatial registration of the two-dimensional and three-dimensional images based on system calibration parameters or feature matching algorithms, aligning them in a unified coordinate system. Registration methods can include geometric correction based on a calibration plate, or automatic matching methods based on image feature points and edge information, thereby ensuring the comparability of data from different time points and different modalities. In the time series construction stage, the control module 7 sorts the processed image data according to the timestamp and associates the two-dimensional image sequences and three-dimensional image sequences of the same training target at different time points to construct the corresponding time difference sequences.

[0032] This embodiment provides real-time image feedback through a two-dimensional imaging module. The control module analyzes the two-dimensional image to obtain the location information and three-dimensional scan information of the culture target, and controls the three-dimensional imaging module based on this to achieve precise three-dimensional image acquisition. This avoids the time waste of traditional imaging methods and significantly improves the accuracy and efficiency of imaging. Furthermore, the two-dimensional and three-dimensional images are processed to generate time-varying sequences, providing comprehensive data support for cell morphology and growth.

[0033] Example 2 A control method for a time difference incubator integrating two-dimensional and three-dimensional imaging, the control method being applicable to the time difference incubator described above, the control method comprising: The two-dimensional imaging module 4 is controlled to acquire two-dimensional images of the culture target, thereby obtaining two-dimensional images. Image processing is performed on the two-dimensional image to obtain the location information and three-dimensional scan information of the culture target, including: The two-dimensional image is preprocessed to obtain a preprocessed two-dimensional image; the specific mathematical content is not limited here, and optionally, it may include background correction, contrast enhancement and noise suppression, etc.

[0034] Figure 8An image processing schematic diagram of an embodiment of the present invention is shown, such as... Figure 8 As shown, a preset contour extraction method is used to extract the contour region of the culture target in the preprocessed 2D image. This preset contour extraction method may include one or more combinations of threshold segmentation, edge detection, active contour modeling, template matching, or deep learning-based instance segmentation methods to extract the contour region of the culture target. For 2D images with clear boundaries, adaptive threshold segmentation and edge detection can be used; for 2D images with complex boundaries or large contrast variations, active contour models or convolutional neural networks can be used for instance segmentation.

[0035] Using a preset position algorithm and the contour region, the centroid coordinates of the culture target are calculated. The centroid can be defined as either the geometric center of the target region or a weighted center based on grayscale values. The position information of the culture target in the two-dimensional image is determined based on the centroid coordinates. The preset position algorithm may include a centroid algorithm, where the centroid coordinates are calculated using the geometric centroid or the grayscale-weighted centroid of the contour region. For a set of pixels in the contour region, the centroid coordinates can be obtained by averaging the coordinates of all target pixels or by weighting the grayscale values ​​of the contour region pixels, thus characterizing the center position of the culture target in the field of view of the two-dimensional imaging module.

[0036] Based on a pre-established coordinate calibration relationship, the position information is transformed from two-dimensional image coordinates to the physical coordinates of the culture target, and then the physical coordinates are transformed to the scanning coordinates of the three-dimensional imaging module to obtain the corresponding three-dimensional scanning information. Specifically, the two-dimensional image coordinates (u, v) are first converted into the physical coordinates (x, y) of the culture target plane based on pixel size, distortion correction parameters, and workstation position parameters, and then converted to the scanning coordinates of the three-dimensional imaging module. The pre-established coordinate calibration relationship can be obtained experimentally. In this embodiment, the conversion from two-dimensional image coordinates to the physical coordinates of the culture target is performed using a two-dimensional affine transformation or projection transformation, and the conversion from physical coordinates to the scanning coordinates of the three-dimensional imaging module is performed using a calibration matrix or lookup table.

[0037] Based on the location information and three-dimensional scanning information, the three-dimensional imaging module 5 is controlled to acquire three-dimensional images of the culture target, including: Based on the location information and the preset calibration transformation matrix, the galvanometer scanning coordinates of the three-dimensional imaging module 5 are obtained; the preset calibration transformation matrix can be obtained experimentally.

[0038] Based on the response relationship between the galvanometer scanning coordinates and the galvanometer deflection angle and the driving voltage, the galvanometer control voltage is obtained. This response relationship can be obtained experimentally. For movable structures, compensation based on the station angle and turntable angle can also be added. For coaxial integrated structures, the field-of-view calibration relationship of the two-dimensional imaging module can be used directly. The control voltage can be generated by the galvanometer controller based on linear calibration parameters or nonlinear fitting parameters.

[0039] The starting point and scanning range of the three-dimensional scan are determined based on the three-dimensional scanning information. The starting point and scanning range of the three-dimensional scan are adaptively determined based on the contour, centroid, and via range information obtained from the two-dimensional image. Furthermore, the centroid of the culture target is used as the scanning center, and the circumscribed rectangle, circumscribed circle, or their margin range of the culture target's contour is used as the lateral scanning window. In the depth direction, the starting and ending depths of the three-dimensional scan are determined by combining the reference focal plane, known structural parameters at the bottom of the via, and echo information from a one-dimensional pre-scan A-scan or a small-range B-scan, ensuring coverage of the depth range where the culture target is located.

[0040] Based on the galvanometer control voltage, the three-dimensional scanning start point, and the scanning range, the three-dimensional imaging module 5 is controlled to acquire three-dimensional images of the culture target and obtain three-dimensional images.

[0041] The two-dimensional and three-dimensional images are temporally correlated and fused to obtain the time difference sequence of the training target, including: The two-dimensional and three-dimensional images are processed according to preset indexing rules to obtain the first sequence based on the cultivation target; the two-dimensional and three-dimensional images can be managed by a unified indexing method of well number + timestamp + imaging modality.

[0042] Based on a pre-established coordinate calibration relationship, the two-dimensional and three-dimensional images in the first sequence are spatially registered to align the same culture target in both the two-dimensional and three-dimensional image coordinates, thus obtaining the second sequence. The pre-established coordinate calibration relationship can be obtained experimentally. For coaxial integrated structures, since the two optical paths share the same objective lens or the same field of view, a stable registration relationship can be obtained through a single calibration. For mobile structures, a coordinate transformation matrix is ​​established by combining the station angle, turntable angle encoder feedback, the physical coordinates of the culture target, and the system's intrinsic and extrinsic parameters.

[0043] Based on the second sequence, the two-dimensional texture information in the two-dimensional image is mapped to the corresponding spatial location in the three-dimensional image to obtain a three-dimensional model that integrates the two-dimensional texture information. Regarding texture mapping, a three-dimensional surface model of the training target is reconstructed from the three-dimensional image. Then, based on the registration results, the pixel values ​​in the two-dimensional image are mapped to three-dimensional surface points, surface mesh vertices, or patches to obtain a three-dimensional model that integrates the two-dimensional texture. Texture mapping can employ nearest-neighbor interpolation, bilinear interpolation, or higher-order interpolation; alternatively, a cross-modal fusion method based on deep learning can be used to generate an enhanced three-dimensional model.

[0044] The two-dimensional images and the three-dimensional model are correlated chronologically to obtain the time difference sequence of the culture target. In the temporal dimension, the time difference sequence is generated by periodically acquiring two-dimensional images and the three-dimensional model at preset time intervals, then storing and playing them back chronologically to form a time difference video. Within each inspection cycle, two-dimensional images of the same target micropore are acquired first, followed by corresponding three-dimensional images within a preset allowable time difference. These two-dimensional / three-dimensional images are then marked as data pairs at the same time point. This forms the time difference sequence of the same sample, achieving spatial and temporal recording of developmental information.

[0045] Furthermore, in this embodiment, Figure 9 A schematic diagram of visual servo centering and hierarchical adaptive focus according to an embodiment of the present invention is shown, as follows: Figure 9 As shown, it also includes, before the two-dimensional imaging module 4 acquires two-dimensional images of the culture target, performing visual servo centering and hierarchical adaptive focusing processing on the culture dish 3 to obtain the spatial alignment state and imaging focal plane parameters of the culture dish 3 and the two-dimensional imaging module 4. The visual servo centering and hierarchical adaptive focusing processing obtains the spatial alignment state and imaging focal plane parameters of the culture dish 3 and the two-dimensional imaging module 4, including: The geometric center of the through-hole is determined based on the boundary information of the through-hole in the culture dish 3 in the two-dimensional image, and compared with the current field of view center of the two-dimensional imaging module 4 to obtain the positional deviation. For approximately circular through-holes, circular detection or arc fitting can be used to obtain the center of the through-hole. For through-holes with perspective distortion or processing errors, elliptical fitting can be used to obtain their geometric center. The relative positional deviation includes at least translational deviation; when it is necessary to consider the error of the culture dish clamping angle or the direction of asymmetrical hole positions, rotational deviation may also be included. In the turntable, the tangential deviation can be converted into an angle compensation amount Δθ=Δs / R according to the turntable radius R. The control system then converts it into motor pulse commands or angle control commands to drive the precision motor to complete closed-loop correction. Furthermore, each through-hole to be detected is aligned once, or sampled and corrected at preset intervals and interpolated to compensate for the intermediate hole position.

[0046] The position of the culture dish 3 is corrected based on the position deviation to align the geometric center of the through hole with the current field of view, thereby obtaining the spatial alignment state between the through hole of the culture dish 3 and the two-dimensional imaging module 4. In terms of focusing, a hierarchical adaptive focusing strategy is adopted. A reference through-hole is determined, and coarse focusing is performed on this reference through-hole to determine the optimal focal plane position within a preset travel range, which is then designated as the reference focal plane. The first through-hole or a preset typical through-hole is selected as the reference hole, and the Z-axis is controlled to perform coarse focusing within the optimal travel range preset by the mechanical structure. By comparing the image sharpness at different Z-axis positions, the Z-axis position z corresponding to the sharpest image is determined. ref And this position is defined as the reference focal plane.

[0047] Based on the reference focal plane and a preset adaptive interval algorithm, a narrowband interval centered on the reference focal plane is constructed; for example, a narrowband interval [z] is established near the reference focal plane. ref -Δz1, z ref +Δz2].

[0048] Based on the narrow band interval, fine focusing is performed on the remaining through holes to obtain the imaging focal plane parameters corresponding to each through hole. The reference focal plane is the focal plane with the clearest embryo imaging. The width of the narrow band interval can be dynamically set according to the flatness of the culture dish, the micro-hole processing tolerance, the droplet height fluctuation, and historical inspection data. If necessary, the narrow band center of the current hole position can also be predicted and corrected based on the best focal plane results of adjacent holes. Through visual servo centering and hierarchical adaptive focusing processing, it is possible to align the through hole center with the optical axis and quickly focus on each hole position, significantly shortening the entire tray inspection cycle while ensuring the continuity of time difference. The drive module receives the centering compensation amount, focal plane search range, galvanometer control amount, and turntable position switching command output by the control module, and drives the turntable motor, Z-axis actuator, galvanometer controller, and related optical components to complete the corresponding actions. The control module is used to perform image acquisition control, target recognition, coordinate transformation, registration and fusion, data storage, and result display. It can acquire two-dimensional time difference sequences and three-dimensional structural sequences of the same sample continuously and automatically over a long period of time, while ensuring the stability of the culture environment and the safety of the samples. It has clear engineering feasibility and industrial application prospects.

[0049] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

[0050] The above description does not provide detailed explanations of the technical aspects of each layer's patterning and etching. However, those skilled in the art should understand that various methods existing in the prior art can be used to form layers and regions of the desired shape. Furthermore, to form the same structure, those skilled in the art can also design methods that are not entirely identical to those described above.

[0051] The present invention has been described above with reference to embodiments thereof. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. The scope of the invention is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

[0052] Although embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions, and modifications can be made to the embodiments of the present invention without departing from the spirit and scope of the invention.

[0053] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A time-difference incubator integrating two-dimensional and three-dimensional imaging, characterized in that, include: The box (1) and the culture environment module, turntable (2), culture dish (3), two-dimensional imaging module (4), three-dimensional imaging module (5), drive module (6) and control module (7) are set in the box (1); The culture environment module is used to provide culture conditions for the culture target; the culture dish (3) is set on the turntable (2) and is used to support the culture target; The two-dimensional imaging module (4), the three-dimensional imaging module (5) and the drive module (6) are connected to the control module (7), and the turntable (2) is connected to the drive module (6) via a transmission. The two-dimensional imaging module (4) is used to acquire two-dimensional images of the culture target. The control module (7) obtains the position information and three-dimensional scanning information of the culture target based on the two-dimensional images, and controls the three-dimensional imaging module (5) to acquire three-dimensional images of the culture target based on the position information and three-dimensional scanning information. The control module (7) processes the two-dimensional and three-dimensional images to generate a time difference sequence.

2. The time-difference incubator integrating two-dimensional and three-dimensional imaging according to claim 1, characterized in that, The culture dish (3) includes several through holes (31), each through hole (31) corresponding to a culture target, and the center of each through hole (31) has the same radius and angular position relative to the axis of the turntable (2).

3. The time-difference incubator integrating two-dimensional and three-dimensional imaging according to claim 1, characterized in that, The two-dimensional imaging module (4) and the three-dimensional imaging module (5) adopt a movable structure. The two-dimensional imaging module (4) and the three-dimensional imaging module (5) are respectively arranged on the first station and the second station on the motion trajectory of the turntable (2). The two-dimensional imaging module (4) is used to perform two-dimensional imaging of the culture target at the first station, and the three-dimensional imaging module (5) is used to perform three-dimensional imaging of the culture target at the second station. The control module (7) controls the drive module (6) to drive the turntable (2) to switch between the first workstation and the second workstation.

4. The time-difference incubator integrating two-dimensional and three-dimensional imaging according to claim 1, characterized in that, The two-dimensional imaging module (4) and the three-dimensional imaging module (5) adopt a coaxial integrated structure; The coaxial integrated structure includes a dichroic beam splitter and a common objective lens. The dichroic beam splitter is used to combine or split the two-dimensional imaging optical path and the three-dimensional imaging optical path, so that the two imaging modes can share the common objective lens.

5. The time-difference incubator integrating two-dimensional and three-dimensional imaging according to claim 4, characterized in that, The two-dimensional imaging module (4) includes a modulation component and a removal component, wherein the removal component is connected to the modulation component; The modulation component is used to modulate the imaging light to form illumination conditions for two-dimensional imaging. The removal component is used to put the modulation component in the working position when the two-dimensional imaging module (4) is working, and to remove the modulation component from the working position when the three-dimensional imaging module (5) is working.

6. A control method for a time-difference incubator integrating two-dimensional and three-dimensional imaging, characterized in that, The control method is applicable to the time difference incubator as described in any one of claims 1-5, and the control method includes: The two-dimensional imaging module (4) is controlled to acquire two-dimensional images of the culture target and obtain two-dimensional images; The two-dimensional image is processed to obtain the location information and three-dimensional scanning information of the culture target; Based on the location information and three-dimensional scanning information, the three-dimensional imaging module (5) is controlled to acquire three-dimensional images of the culture target to obtain three-dimensional images; The two-dimensional and three-dimensional images are subjected to time correlation and fusion processing to obtain the time difference sequence of the training target.

7. The control method for the time-difference incubator integrating two-dimensional and three-dimensional imaging according to claim 6, characterized in that, The image processing of the two-dimensional image to obtain the location information and three-dimensional scan information of the culture target includes: The two-dimensional image is preprocessed to obtain a preprocessed two-dimensional image; Using a preset contour extraction method, the contour region of the culture target in the preprocessed two-dimensional image is extracted; Using a preset position algorithm and the contour region, the centroid coordinates of the culture target are calculated, and the position information of the culture target in the two-dimensional image is determined based on the centroid coordinates. Based on the pre-established coordinate calibration relationship, the position information is converted from two-dimensional image coordinates to the physical coordinates of the culture target, and the physical coordinates are converted to the scanning coordinates of the three-dimensional imaging module to obtain the corresponding three-dimensional scanning information.

8. The control method for the time-difference incubator integrating two-dimensional and three-dimensional imaging according to claim 7, characterized in that, The step of controlling the three-dimensional imaging module (5) to acquire three-dimensional images of the culture target based on the location information and three-dimensional scanning information, and obtaining three-dimensional images, includes: Based on the location information and the preset calibration transformation matrix, the galvanometer scanning coordinates of the three-dimensional imaging module (5) are obtained; Based on the response relationship between the galvanometer scanning coordinates and the galvanometer deflection angle and the driving voltage, the galvanometer control voltage is obtained; The starting point and scanning range of the three-dimensional scan are determined based on the three-dimensional scan information; Based on the galvanometer control voltage, three-dimensional scanning start point and scanning range, the three-dimensional imaging module (5) is controlled to acquire three-dimensional images of the culture target and obtain three-dimensional images.

9. The control method for the time-difference incubator integrating two-dimensional and three-dimensional imaging according to claim 8, characterized in that, The step of performing time correlation and fusion processing on the two-dimensional and three-dimensional images to obtain the time difference sequence of the training target includes: The two-dimensional and three-dimensional images are processed according to a preset indexing rule to obtain a first sequence based on the training objective; Based on the pre-established coordinate calibration relationship, the two-dimensional and three-dimensional images in the first sequence are spatially registered so that the same cultivation target is aligned in the two-dimensional image coordinates and the three-dimensional image coordinates to obtain the second sequence; Based on the second sequence, the two-dimensional texture information in the two-dimensional image is mapped to the corresponding spatial position in the three-dimensional image to obtain a three-dimensional model that integrates the two-dimensional texture information; By associating the two-dimensional image with the three-dimensional model in chronological order, the time difference sequence of the training target is obtained.

10. The control method for the time-difference incubator integrating two-dimensional and three-dimensional imaging according to claim 6, characterized in that, It also includes, before the two-dimensional imaging module (4) acquires two-dimensional images of the culture target, performing visual servo centering and hierarchical adaptive focusing processing on the culture dish (3) to obtain the spatial alignment state and imaging focal plane parameters of the culture dish (3) and the two-dimensional imaging module (4); The visual servo centering and hierarchical adaptive focusing processing obtains the spatial alignment state and imaging focal plane parameters of the petri dish (3) and the two-dimensional imaging module (4), including: The geometric center of the through hole is determined based on the through hole boundary information of the culture dish (3) in the two-dimensional image, and compared with the current field center of the two-dimensional imaging module (4) to obtain the position deviation. The position of the culture dish (3) is corrected based on the position deviation, so that the geometric center of the through hole is aligned with the current field of view, and the spatial alignment state of the through hole of the culture dish (3) and the two-dimensional imaging module (4) is obtained. Determine the reference through hole, perform coarse focusing on the reference through hole, determine the optimal focal plane position within a preset travel range, and determine it as the reference focal plane; Based on the reference focal plane and the preset adaptive interval algorithm, a narrow band interval centered on the reference focal plane is constructed; Based on the narrow band range, fine focusing is performed on the remaining through holes to obtain the imaging focal plane parameters corresponding to each through hole.

Images