An automated imaging method and system and apparatus therefor

Abstract

The application discloses an automatic imaging method and a system and equipment thereof. The application is characterized in that a sample interference surface and a sample boundary are positioned, and then non-equidistant three-dimensional OCT image acquisition is performed. The non-equidistant acquisition strategy based on the sample structure features can automatically adjust the acquisition step according to the complexity of different regions. In the Non-ROI region, a large step is adopted for the sample arm to realize fast imaging, and in the ROI region, a small step is adopted for the sample arm to keep high-resolution imaging, so that the redundant data acquisition is reduced and the acquisition efficiency is improved. The dynamic acquisition strategy can ensure high-resolution imaging of the region of interest while significantly shortening the total imaging time.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of full-field optical coherence tomography, and particularly relates to an automatic imaging method, system and device. BACKGROUND

[0002] Optical coherence tomography (OCT) systems, especially full-field optical coherence tomography (FF-OCT), play an important role in the field of biomedical imaging. They have advantages such as high resolution and no labeling, and can perform three-dimensional imaging of biological samples at the micron level, providing real-time histological information for biological tissue research. However, traditional OCT systems still face a series of challenges in application:

[0003] Long imaging time: Traditional OCT systems take a long time to image complex heterogeneous and irregular samples such as three-dimensional cell culture models due to the use of equidistant scanning. This not only limits the efficiency of real-time imaging, but also may have adverse effects on sample activity due to long imaging time, thereby interfering with the accuracy of experimental results.

[0004] Collection efficiency and data storage problems: Existing research focuses on improving the imaging speed of OCT systems, but ignores the optimization of collection efficiency. Taking the FF-OCT system as an example, when faced with complex three-dimensional cell culture models, its fixed step-by-step collection method leads to low collection efficiency and generates a large amount of data, which puts a heavy burden on subsequent data processing and storage.

[0005] Drawbacks of manual operation for interference surface positioning: The positioning of the interference surface in traditional OCT systems usually relies on manual operation, and the operator needs to observe the interference fringes with the naked eye to determine the position of the interference surface. This method not only requires high experience and skill of the operator, but also introduces human error due to the large number of layers caused by high-resolution layer-by-layer scanning. Especially for active biological samples such as cell spheres or organoids in normal culture, due to their uneven size and distribution, they will produce motion artifacts when floating in the culture medium, making it difficult to directly observe the interference fringes, and thus causing imaging difficulties, and the accuracy and efficiency of manual positioning are greatly reduced.

[0006] In summary, traditional OCT systems face challenges such as long imaging time, low collection efficiency, and inaccurate interference surface positioning when imaging complex samples. There is an urgent need for a more efficient, accurate, and highly automated imaging method and related system and device to overcome these limitations. SUMMARY

[0007] The present application aims to overcome the deficiencies of the prior art, and provide an automatic imaging method, system and device.

[0008] The present application is implemented in the following manner. In a first aspect, the present application provides an automatic imaging method, which employs non-equidistant three-dimensional OCT image acquisition.

[0009] The non-equidistant three-dimensional OCT image acquisition is performed by reconstructing OCT images from focal plane layer images under different phase shift amounts; the current sample focal plane is screened for a region of interest and a region of non-interest according to the gray variance of the OCT images; the sample arm is moved to the next focal plane layer with a small step size in the region of interest, and the sample arm is moved to the next focal plane layer with a large step size in the region of non-interest.

[0010] Preferably, in the non-equidistant three-dimensional OCT image acquisition, whether the current focal plane layer is still being acquired is identified according to a comparison between the number of OCT images and the size of a sliding window; if the number of OCT images is greater than the size of the sliding window, the sample arm is moved to the next focal plane layer with a large step size.

[0011] Preferably, before the non-equidistant three-dimensional OCT image acquisition is performed, the sample interference surface needs to be located, and the focal plane layer needs to be located at the sample boundary.

[0012] Preferably, the locating of the sample interference surface comprises:

[0013] The OCT images are reconstructed from focal plane layer images under different phase shift amounts; a first sliding window change rate is calculated according to the gray variance of the OCT images; if the first sliding window change rate is greater than a preset threshold value ξ1, it is considered that the current sample focal plane is interfered; if the first sliding window change rate is less than or equal to the preset threshold value ξ1, the reference arm is moved in the direction of increasing optical path.

[0014] More preferably, in the locating of the sample interference surface, whether the current focal plane layer is still being acquired is identified according to a comparison between the number of OCT images and the size of a sliding window; if the number of OCT images is less than or equal to the size of the sliding window, the reference arm is moved in the direction of increasing optical path.

[0015] Preferably, the locating of the focal plane layer at the sample boundary comprises:

[0016] A second sliding window change rate is calculated according to the image definition of the focal plane layer images;

[0017] If the second sliding window change rate is less than a preset threshold value ξ3, it is considered that the current focal plane layer is the sample boundary; if the second sliding window change rate is greater than or equal to the preset threshold value ξ3, the sample arm is moved to the next focal plane layer.

[0018] More preferably, the positioning focal plane layer is in the sample boundary, according to the comparison between the current focal plane layer image quantity and the preset threshold value ξ2, it is identified whether the current focal plane layer is still in the current focal plane layer, and if the current focal plane layer image quantity is less than or equal to the preset threshold value ξ2, the sample arm is moved to the next focal plane layer.

[0019] Preferably, the sample arm moving direction in the non-equidistant three-dimensional OCT image acquisition process is opposite to the sample arm moving direction in the process of positioning the focal plane layer in the sample boundary.

[0020] In a second aspect, the present application provides an automatic imaging system, comprising:

[0021] A sample interference surface positioning module is responsible for positioning the sample interference surface, and realizes the equal optical path of the sample arm and the reference arm;

[0022] A sample boundary positioning module is responsible for positioning the focal plane layer in the sample boundary;

[0023] An image acquisition module is responsible for non-equidistant three-dimensional OCT image acquisition of the sample.

[0024] In a third aspect, the present application provides an automatic imaging device, comprising an illumination module, an imaging module and a control module, wherein the imaging module comprises a reference arm module, a sample arm module and a detection arm module;

[0025] The illumination module provides a low coherence light source, and then sends the light into the reference arm module and the sample arm module respectively; the reflected light returned by the sample arm module and the reflected light returned by the reference arm module interfere with each other and enter the detection arm module; the detection arm module acquires the interference signal;

[0026] The illumination module comprises a low coherence light source, a first convex lens f1, an aperture stop AS, a variable field stop FS, a second convex lens f2 and a beam splitting cube prism BS; the light emitted by the low coherence light source is focused by the first convex lens f1 to form an image at the aperture stop AS, the light source image point passes through the variable field stop FS, and then enters the beam splitting cube prism BS through the second convex lens f2 behind;

[0027] The sample arm module comprises a first objective lens MO and a first displacement table; a culture dish for placing the sample to be measured is arranged on the first displacement table, and the objective lens is located above the culture dish of the first displacement table;

[0028] The reference arm module comprises a filter ι, a second objective lens MO, a reference mirror, a phase shifter and a second displacement table which are connected in sequence through an optical path; the phase shifter is used for phase shifting to realize the acquisition of the current focal plane layer image with different phase shifting amounts; and the second displacement table is used for moving the reference arm;

[0029] The probe arm module comprises a third convex lens f3 and a camera; interference light is clearly imaged on the camera through the third convex lens f3; and the first displacement table is used to move the sample arm.

[0030] The control module is used to control the first displacement table, the second displacement table, the phase shifter and the camera.

[0031] The present application has the following advantages:

[0032] The present application uses an automatic interference surface detection method, significantly reduces the variability caused by manual operation and human factors, improves the repeatability of the focal surface detection and the reproducibility of the experiment, and can realize rapid and accurate interference surface positioning under the support of an automatic control system.

[0033] The present application also proposes a non-equidistant collection strategy based on the structural characteristics of a sample, which can automatically adjust the collection step according to the complexity of different regions, uses a large step for the sample arm in the Non-ROI region to realize rapid imaging, and uses a small step for the sample arm in the ROI region to maintain high-resolution imaging, thereby reducing redundant data collection and improving collection efficiency. This dynamic collection strategy significantly shortens the total imaging time while ensuring high-resolution imaging of the region of interest.

[0034] In addition, the present application is not limited to imaging a three-dimensional cultured cell model with obvious structural stratification, and is also applicable to imaging of other complex samples with obvious optical scattering characteristics, such as imaging of the internal structure of a three-dimensional cultured cell model. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0036] Figure 1 is a structural schematic diagram of an OCT automatic imaging device provided by the embodiment of the present application.

[0037] Figure 2 is a total flowchart of an OCT automatic imaging method provided by the embodiment of the present application.

[0038] Figure 3 is a flowchart of positioning a sample interference surface in an OCT automatic imaging method provided by the embodiment of the present application.

[0039] Figure 4 is a flowchart of positioning a sample boundary in an OCT automatic imaging method provided by the embodiment of the present application.

[0040] Figure 5 is a non-equidistant three-dimensional OCT image acquisition flowchart in the OCT automatic imaging method provided by the embodiment of the application.

[0041] Figure 6 is a verification result of the OCT automatic imaging method provided by the embodiment of the application, wherein (a)-(c) are microscope images and interference images of a culture dish bottom, an onion inner epidermis and a RG cell three-dimensional culture model respectively; (d) is a gray scale variance change curve in the interference surface detection process; (e) is a brenner gradient value change curve in the boundary detection process; (f)-(g) are gray scale variance value change curves in the equidistant acquisition and non-equidistant acquisition processes respectively. DETAILED DESCRIPTION

[0042] The interference surface positioning of the conventional OCT system depends on manual operation, which requires observing the interference fringes by naked eyes to determine the accurate position of the interference surface, and due to the need of high-resolution layer-by-layer scanning, the number of layers is large, the imaging time is long, and human errors are prone to occur. For the cell spheres or organoids in normal culture and other active biological samples, the size and distribution are uneven, and the floating in the culture medium will cause motion artifacts, and it is particularly difficult for complex sample imaging.

[0043] To solve this problem, the application adopts an automatic imaging method, which automatically moves the reference arm displacement table by controlling the reference arm displacement table and combines the phase shifter to change the phase to collect images, and uses the gray scale variance calculation and the sliding window change rate to determine the interference surface position. For example, in the HepaRG cell three-dimensional culture model experiment, the reference arm displacement table automatically steps in the optical path increasing direction by, for example, 10 μm, and the interference surface is accurately found in a short time (within one minute), while the conventional manual operation takes a longer time and the accuracy is easily affected by the experience of the operator, thereby effectively reducing human errors and improving the positioning efficiency.

[0044] In the sample boundary detection, the application uses the image definition calculation and the sliding window change rate to determine the sample boundary. During the automatic movement of the sample arm, the definition values of the images at different positions are calculated, and the sample boundary can be accurately found. For example, in the boundary detection of the HepaRG cell three-dimensional culture model, the sample arm automatically steps down by, for example, 2 μm, and the boundary is determined according to the definition value change trend (the image gradually blurs, and the gradient value tends to be stable), so that the subsequent acquisition can obtain the complete three-dimensional structure of the sample.

[0045] The application also proposes a non-equidistant acquisition strategy based on sample structure characteristics. According to the structural differences between ROI and Non-ROI regions, the sample arm step distance is automatically adjusted by analyzing the gray variance value change of the OCT image. In the Non-ROI region, large step (such as 10 um) fast imaging reduces the data volume, and in the ROI region, small step (such as 2 um) high-resolution imaging retains key information. Compared with the traditional equidistant acquisition, in the imaging of HepaRG cell three-dimensional culture model, the non-equidistant acquisition greatly reduces the data acquisition in the cell sparse structure region, maintains high resolution in the key area such as the bottom cluster structure, effectively balances the imaging speed and quality, improves the collection efficiency, reduces the data storage burden, and is suitable for imaging of various complex samples, with wide application range.

[0046] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application.

[0047] As shown in Figure 1 The application embodiment provides an automatic imaging equipment, which comprises an illumination module, an imaging module and a control module, the imaging module comprises a reference arm module, a sample arm module and a detection arm module.

[0048] The illumination module provides a low coherence light source, and then sends the light into the reference arm module and the sample arm module respectively; the reflected light returned by the sample arm module and the reflected light returned by the reference arm module interfere with each other and enter the detection arm module; and the detection arm module acquires the interference signal.

[0049] The illumination module comprises a low coherence light source, a first convex lens f1, an aperture stop AS, a variable field stop FS, a second convex lens f2 and a beam splitting cube prism BS; the illumination module serves as a light source supply part of the whole imaging system, light emitted by the core component low coherence light source of the illumination module is firstly subjected to focusing imaging operation by the first convex lens f1. This focusing process enables the light emitted by the light source to form a clear image point at the aperture stop AS, and the aperture stop AS is used to limit the aperture size of the light beam, thereby effectively controlling the light energy entering the subsequent optical system and helping to reduce the interference of stray light and improve the contrast and clarity of imaging. Then, the light source image point passing through the aperture stop AS continues to propagate and passes through the variable field stop FS, and the variable field stop FS can flexibly adjust the field size according to the actual imaging requirement, so as to adapt to the imaging of samples with different sizes and structures. Finally, the light passes through the postposed second convex lens f2 and enters the beam splitting cube prism BS for beam splitting operation, and the split light is sent into the reference arm module and the sample arm module respectively, thereby providing the necessary light beam basis for the subsequent interference imaging process.

[0050] The sample arm module comprises a first objective lens MO, a first displacement table; a culture dish for placing a sample to be tested is arranged on the first displacement table, and the first objective lens MO is located above the culture dish of the first displacement table; the first objective lens MO in the sample arm module can converge and collimate the light coming from the illumination module, so as to ensure that the light uniformly irradiates the sample to be tested (such as a biological sample of a cell ball) placed on the first displacement table at a suitable angle and intensity. The first displacement table is a device capable of accurately controlling the position of the sample, and by adjusting the position, the movement of the sample in the optical axis direction can be realized, which is crucial for finding different focal plane layers of the sample and realizing three-dimensional imaging. During imaging, the main task of the sample arm module is to accurately transmit the light signal reflected by the sample back to the detection arm module, so as to interfere with the light returned by the reference arm module.

[0051] The reference arm module comprises a filter I, a second objective lens MO, a reference mirror RM, a phase shifter and a second displacement table connected in sequence through an optical path; the phase shifter is used for phase movement to realize the acquisition of current focal plane layer images of different phase shift amounts; and the second displacement table is used for movement of the reference arm. The filter I in the reference arm module is mainly used for screening light of a specific wavelength range, so as to ensure that the spectral characteristics of the reference light meet the requirements of the imaging system and reduce the influence of unnecessary spectral components on the imaging result. The second objective lens MO also plays a role in converging and collimating light rays, so that the reference light can irradiate the reference mirror RM in the form of parallel light. The function of the second displacement table in the reference arm module is to accurately control the length of the reference arm, and the optical path matching is realized by changing the length of the reference arm, which is one of the key steps for obtaining high-quality interference signals. The phase shifter is a key component in the reference arm module, which can realize the acquisition of current focal plane layer images of different phase shift amounts by changing the phase of the reference light, and provide rich phase information for subsequent image reconstruction and analysis.

[0052] The detection arm module comprises a third convex lens f3 and a camera; the interference light is clearly imaged on the camera through the third convex lens f3; the first displacement table is used for movement of the sample arm; the third convex lens f3 in the detection arm module is responsible for focusing and imaging the light signal returned from the sample arm module and the reference arm module and subjected to interference, so that the interference signal can be clearly imaged on the camera. The camera, as the final receiving device of the imaging system, converts the light signal into an electric signal and records the intensity and phase information of the interference image. These image data will be transmitted to the control module for subsequent processing and analysis.

[0053] The control module realizes precise control of the whole imaging system through the host computer and the controller. It can control the moving speed, displacement and moving direction of the first displacement table, so as to realize the precise positioning and movement of the sample arm in three-dimensional space. For the second displacement table, the control module can precisely adjust the displacement to ensure the accurate change of the reference arm length, so as to realize the optical path matching. At the same time, the control module can also control the phase displacement and speed of the phase shifter, as well as the exposure time, acquisition frame rate and other parameters of the camera, to ensure the automation, precision and high efficiency of the whole imaging process.

[0054] For example, the light of the sample arm uniformly illuminates the sample after passing through the first objective lens MO. The light of the reference arm is emitted as parallel light to the reference mirror RM after passing through the filter and the second objective lens MO. The sample arm and the reference arm are adjusted to make the sample light and the reference light interfere, and the interference light is clearly imaged on the camera through the third convex lens f3.

[0055] For example, the automatic imaging device is suitable for an OCT system, which can be an FF-OCT, a time-domain OCT, a frequency-sweeping OCT or a spectral-domain OCT imaging system that can realize tomographic scanning, and is also suitable for a laser confocal microscope and a light sheet microscope system that scans layer by layer. The specific adjustment can be made according to the actual situation.

[0056] The embodiment also provides an automatic imaging method based on the above device, taking FF-OCT as an example, which includes the following steps: Figure 2 The steps include:

[0057] Step S1, positioning the sample interference surface to realize equal optical path of the sample arm and the reference arm;

[0058] Step S2, positioning the focal plane layer at the sample boundary;

[0059] Step S3, collecting non-equidistant three-dimensional FF-OCT images of the sample;

[0060] Specifically, before the automatic search of the sample interference surface, the strong backscattering signal stripe of the culture dish bottom is used for positioning, and then the sample arm first displacement table is moved to search for the sample position and the reference arm second displacement table is moved to find the sample interference surface. Figure 3 The specific method steps are as follows:

[0061] S11, finding the interference stripe by using the culture dish bottom surface;

[0062] First, place the culture dish containing the sample, such as cell balls, above the first objective lens. Adjust the focal plane to find the cell ball sample in the field of view. Then move the sample arm to the first displacement stage to find the focal plane of the culture dish bottom. After that, move the reference arm to the second displacement stage to find the interference plane of the culture dish bottom. When the interference fringes are observed, it indicates that the coherence gate and the focal plane are coincident.

[0063] S12, move the sample arm to find the clear focal plane of the sample:

[0064] Move the sample arm displacement stage to find the clear focal plane of the cell balls. At this time, the light irradiated to the cell balls is refracted through the culture dish bottom, so the actual optical path of the sample arm increases, while the optical path of the reference arm does not change at this time, resulting in a larger optical path difference l0 between the sample arm and the reference arm. l0 is greater than the coherence length of the light source, which prevents interference from occurring. Therefore, in order to ensure subsequent stable and clear imaging, the reference arm needs to be adjusted in the direction of increasing optical path.

[0065] The culture dish bottom usually has a relatively flat structure, and the reflected light can form relatively stable and easily observed interference fringes, which provides a reference position for subsequent search for the sample interference plane. After finding the interference fringes of the culture dish bottom, the clear focal plane of the sample is found by moving the sample arm. This process requires precise control of the movement of the sample arm to ensure that the sample is in the best imaging position.

[0066] Compared to the planar structure of the culture dish bottom, the cell balls are complex in structure, and the interference fringes are difficult to directly observe. Therefore, the following steps S13-S15 are performed.

[0067] S13, phase shift by the reference arm phase shifter, and collect the current focal plane layer images of different phase shift amounts; perform static reconstruction on the focal plane layer images under different phase shift amounts to calculate the FF-OCT image of the sample;

[0068] Since the FF-OCT system itself collects layer images, the current focal plane layer images of different phase shift amounts can be directly collected by phase shifting. If other OCT systems are used, such as for time-domain OCT systems, the scanning mode is A-scan. In order to realize the imaging of layer images, the galvanometer needs to be controlled to scan in the X and Y directions to obtain the surface image. For frequency-sweeping OCT and spectral-domain OCT systems, the scanning mode is B-scan. In order to realize the imaging of layer images, the galvanometer needs to be controlled to scan in the X direction to obtain the surface image.

[0069] S14, update the number of FF-OCT images n = n + 1, and then determine whether n is greater than the sliding window size. If yes, go to step S15, and if no, move the reference arm in the direction of increasing optical path by a step w1, and then return to step S13. In this embodiment, w1 is 10 μm.

[0070] S15, calculate the gray scale variance of the current focal plane layer FF-OCT image; calculate the first sliding window change rate according to the gray scale variance, and judge whether the current first sliding window change rate is greater than a preset threshold ξ1 (the value of this embodiment is 0.05, which can be replaced according to experience), if yes, it is considered that the current sample focal plane interferes, if not, the reference arm is moved in the direction of increasing optical path length by a step w1, and then returns to step S13.

[0071] Step S13 is specifically: the host computer controls the phase shifter of the reference arm to change the optical path difference between the sample arm and the reference arm, and the camera collects one image of the focal plane layer each time the phase shifter is shifted.

[0072] (1)

[0073] In the formula, is a direct current term, is the sample FF-OCT image information to be solved, is a phase difference, is a phase shift introduced by the reference arm phase shifter.

[0074] The static reconstruction in step S13 can be realized by four-step phase shift interference method, and can also be realized by two-step, three-step, five-step or six-step phase shift interference method. The specific imaging mode can be selected according to the application scene of the system and combined with the imaging accuracy and efficiency.

[0075] For example, the four-step phase shift interference method is specifically to collect Four pictures under four phase shifts are respectively The FF-OCT image of the sample can be represented as

[0076] (2)

[0077] In the process of moving the reference arm displacement table, the FF-OCT image containing the current layer information of the cell ball can be reconstructed by using the four-step phase shift method. The effective information of the FF-OCT image is calculated to determine whether the interference surface of the cell ball is found, that is, whether the cell ball reaches the position where the coherence gate and the focal plane coincide.

[0078] In step S15, the first sliding window change rate is calculated according to the gray scale variance, wherein the gray scale variance is calculated as follows:

[0079] (3)

[0080] In the formula, and respectively represent the horizontal pixel number and the vertical pixel number of the image, represents the pixel value, represents the average value of all pixels in the image. If the calculated gray scale variance value changes abruptly, it indicates that the sample arm and the reference arm interfere at this position, i.e. the FF-OCT image contains effective information, indicating that the interference surface of the cell ball is found, and the more effective information, the greater the gray scale variance value.

[0081] first sliding window change rate The calculation is as follows:

[0082] (4)

[0083] where abs() represents the absolute value function; represents the average value of the gray scale variance in the sliding window windows1; represents the average value of the gray scale variance in the sliding window windows2; the size of the sliding window windows1 and the sliding window windows2 is equal, and the position of window2 is one bit ahead of the position of window1, and the relative positions of the two windows remain unchanged when sliding and gradually move to the right.

[0084] After finding the interference surface of the cell ball, the three-dimensional data of the cell ball can be collected by moving the sample arm displacement stage and cooperating with the camera and the phase shifter, and the three-dimensional structure of the cell ball can be reconstructed by processing and analyzing the data. However, at this time, the focal surface of the first objective of the sample arm and the coherence gate of the reference arm are located inside the cell ball rather than at the boundary, so in order to collect the complete three-dimensional structure of the cell ball, it is necessary to move the sample arm displacement stage to place the focal surface and the coherence gate at the boundary of the cell ball.

[0085] Specifically, as Figure 4 The specific steps of step S2 include:

[0086] S21, collect the current focal plane layer image;

[0087] S22, update the number of current focal plane layer images m = m + 1, and then judge whether m is greater than a preset threshold ξ2, if yes, step S23, if not, move the sample arm to the next focal plane layer by a step w2, and return to step S21; w2 is 2 μm in this embodiment; m = n, n represents the number of FF-OCT images; the threshold value ξ2 in this embodiment can be taken as ξ2 = 2Q-1, wherein Q represents the size of the sliding window;

[0088] S23, calculate the image sharpness of the current focal plane layer image; calculate the second sliding window change rate according to the image sharpness, and judge whether the current second sliding window change rate is less than a preset threshold ξ3 (the value of this embodiment is 0.005, which can be replaced according to experience), if yes, it is considered that the current focal plane layer is the sample boundary, and if not, the sample arm is moved to the next focal plane layer by a step w2, and then returns to step S21.

[0089] In this embodiment, the image sharpness adopts the brenner gradient function. Since the focal plane and the coherence gate position are always unchanged during the movement of the sample arm displacement table, the focal plane and the coherence gate are placed at the boundary of the cell ball by moving the cell ball position. At this time, the cell ball is gradually out of focus in the picture collected by the camera until it is blurred. By judging the sharpness of the pictures collected at different positions, it can be judged whether the cell ball is moved to the predetermined position. The brenner gradient value of each image collected by the camera is calculated as

[0090] (5)

[0091] As the image becomes more and more blurred, the brenner gradient value calculated by the image also tends to be stable, and the stable place of the function value is found by using the sliding window, that is, the defocus position.

[0092] For the evaluation function of image sharpness, the application is not limited to the Brenner gradient function, but also includes Tenengrad gradient function, Laplacian gradient function and gray variance function and various image sharpness evaluation functions. The optimal evaluation function can be selected according to the application scene and combined with accuracy and efficiency.

[0093] The second sliding window change rate is calculated as follows:

[0094] (6)

[0095] Wherein, abs() represents the absolute value function; represents the average value of the image sharpness in the sliding window windows1; represents the average value of the image sharpness in the sliding window windows2; the size of the sliding window windows1 and the sliding window windows2 is equal, the initial position of window1 is kept unchanged, and only window2 is gradually slid when sliding.

[0096] Specifically, step S3 is based on the ROI non-equidistant three-dimensional FF-OCT synchronization timing automatic acquisition protocol related to the sample, so as to realize the improvement of the acquisition efficiency. The acquisition process is controlled by the upper computer to control the sample arm first displacement table, the reference arm phase shifter and the camera to work cooperatively and run synchronously. The specific steps include:

[0097] S31, collect current focal plane layer images of different phase shift amounts; perform static reconstruction through the focal plane layer images under different phase shift amounts to calculate the FF-OCT image of the sample; the static reconstruction method herein can refer to step S13;

[0098] S32, update the number n of FF-OCT images to n+1, and then determine whether n is greater than the size of the sliding window; if yes, proceed to step S33, and if not, move the sample arm to the next focal plane layer by a step w3 and return to step S31; in this embodiment, w3 is 10 μm;

[0099] S33, calculate the gray scale variance of the current focal plane layer FF-OCT image; construct a gray scale variance value change curve according to the gray scale variance, and screen out a region of interest (ROI) and a non-region of interest (Non-ROI) of the current sample focal plane; move the sample arm to the next focal plane layer by a step w4 in the region of interest and return to step S31; move the sample arm to the next focal plane layer by a step w3 in the non-region of interest (Non-ROI) and return to step S31; in this embodiment, w4 is 2 μm;

[0100] According to the gray scale variance, a third sliding window change rate is calculated, and it is determined whether the current third sliding window change rate is greater than a threshold value ξ4 (in this embodiment, the value is 0.001, which can be replaced according to experience); if yes, the current sample focal plane reaches the other boundary of the sample, and the image collection is stopped; if not, move the sample arm to the next focal plane layer by a step w4 and return to step S31.

[0101] The third sliding window change rate is calculated as follows:

[0102] (7)

[0103] The settings of the sliding windows windows1 and windows2 are the same as the first sliding window change rate, that is, the sizes of the sliding windows windows1 and windows2 are equal, the position of window2 is one position ahead of the position of window1, and the relative positions of the two windows remain unchanged when sliding.

[0104] The moving direction of the sample arm to the next focal plane layer in the above steps S32 and S33 is opposite to the moving direction of the sample arm to the next focal plane layer in step S2.

[0105] In step S3, after determining the overlap of the coherence gate and focal plane, as well as the sample boundary position, four different phase images are acquired at the current position. Then, the reference arm displacement stage moves towards the other end of the sample with large steps. With each step, the camera and phase shifter collaboratively acquire four different phase images. Simultaneously, a four-step phase-shifting method is used for static reconstruction to obtain an FF-OCT image, and the gray-level variance of the FF-OCT image is calculated. When the sample overlaps with the coherence gate, the FF-OCT image contains structural information of the current layer of the sample. At this time, the gray-level variance value calculated from this FF-OCT image will undergo a sudden change compared to before, and the gray-level variance value increases as the FF-OCT image contains more structural information. Therefore, for complex biological samples, such as organoids, non-equidistant acquisition of the ROI of the biological sample structure can be achieved based on the amount of effective information contained in the FF-OCT images of different layers. Large steps are used in non-ROI regions to increase the acquisition speed, while small steps are used in ROI regions to retain more effective information.

[0106] To achieve non-isolated 3D acquisition, it is necessary to automatically distinguish the boundaries between ROI and Non-ROI. The boundaries between ROI and Non-ROI are determined by utilizing the structural differences between ROI and Non-ROI regions, i.e., the changes in the gray-level variance values ​​of the FF-OCT images acquired from ROI and Non-ROI regions. Figure 6 (b) Figure 6 (d) shows the gray-scale variance value change curves of three FF-OCT images from the cell spheroid model detection experiment. The black dots represent the boundaries of ROI and Non-ROI. In step S33, the gray-scale variance value change curve of the FF-OCT image is constructed based on the gray-scale variance to screen out the region of interest (ROI) and non-ROI of the current sample focal plane. Specifically, based on the gray-scale variance value change curve of the FF-OCT image, it is checked whether each gray-scale variance value meets one of the boundary change conditions (5). If it meets the condition, it is considered that the current sample focal plane has reached the boundary of the ROI. The sample arm is moved to the next focal plane layer with a step size w4, and then the process returns to step S31. If the condition does not meet the condition, the sample arm is moved to the next focal plane layer with a step size w3, and then the process returns to step S31.

[0107] The specific boundary change conditions are:

[0108] (8)

[0109] In the formula , Let x and y represent the grayscale variance values ​​of the x-th OCT image, respectively. The first-order derivative and the second-order derivative.

[0110] If condition 1 above is satisfied, then If the current gray scale variance value change curve is a falling mark point, the change curve starts to show a downward trend; if condition 2 is met, as above, then If the current gray scale variance value change curve is a horizontal mark point, the change curve starts to show a horizontal gentle trend; if condition 3 is met, as above, then If the current gray scale variance value change curve is an upward mark point, the change curve starts to show an upward trend.

[0111] The method can accurately find the boundary between different structures, that is, the boundary between the ROI and the Non-ROI region, and then change the moving step length of the sample arm first displacement table, so as to realize non-equidistant three-dimensional FF-OCT image acquisition.

[0112] Therefore, by using the above imaging method, the three-dimensional cell culture model is imaged, and since the development state of the three-dimensional cell culture model can be represented by morphological characteristics, in order to more accurately quantify the morphological characteristics of the three-dimensional cell culture model, the obtained FF-OCT image needs to be preprocessed. The FF-OCT image is automatically segmented by using a pre-trained YOLOv8 neural network model to obtain a precisely segmented three-dimensional cell culture model image. The FF-OCT image data is reconstructed in three dimensions by using Amira software, the basic structure of the three-dimensional model is reproduced, and the morphological characteristics are accurately evaluated. At the same time, based on connected domain calculation, the morphological parameters such as the volume and surface area of the reconstructed three-dimensional cell culture model are quantified. The specific steps are as follows:

[0113] (1) The step S3 FF-OCT image is preprocessed, each X-Y image after preprocessing is binarized, then hole filling is performed, and the filled image is saved as three-dimensional image data.

[0114] (2) The three-dimensional data after hole filling is labeled by connected domain, and an independent label is generated for each connected domain.

[0115] (3) The actual volume pixel number and actual surface area pixel number of each connected domain are calculated.

[0116] (4) The single-pixel volume and single-pixel surface area are obtained through the lateral resolution and longitudinal resolution, and then the actual volume and actual surface area are calculated.

[0117] Since FF-OCT has great advantages in detecting three-dimensional cell culture models, various morphological parameters of the three-dimensional cell culture model can be analyzed from the obtained images, including volume, surface area, etc., and the main morphological parameter calculation formula is:

[0118] (1) Volume of a single three-dimensional cell culture model

[0119] (9)

[0120] wherein, represents the actual volume of the th three-dimensional cell culture model, represents the number of volume pixels collected by the th three-dimensional cell culture model with a step size of represents the step size of the th three-dimensional cell culture model, represents the lateral resolution.

[0121] (2) Surface area of a single three-dimensional cell culture model

[0122] (10)

[0123] wherein, represents the actual surface area of the th three-dimensional cell culture model, represents the number of surface area pixels collected by the th three-dimensional cell culture model with a step size of

[0124] (3) Total volume of three-dimensional cell culture models

[0125] (11)

[0126] wherein, represents the total volume of three-dimensional cell culture models.

[0127] (4) Total surface area of three-dimensional cell culture models

[0128] (12)

[0129] wherein, represents the total surface area of three-dimensional cell culture models.

[0130] In order to evaluate the universality of the automatic interference surface detection and non-equidistant three-dimensional OCT image acquisition of the OCT automatic imaging method, first, the FF-OCT imaging experiment of the HepaRG cell three-dimensional culture model grown in the microporous hydrogel was carried out. This embodiment focuses on verifying the difference between the method of the present application and the traditional manual interference surface detection method, and comparing the difference in efficiency between the traditional equidistant acquisition and non-equidistant acquisition of the FF-OCT system.

[0131] The backscattering absorption signal of the HepaRG cell three-dimensional culture model is weak, and the interference fringes are difficult to observe by the human eye, Figure 6 (a)- Figure 6 ​​Figure 3(c) shows the comparison of the interference fringes before and after the interference of the bottom surface of the petri dish, the inner epidermis of the onion and the HepaRG cell 3D model cultured in the micro-well hydrogel. It can be seen that the interference fringes of the HepaRG cell 3D model are hardly observed directly. Using the automated interference surface detection algorithm, according to Figure 3 Figure 3(d) shows the gray level variance curve of the FF-OCT images collected during the automated interference surface positioning process. The two sub-figures represent the FF-OCT images at the beginning and end positions of the interference surface detection process, respectively. It can be seen that the gray level variance value of the end position FF-OCT image changes abruptly because it contains structural information. Therefore, it is determined that the focal plane and the interference surface coincide at this position. Further, in order to collect the complete 3D structural information of the HepaRG cell 3D model, it is necessary to start collecting from the boundary of the model. The sample arm is automatically stepped down by 2 μιη, Figure 6 Figure 3(d) shows the gray level variance curve of the FF-OCT images collected during the automated interference surface positioning process. The two sub-figures represent the FF-OCT images at the beginning and end positions of the interference surface detection process, respectively. It can be seen that the gray level variance value of the end position FF-OCT image changes abruptly because it contains structural information. Therefore, it is determined that the focal plane and the interference surface coincide at this position. Further, in order to collect the complete 3D structural information of the HepaRG cell 3D model, it is necessary to start collecting from the boundary of the model. The sample arm is automatically stepped down by 2 μιη, Figure 6 Figure 3(e) shows the variation trend of the brenner gradient value of the images during the boundary detection process. The sub-figures represent the image clarity at the beginning and end of the boundary detection process, respectively. It can be seen that the image of the HepaRG cell 3D model gradually blurs as the sample arm moves. The brenner gradient value is no longer significantly decreased, indicating that the boundary of the model has been reached. The red dashed line indicates that the boundary has been detected and stopped.

[0132] This embodiment further evaluates the efficiency of the non-equidistant acquisition strategy compared to the traditional equidistant sampling. The non-equidistant acquisition strategy dynamically adjusts the step distance of the sample arm according to the complexity of the sample structure. In the Non-ROI region where the structure changes less, a step distance of 10 μιη is used to reduce unnecessary data acquisition, while in the ROI region where the structure is complex, a step distance of 2 μιη is used to maintain high resolution. Figure 6 Figure 3(e), Figure 6 Figure 3(f) shows the brenner gradient value curves of images at different depths using traditional equidistant sampling and non-equidistant acquisition for the same HepaRG cell 3D model, respectively. The red dashed lines represent the boundary between the bottom cluster structure and the top sparse structure region and the end position of acquisition, respectively. It can be seen that the image data collected using the non-equidistant acquisition protocol is greatly reduced in the sparse cell structure region. Figure 6 The sub-figure of Figure 3(e) is the FF-OCT image of the sparse structure of the HepaRG cell 3D model adhered to the upper part of the micro-well. The green arrow indicates the sparse and dispersed cells at this depth; Figure 6The subgraph (g) of the middle (g) can clearly show the complex cell structure at the bottom of the FF-OCT image of a certain depth of the cell cluster in the three-dimensional culture model of HepaRG cells. The bottom cluster structure can reflect the biological characteristics of the three-dimensional model because of the high cell density and more information exchange between cells.

[0133] In addition, the embodiment also provides an OCT automatic imaging system, comprising:

[0134] A sample interference surface positioning module is responsible for positioning the sample interference surface, and equalizes the optical path of the sample arm and the reference arm;

[0135] A sample boundary positioning module is responsible for positioning the focal plane layer at the sample boundary;

[0136] An image acquisition module is responsible for non-equidistant three-dimensional OCT image acquisition of the sample.

[0137] To sum up, the OCT automatic imaging device of the present application integrates the illumination, imaging and control modules, and the internal components of each module work cooperatively. The illumination module provides a suitable light source and splits the light through precise optical control; the sample arm and the reference arm module realize optical signal transmission and phase control through the objective lens, displacement table and phase shifter; the detection arm module accurately receives the interference signal imaging; the control module accurately controls the actions of each component, such as the cooperative work of the first displacement table, the second displacement table, the phase shifter and the camera, to realize the automatic operation of image acquisition, phase movement, sample position adjustment and the like, improve the overall stability and reliability of the imaging system, and ensure the efficient and accurate imaging process.

[0138] The above is the preferred embodiment of the present application, and it should be noted that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements are also considered to be within the scope of protection of the present application.

Claims

1. An automated imaging method, characterized by, The method adopts non-equidistant three-dimensional OCT image acquisition; The non-equidistant three-dimensional OCT image acquisition is achieved by reconstructing OCT images from the focal plane layer images under different phase shift amounts; According to the gray variance of the OCT images, the interested region and the non-interested region of the current sample focal plane are screened out, the sample arm is moved to the next focal plane layer with a small step in the interested region, and the sample arm is moved to the next focal plane layer with a large step in the non-interested region; Before the non-equidistant three-dimensional OCT image acquisition, the sample interference surface needs to be positioned, and the focal plane layer is positioned at the sample boundary; The positioning of the sample interference surface comprises: The OCT images are reconstructed from the focal plane layer images under different phase shift amounts; the first sliding window change rate is calculated according to the gray variance of the OCT images; if the first sliding window change rate is greater than a preset threshold value ξ1, it is considered that the current sample focal plane interferes; if the first sliding window change rate is less than or equal to the preset threshold value ξ1, the reference arm is moved in the direction of increasing optical path; The positioning of the focal plane layer at the sample boundary comprises: The second sliding window change rate is calculated according to the image definition of the focal plane layer images; If the second sliding window change rate is less than a preset threshold value ξ3, it is considered that the current focal plane layer is the sample boundary; if the second sliding window change rate is greater than or equal to the preset threshold value ξ3, the sample arm is moved to the next focal plane layer.

2. The method of claim 1, wherein, In the non-equidistant three-dimensional OCT image acquisition, whether it is still in the current focal plane layer is identified according to the comparison between the number of OCT images and the size of the sliding window; if the number of OCT images is less than the size of the sliding window, the sample arm is moved to the next focal plane layer with a large step; if the number of OCT images is greater than the size of the sliding window, the interested region and the non-interested region of the current sample focal plane are screened out according to the gray variance of the OCT images.

3. The method of claim 1, wherein, In the positioning of the sample interference surface, whether it is still in the current focal plane layer is identified according to the comparison between the number of OCT images and the size of the sliding window; if the number of OCT images is less than or equal to the size of the sliding window, the reference arm is moved in the direction of increasing optical path; if the number of OCT images is greater than the size of the sliding window, the first sliding window change rate is calculated according to the gray variance of the OCT images.

4. The method of claim 1, wherein, In the positioning of the focal plane layer at the sample boundary, whether it is still in the current focal plane layer is identified according to the comparison between the number of the current focal plane layer images and the preset threshold value ξ2; if the number of the current focal plane layer images is less than or equal to the preset threshold value ξ2, the sample arm is moved to the next focal plane layer; if the number of the current focal plane layer images is greater than the preset threshold value ξ2, the second sliding window change rate is calculated according to the image definition of the focal plane layer images.

5. The method of claim 1, wherein, The moving direction of the sample arm in the non-equidistant three-dimensional OCT image acquisition process is opposite to the moving direction of the sample arm in the positioning of the focal plane layer at the sample boundary.

6. An automated imaging system implementing the method of any one of claims 1-5, characterized in that It comprises: a sample interference surface positioning module, which is responsible for positioning the sample interference surface and realizing the equal optical path of the sample arm and the reference arm; a sample boundary positioning module, which is responsible for positioning the focal plane layer at the sample boundary; an image acquisition module, which is responsible for non-equidistant three-dimensional OCT image acquisition of the sample.

Images