Sample positioning method of microscopic system, microscopic system, equipment and medium

Abstract

The invention relates to the technical field of microscopic imaging, and discloses a sample positioning method of a microscopic system, the microscopic system, equipment and a medium, and the method comprises the following steps: irradiating a sample to be detected by using a light source of the microscopic system; acquiring a color reference image of the to-be-detected sample containing the region-of-interest mark; continuously collecting multiple frames of color polarization images of the sample to be detected; performing chromatography on all the color polarization images to generate corresponding chromatography images; positioning in all the tomographic images by using the color reference image to obtain a target position of the region of interest; and performing corresponding displacement adjustment based on the target position to maintain dynamic focus tracking of the region of interest. According to the microscopic system, a color polarization combined light source is combined with a non-correction chromatic aberration objective lens structure, multiple groups of images can be obtained only through one-time image acquisition, and dynamic tracking of a living body sample, especially a weak-absorption unmarked sample, is further realized by utilizing a sample positioning mode of combining image chromatography and motion control; and the sample positioning requirement is met.

Description

Technical Field

[0001] The present invention relates to the field of microscopic imaging technology, and in particular to a sample positioning method of a microscopic system, a microscopic system, a device and a medium. Background Art

[0002] Microscopic imaging is widely used in various fields such as industrial detection, biological research, and medical diagnosis. For samples with weak light absorption and no exogenous labeling, i.e., weakly absorbing unlabeled samples, it is difficult to image them clearly under traditional optical microscopes due to the lack of sufficient contrast sources. However, the existing technology has the demand for online / in vivo detection of such samples, so high-contrast, low-phototoxicity label-free imaging of weakly absorbing unlabeled samples has always been one of the most concerned directions in microscopic imaging research.

[0003] At present, the main microscopy technologies for imaging weakly absorbing label-free samples include phase contrast microscopy, differential interference contrast microscopy, differential phase contrast microscopy, etc. Imaging systems built using the above microscopy technologies have problems such as the balance between resolution and contrast and a short depth of field. The above microscopy imaging systems are more suitable for static observation, and it is difficult to achieve motion positioning of samples, and cannot meet the positioning requirements for dynamic observation of samples. Summary of the invention

[0004] In view of this, the present invention provides a sample positioning method, a microscope system, a device and a medium for a microscope system to solve the problem that the existing microscope imaging system is difficult to achieve motion positioning and dynamic focusing of the sample, and thus cannot meet the positioning requirements of sample following, dynamic focusing, tracking shooting, etc.

[0005] In a first aspect, the present invention provides a sample positioning method for a microscopic system, the method comprising:

[0006] Using the light source of the microscope system to illuminate the sample to be tested;

[0007] Acquire a color reference image of the sample to be tested including a mark of a region of interest;

[0008] Continuously collect multiple frames of color polarization images of the sample to be tested;

[0009] Perform tomography on all color polarization images to generate corresponding tomographic images;

[0010] The color reference image is used for positioning in all tomographic images to obtain the target position of the region of interest;

[0011] Make corresponding displacement adjustments based on the target position to maintain dynamic focus on the area of ​​interest.

[0012] The present invention is applied to a microscope system that uses color multi-polarization direction encoding to obtain an imaging light source. The microscope system is specifically designed to use a color polarization combined light source combined with a non-corrected chromatic aberration objective lens structure. Multiple groups of color polarization images with different colors and different polarization directions can be obtained through only one image acquisition. A sample positioning method combining image tomography and motion control is used to dynamically track living samples, especially weakly absorbing unlabeled samples. Sample following, focus tracking and tracking shooting can be achieved, which greatly meets the dynamic positioning requirements of living samples.

[0013] In an optional embodiment, obtaining a color reference image of the sample to be tested containing a region of interest mark includes:

[0014] Synchronously collect multiple groups of polarization images corresponding to four polarization angles of each monochromatic light of the sample to be tested;

[0015] Acquire all polarization images corresponding to the same monochromatic light respectively, and calculate the target polarization images of the corresponding monochromatic light respectively according to all polarization images of each monochromatic light;

[0016] Based on preset marking features, the regions of interest are marked in each target polarization image of monochromatic light, and a corresponding color reference image is obtained, wherein the preset marking features at least include regions corresponding to biological cell structures and microscopic structures.

[0017] The present invention can collect multiple sets of color polarization images containing different colors and polarization directions for weakly absorbing unlabeled samples at one time, and adaptively mark the region of interest according to actual needs, which can improve the flexibility of sample positioning and thus meet the needs of flexible and efficient dynamic tracking of living samples.

[0018] In an optional implementation, calculating the target polarization image of the corresponding monochromatic light respectively according to all polarization images of each monochromatic light includes:

[0019] For each monochromatic light, the corresponding azimuth distribution of each corresponding polarization image is used to perform contrast enhancement processing, and a target polarization image of the monochromatic light is generated accordingly.

[0020] The present invention adopts contrast enhancement processing to strengthen the brightness or color differences between different structures, further expand the brightness and darkness dynamic range of the image, make low-contrast areas such as fine fibers in cells more prominent, and retain the true color of the sample. It significantly improves the analysis accuracy and efficiency of color polarization images in microscopic imaging, thereby ensuring the accuracy of subsequent sample positioning.

[0021] In an optional implementation, all color polarization images are subjected to tomography to generate corresponding tomographic images, including:

[0022] All color polarization images are three-dimensionally reconstructed based on a preset tomography algorithm to generate corresponding tomography images of the sample to be tested, wherein the preset tomography algorithm is implemented by calling a function including Fourier transform, inverse Fourier transform and translation operations.

[0023] The present invention designs a tomography algorithm including Fourier transform, inverse Fourier transform and translation operation functions, which provides high-precision three-dimensional reconstruction capability for microscopic imaging through the efficiency and flexibility of frequency domain processing. It is particularly suitable for scenarios that need to break through the resolution limit or perform dynamic analysis. The obtained tomography images have super-resolution capability, high efficiency, noise robustness and multi-modal compatibility.

[0024] In an optional implementation, the color reference image is used to locate in all tomographic images to obtain the target position of the region of interest, including:

[0025] Selecting a localization image that matches the color reference image from all tomographic images;

[0026] The marked position of the region of interest in the color reference image is obtained, and positioning is performed in the positioning image according to the marked position, and the target position of the region of interest is obtained accordingly.

[0027] The present invention uses a color reference image with a marked position of a region of interest to perform matching and positioning from all tomographic images, which can ensure accurate recognition of the positioning image and precise positioning of the region of interest, further improving the precise positioning of living samples.

[0028] In an optional embodiment, before performing corresponding displacement adjustment based on the target position to maintain dynamic focus on the region of interest, the sample positioning method of the microscope system further includes:

[0029] Calculate the current depth of field of the positioning image;

[0030] Determine whether the current depth of field meets the preset depth of field range;

[0031] If the conditions are met, a step of performing corresponding displacement adjustment based on the target position to maintain dynamic tracking focus on the region of interest is performed;

[0032] If not, the process returns to the step of performing tomography on all color polarization images to generate corresponding tomographic images.

[0033] The present invention can ensure the authenticity and effectiveness of the positioning image by designing the relationship between the current depth of field of the positioning image and the preset depth of field range, thereby ensuring the accuracy of living body positioning.

[0034] In an optional embodiment, before irradiating the sample to be tested with the light source of the microscope system, the sample positioning method of the microscope system further includes:

[0035] Preset preliminary preparations are performed on the microscope system, including a first setting operation of incident angles of red light and blue light corresponding to the annular light source, and a second setting operation of pixel size in an image captured by a polarization camera; wherein the incident angle of the red light source is determined according to the numerical aperture of the objective lens and the ratio of the wavelength of red light to the wavelength of green light; the incident angle of the blue light source is determined according to the numerical aperture of the objective lens and the ratio of the wavelength of blue light to the wavelength of green light; and the pixel size is determined according to the color filter, objective lens magnification, green light wavelength and objective lens numerical aperture configured for the polarization camera.

[0036] The present invention can ensure that the resolution of color images corresponding to different colors is consistent by designing the incident angles of red and blue light sources, thereby avoiding the difference in imaging resolution under illumination of different wavelengths, so as to facilitate the implementation of the focus-tracking image analysis algorithm for the area of ​​interest of the sample; at the same time, by designing the size of the polarization camera, that is, the pixel size, the color information of the sample can be effectively obtained, thereby greatly ensuring the imaging quality of the sample.

[0037] In a second aspect, the present invention provides a microscope system, which includes: a stage, a three-dimensional motion stage, an industrial computer server, a light source, an objective lens, a translation stage, a tube lens and a polarization camera; wherein the three-dimensional motion stage is a three-axis linkage mechanism including an x-axis, a y-axis and a z-axis, the light source uses an annular light source with three primary colors of red, green and blue and four polarizations, the annular light source includes a plurality of annular polarized light sources composed of monochromatic light, and each annular polarized light source is a 360° annular light source surface; each annular polarized light source is nested in sequence from the inside to the outside according to the size of the spectral wavelength, each annular polarized light source includes four polarization angles of 0°, 45°, 90° and 135°, the four light sources with different polarization angles are arranged around the center of a circle at a certain angle interval, and repeated in several groups, wherein the light sources of the 0° and 90° polarization angle channels are arranged relative to each other, and the light sources of the 45° and 135° polarization angle channels are arranged relative to each other.

[0038] The microscope system of the present invention uses a color multi-polarization direction coded imaging light source, i.e., a ring light source. Specifically, a color polarization combined light source is combined with a non-corrected chromatic aberration objective lens structure, so that in actual use, only one image acquisition is required to obtain multiple groups of color polarization images of different colors and different polarization directions, which can effectively improve the three-dimensional imaging capability of living samples, thereby meeting the dynamic positioning requirements of living samples.

[0039] In a third aspect, the present invention provides a sample positioning device, which includes a microscope system as described in the second aspect and a sample to be tested, wherein the microscope system is used to execute a sample positioning method of a microscope system according to the first aspect or any corresponding embodiment thereof.

[0040] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute a sample positioning method for a microscope system according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 is a block diagram of a microscope system according to an embodiment of the present invention;

[0043] Figure 2 is a schematic diagram of a ring light source;

[0044] Figure 3 It is a schematic diagram of the structure of the microscopic system;

[0045] Figure 4 It is a schematic diagram of the structure of another microscopic system;

[0046] Figure 5 is a schematic diagram of the light source structure;

[0047] Figure 6 This is the light source hardware composition diagram;

[0048] Figure 7 It is a schematic diagram of the target surface of the polarization camera;

[0049] Figure 8 is a schematic flow chart of a sample positioning method of a microscope system according to an embodiment of the present invention;

[0050] Fig. 9 is a schematic flow chart of a sample positioning method of another microscope system according to an embodiment of the present invention;

[0051] Fig.10 It is a schematic diagram of the working process of the microscope system;

[0052] Fig.11 Schematic diagram of the structure of a sample positioning device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0054] In this embodiment, a microscopic system is provided. Figure 1 is a block diagram of a microscopic system according to an embodiment of the present invention. Figure 1 As shown, the microscope system includes: a stage, a three-dimensional motion stage, an industrial computer server, a light source, an objective lens, a translation stage, a tube lens and a polarization camera; wherein the three-dimensional motion stage is a three-axis linkage mechanism including an x-axis, a y-axis and a z-axis, the light source uses an annular light source with three primary colors of red, green and blue and four polarizations, the annular light source includes a plurality of annular polarized light sources composed of monochromatic light, and each annular polarized light source is a 360° annular light source surface; each annular polarized light source is nested in sequence from the inside to the outside according to the size of the spectral wavelength, each annular polarized light source includes four polarization angles of 0°, 45°, 90° and 135°, the four light sources with different polarization angles are arranged around the center of a circle at a certain angle interval, and repeated for several groups, wherein the light sources of the 0° and 90° polarization angle channels are arranged relatively, and the light sources of the 45° and 135° polarization angle channels are arranged relatively.

[0055] It should be noted that the specific components of the microscope system of this embodiment, such as the actual functions of the stage, light source, etc., can be adaptively understood by referring to relevant knowledge in the field of microscope imaging, and will not be elaborated on here.

[0056] The microscope system of the embodiment of the present invention uses a color multi-polarization direction coded imaging light source, i.e., a ring light source. Specifically, a color polarization combination light source is combined with a non-corrected chromatic aberration objective lens structure, so that in actual use, only one image acquisition is required to obtain multiple groups of color polarization images of different colors and different polarization directions, which can effectively improve the three-dimensional imaging capability of living samples and thus meet the dynamic positioning requirements of living samples.

[0057] In this embodiment, Figure 2 is a schematic diagram of a ring light source. Figure 2 It can be seen that the annular light source includes three circularly polarized light sources of blue light, green light and red light from the inside to the outside, wherein the number of repeated arrangements of the four polarization angles of 0°, 45°, 90° and 135° included in the circularly polarized light source of each color can be adaptively adjusted according to actual needs and is not limited in detail here.

[0058] In this embodiment, Figure 3 It is a schematic diagram of the structure of the microscope system. Figure 3It can be seen that the system is composed of ① optical microscope system, ② z motion axis, ③ y motion axis, ④ stage, ⑤ x motion axis, and ⑥ industrial computer server. Among them, the structure of ① optical microscope system is as follows Figure 4 As shown, it is composed of ⑦ multi-spectral, multi-polarization direction coded oblique illumination light source, ⑧ microscope objective lens, ⑨ micro-motion displacement stage, ⑩ tube lens, It should be noted that ⑦ the oblique illumination light source with multi-spectral and multi-polarization direction coding is recorded as the annular light source mentioned above in this embodiment.

[0059] In this embodiment, Figure 5 is a schematic diagram of the light source structure. It should be noted that in this embodiment Figure 5 The light source structure is that the four polarization angles of 0°, 45°, 90° and 135° contained in each color circular polarized light source are not arranged repeatedly (i.e., they are arranged only once), so as to facilitate the explanation of the subsequent sample positioning scheme of this embodiment. The specific arrangement of the corresponding oblique illumination light source is referred to below; in addition, the corresponding light source structure and positioning scheme of the four polarization angles of 0°, 45°, 90° and 135° in this embodiment can be adaptively understood in the case of repeated arrangement of multiple times. Specifically, refer to Figure 5 and Figure 6 The multi-spectral, multi-polarization coded oblique illumination source is composed of 48 single-spectrum, single-polarization coded light sources composed of different polarizations and filters; each single-spectrum, single-polarization coded oblique illumination source is composed of Narrow-band point light source, Collimating lens and The narrowband polarizer constitutes the hardware. It should be noted that the oblique illumination light source with single spectrum and single polarization direction encoding in the oblique illumination light source with multi-spectrum and multi-polarization direction encoding of the present embodiment is three rings of blue light (B), green light (G) and red light (R) from the inside to the outside according to wavelength, and 12 are arranged at equal intervals in each ring; four polarization direction polarizers are fixedly installed in the oblique illumination light source of each wavelength, which are 0°, 45°, 90° and 135° respectively, wherein the 0° polarization light source is arranged opposite to the 90° polarization light source, and the 45° polarization light source is arranged opposite to the 135° polarization light source.

[0060] In practical applications, due to the wavelength difference between the central wavelength corresponding to blue light (generally between 440-460nm), the central wavelength corresponding to green light (generally between 520-540nm) and the central wavelength corresponding to red light (generally between 630-650nm), the resolution of the color images corresponding to different colors under different wavelength illumination is different, that is, the resolution of blue light is high and the resolution of red light is low, which makes the resolution inconsistent when the sample is imaged, and it is difficult to implement the focus image analysis algorithm for the sample area of ​​interest, which seriously affects the imaging quality of the sample. Based on this, before the above-mentioned microscope system performs sample positioning analysis, this embodiment can ensure that the resolution of the color images corresponding to different colors is consistent by designing the incident angles of the light sources corresponding to red light and blue light, thereby avoiding the difference in imaging resolution under different wavelength illumination, so as to facilitate the implementation of the focus image analysis algorithm for the sample area of ​​interest; at the same time, by designing the size of the polarization camera, that is, the pixel size, the color information of the sample can be effectively obtained, which greatly guarantees the imaging quality of the sample. Therefore, in this embodiment, the center of the light emitted by the oblique illumination light source encoded with a single spectrum and a single polarization direction points to the same place of the sample and the illumination incident angle α is equal. Specifically, the process of determining the incident angles corresponding to the blue light source and the red light source includes:

[0061] 1. Calculate the reference incident angle α based on the wavelength of the intermediate light source, i.e. green light G , the formula is as follows:

[0062]

[0063] Wherein, NA is the numerical aperture of the objective lens, and n is the refractive index of the medium between the light source and the sample. Generally, if it is in air, the refractive index n is approximately equal to 1. It should be noted that the specific value of the numerical aperture of the objective lens can be determined according to conventional techniques in the art and is not limited in detail here.

[0064] 2. Set the incident angle α of the blue light source B and the incident angle α of the red light source R , and the corresponding formulas are as follows:

[0065]

[0066] Among them, λ B , G and λ R They are the corresponding wavelengths of red light, green light and blue light respectively; by setting the incident angles of blue light and red light as mentioned above, the resolution change of the microscopic image caused by the wavelength difference of different colors of light can be compensated.

[0067] 3. Setting the pixel size in the image captured by the polarization camera (it should be noted that the setting of the pixel size in this embodiment essentially represents the selection of different types of cameras). Figure 7 This is a schematic diagram of the polarization camera target surface. The designed polarization camera target surface needs to meet the Bayer filter rule and the pixel size L needs to meet the following conditions:

[0068]

[0069] Where M is the objective lens magnification, if λ G =530nm, objective lens numerical aperture NA=0.75, magnification M=40 to calculate the pixel size L, that is, L<4.31um is required to meet the corresponding color image acquisition requirements.

[0070] It should be explained that the Bayer filter rule is a color filter array widely used in digital cameras. Specifically, it achieves a balance between hardware cost and image quality through the "green as the main and red and blue as the auxiliary" pixel arrangement (that is, red, green and blue filters are placed alternately in the pixel array to efficiently record color information). It is the basic technology of modern digital imaging. In this embodiment, the polarization camera can better process light and improve the color reproduction and clarity of the image by applying the Bayer filter rule, so as to perform well in various shooting scenes.

[0071] It should be noted that this embodiment uses a Bayer color filter, also known as a Bayer filter, for human viewing. In this embodiment, other filters can also be used for corresponding arrangement, and the content of the above formula (3) is also adaptively adjusted according to the actual filter type. In summary, through the above-mentioned preliminary preparation of the microscope system, the light source in the microscope imaging system is specifically formed into a ring light source in order from large to small wavelengths, that is, red, green, and blue correspond to light sources. According to the resolution formula, it can be seen that due to the short wavelength of blue light, the incident angle of the light source corresponds to a high resolution of blue light, while the wavelength of red light is longer and the resolution is low; the blue light source is placed in the inner ring, and the incident angle is small when it is used for illumination; because there is another item in the resolution formula that is related to the angle (small angle corresponds to low resolution), the blue light is set in the inner ring and the red light is set in the outer ring, so that a color image with a similar resolution can be obtained, thereby balancing the balanced resolution of imaging of different color lights, which helps to ensure the imaging quality.

[0072] An embodiment of the present invention provides an embodiment of a sample positioning method for a microscope system. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0073] In this embodiment, a sample positioning method of a microscope system is provided. Figure 8 FIG. 1 is a flow chart of a sample positioning method of a microscope system according to an embodiment of the present invention. Figure 8 As shown, the process includes the following steps:

[0074] Step S801, using the light source of the microscope system to illuminate the sample to be tested.

[0075] It should be noted that the specific content of the sample to be tested in this embodiment can be adaptively adjusted according to actual needs and is not limited in detail here.

[0076] Step S802: obtaining a color reference image of the sample to be tested containing a mark of a region of interest.

[0077] In this embodiment, the specific content of the region of interest can be adaptively adjusted according to actual needs, for example, the region of interest is a region with a biological cell membrane structure or a specific microscopic morphology. It should be noted that the region of interest in this embodiment needs to be marked accordingly with artificial assistance.

[0078] It should be noted that the specific content and acquisition method of the color reference image in this embodiment are not limited herein. For example, a color polarization camera is used to collect color images of the sample to be tested in multiple polarization directions and different spectra at one time, and good quality images are screened out and a region of interest with microstructural features is selected in the corresponding image to obtain a corresponding color reference image. This is only an exemplary description.

[0079] Step S803, continuously acquiring multiple frames of color polarization images of the sample to be tested.

[0080] In this embodiment, the specific method of acquiring multiple frames of color polarization images can be obtained according to actual microscopic imaging technology, which will not be described in detail here.

[0081] Step S804: performing tomography on all color polarization images to generate corresponding tomographic images.

[0082] It needs to be explained that image tomography is a technology that reconstructs the structure of three-dimensional objects through two-dimensional projection data. Its core idea is to utilize signal acquisition at different angles or in different modes, combined with mathematical algorithms to invert the spatial distribution inside the object.

[0083] In this embodiment, the polarization characteristics of the color polarization image are taken into account, that is, the tomography of the color polarization image combines polarization optics and tomography technology, and specifically extracts the optical characteristics (such as birefringence, dichroism) and three-dimensional structure of the object by analyzing the polarization state difference of light, and then obtains the corresponding tomography image. It should be noted that the specific tomography method of the color polarization image in this embodiment is not limited in detail here, such as using the polarization state change as the contrast source, combining the mathematical algorithm to reconstruct the three-dimensional spatial distribution, and generating the corresponding tomography image of the color polarization image, which is only an exemplary description.

[0084] Step S805 , positioning is performed using the color reference image in all tomographic images to obtain the target position of the region of interest.

[0085] In this embodiment, the process of determining the target position of the region of interest is essentially a process of performing positioning matching in any tomographic image using the color reference image, and the specific matching process is adaptively set according to actual needs.

[0086] Step S806: Perform corresponding displacement adjustment based on the target position to maintain dynamic focus on the region of interest.

[0087] In this embodiment, the displacement adjustment is achieved by controlling the corresponding movement of the three-dimensional motion stage in the microscope system, thereby performing real-time focusing on the region of interest.

[0088] The sample positioning method of the microscope system of the embodiment of the present invention is applied to the microscope system that uses color multi-polarization direction encoding to obtain the imaging light source, that is, the microscope system adopts a color polarization combination light source combined with a non-corrected chromatic aberration objective structure, and can obtain multiple groups of color polarization images of different colors and different polarization directions through only one image acquisition. The sample positioning method combining image tomography and motion control is used to dynamically track living samples, especially weakly absorbing unlabeled samples, and sample following, focus tracking and tracking shooting can be realized, which greatly meets the dynamic positioning needs of living samples.

[0089] In this embodiment, a sample positioning method of a microscope system is provided. Fig. 9 FIG. 1 is a flow chart of a sample positioning method of another microscopic system according to an embodiment of the present invention. Fig. 9 As shown, the process includes the following steps:

[0090] Step S901, using the light source of the microscope system to illuminate the sample to be tested.

[0091] It should be noted that before executing step S901 to illuminate the sample to be tested using the light source of the microscope system, a series of preliminary preparations need to be performed on the microscope system to ensure the quality of the microscope imaging. It should be noted that the relevant contents of the preliminary preparations are referred to above and will not be repeated here. Specifically, before irradiating the sample to be tested using the light source of the microscope system, the sample positioning method of the microscope system of this embodiment also includes:

[0092] Preset preliminary preparations are performed on the microscope system, including a first setting operation of incident angles of red light and blue light corresponding to the annular light source, and a second setting operation of pixel size in an image captured by a polarization camera; wherein the incident angle of the red light source is determined according to the numerical aperture of the objective lens and the ratio of the wavelength of red light to the wavelength of green light; the incident angle of the blue light source is determined according to the numerical aperture of the objective lens and the ratio of the wavelength of blue light to the wavelength of green light; and the pixel size is determined according to the color filter, objective lens magnification, green light wavelength and objective lens numerical aperture configured for the polarization camera.

[0093] In the embodiment of the present invention, by designing the incident angles of the red light and the blue light corresponding to the light sources, it is possible to ensure that the resolution of the color images corresponding to different colors is consistent, thereby avoiding the difference in imaging resolution under illumination of different wavelengths, so as to facilitate the implementation of the focus tracking image analysis algorithm for the area of ​​interest of the sample; at the same time, by designing the size of the polarization camera, that is, the pixel size, the color information of the sample can be effectively obtained, which greatly guarantees the imaging quality of the sample.

[0094] Step S902: obtaining a color reference image of the sample to be tested containing a mark of a region of interest.

[0095] Specifically, the above step S902 includes:

[0096] Step S9021, synchronously collect multiple groups of polarization images corresponding to four polarization angles of each monochromatic light of the sample to be tested.

[0097] Step S9022 , respectively acquiring all polarization images corresponding to the same monochromatic light, and respectively calculating the target polarization images of the corresponding monochromatic light according to all polarization images of each monochromatic light.

[0098] Specifically, in the above step S9022, the target polarization images of the corresponding monochromatic lights are calculated respectively according to all polarization images of the monochromatic lights, including:

[0099] For each monochromatic light, the corresponding azimuth distribution of each corresponding polarization image is used to perform contrast enhancement processing, and a target polarization image of the monochromatic light is generated accordingly.

[0100] In this embodiment, contrast enhancement processing is also called contrast enhancement, which is specifically achieved by transforming the pixel brightness value (also called grayscale or grayscale value). In this embodiment, the enhancement processing of image light intensity in different azimuth structures in the plane is mainly considered, that is, differential contrast enhancement processing is performed on each polarization image to generate an enhanced color polarization image, which is only used as an exemplary description.

[0101] In the embodiment of the present invention, by adopting contrast enhancement processing, the brightness or color differences between different structures can be strengthened, and the brightness and darkness dynamic range of the image can be further expanded, making low-contrast areas such as fine fibers in cells more prominent, while retaining the true color of the sample, significantly improving the analysis accuracy and efficiency of the color polarization image in microscopic imaging, thereby ensuring the accuracy of subsequent sample positioning.

[0102] Step S9023, marking the regions of interest in each target polarization image of monochromatic light based on preset marking features, and obtaining a corresponding color reference image, wherein the preset marking features at least include regions corresponding to biological cell structures and microscopic structures.

[0103] The embodiments of the present invention can collect multiple sets of color polarization images containing different colors and polarization directions for weakly absorbing unlabeled samples at one time, and adaptively mark the region of interest according to actual needs, thereby improving the flexibility of sample positioning and thus meeting the requirements of flexible and efficient dynamic tracking of living samples.

[0104] Step S903, continuously collect multiple frames of color polarization images of the sample to be tested. Figure 8 Step S803 of the illustrated embodiment will not be described in detail here.

[0105] Step S904: performing tomography on all color polarization images to generate corresponding tomographic images.

[0106] Specifically, the above step S904 includes:

[0107] Step S9041, performing three-dimensional reconstruction on all color polarization images based on a preset tomography algorithm to generate a corresponding tomography image of the sample to be tested, wherein the preset tomography algorithm is implemented by calling a function including Fourier transform, inverse Fourier transform and translation operations.

[0108] In this embodiment, the specific content of the preset tomography algorithm can be adaptively adjusted according to actual needs and is not limited in detail here. Specifically, by designing a tomography algorithm including Fourier transform, inverse Fourier transform and translation operation functions, high-precision three-dimensional reconstruction capabilities are provided for microscopic imaging through the efficiency and flexibility of frequency domain processing, which is particularly suitable for scenes that need to break through the resolution limit or perform dynamic analysis. The obtained tomography images have super-resolution capabilities, high efficiency, noise robustness and multi-modal compatibility.

[0109] Step S905 , positioning is performed using the color reference image in all tomographic images to obtain the target position of the region of interest.

[0110] Specifically, the above step S905 includes:

[0111] Step S9051: Filter the positioning image that matches the color reference image from all tomographic images.

[0112] In this embodiment, the specific method of image matching is adaptively determined according to actual needs, such as template matching, which is only used as an exemplary description.

[0113] Step S9052, obtaining the marked position of the region of interest in the color reference image, and positioning in the positioning image according to the marked position, and correspondingly obtaining the target position of the region of interest.

[0114] In the embodiment of the present invention, the color reference image with the marked position of the region of interest is used to perform matching and positioning from all tomographic images, which can ensure the accurate recognition of the positioning image and the precise positioning of the region of interest, and further improve the precise positioning of the living sample.

[0115] Step S906: Perform corresponding displacement adjustment based on the target position to maintain dynamic focus on the region of interest.

[0116] It should be noted that before executing step S906 to make corresponding displacement adjustments based on the target position to maintain dynamic tracking focus on the region of interest, it is also necessary to determine whether the positioning image of the target position is reasonable. Therefore, in this embodiment, a process for determining the relationship between the current depth of field of the positioning image and the preset depth of field range is designed to ensure the authenticity and effectiveness of the positioning image. Specifically, before executing the above step S906, the sample positioning method of the microscope system of this embodiment also includes:

[0117] Step A1, calculating the current depth of field of the positioning image.

[0118] It should be noted that the depth of field (DOF) refers to the range of distances in front and behind the object measured by the imaging that can obtain a clear image at the front of the camera lens or other imager. The specific method of obtaining it is determined according to the conventional depth of field formula in this field.

[0119] Step A2, determining whether the current depth of field meets a preset depth of field range.

[0120] In this embodiment, the specific value of the preset depth of field range is not limited here and is adaptively adjusted based on actual needs.

[0121] Step A3: If the conditions are met, a step of performing corresponding displacement adjustment based on the target position to maintain dynamic focus on the region of interest is performed.

[0122] Step A4: if the condition is not satisfied, then the process returns to the step of performing tomography on all color polarization images to generate corresponding tomographic images.

[0123] In the embodiment of the present invention, by designing the relationship between the current depth of field of the positioning image and the preset depth of field range, the authenticity and effectiveness of the positioning image can be ensured, thereby ensuring the accuracy of the living body positioning.

[0124] In a specific embodiment, reference Figure 3 and Figure 4 The microscopic structure of the system, and Figure 5 and Figure 6 In the light source structure, a positioning scheme is proposed to realize dynamic adjustment of the focal length of the sample, that is, the microscopic positioning and tracking of three-dimensional weakly absorbing label-free samples. It should be noted that the light source in the positioning scheme, that is, the four polarization angles of 0°, 45°, 90° and 135° contained in each color circular polarized light source, are arranged only once; for the case of repeated arrangement multiple times, the corresponding light source structure and positioning scheme are adaptively adjusted, that is, the specific contents of the relevant calculation formulas (4) and (5) below are also adaptively adjusted. Specifically, the positioning scheme calculates and extracts the position of the region of interest of the weakly absorbing label-free sample by using oblique illumination encoded with color multiple polarization directions, and then focuses to realize dynamic tracking of the sample. Fig.10 It is a schematic diagram of the working process of the microscope system. Fig.10 It can be seen that the process includes:

[0125] 1. Fix the sample on the stage and focus.

[0126] In this embodiment, the light source of the microscope system is turned on, and the camera is focused to find the location of the sample.

[0127] 2. Take pictures to obtain color polarization images.

[0128] In this embodiment, the information of the color polarization image is as follows: 1 ,I 2 ,I 3 ,I 4 Corresponding to red light polarization 0°, 45°, 90°, 135°; I 1 ′、I 2 ′、I 3 ′、I 4 ' corresponds to green light polarization 0°, 45°, 90°, 135°; I 1 ″、I 2 ″、I 3 ″、I 4 ″ corresponds to blue light polarization 0°, 45°, 90°, and 135°.

[0129] It should be noted that polarizers have different transmission wavelengths, and each polarization direction can be rotated to obtain a corresponding polarization angle, thereby obtaining a corresponding polarization image.

[0130] 3. Obtain the differential contrast enhanced color polarization image of the transparent sample by calculation.

[0131] In this embodiment, the red light I R 、Green Light I G and Blu-ray I B The specific calculation method of the corresponding enhanced color polarization image is as follows:

[0132]

[0133] It should be noted that the form of the above formula 4 is not unique and can be adaptively determined according to actual needs, that is, considering the structure of light intensity in different directions in the plane, the numerator and denominator of the color polarization image corresponding to each color are adaptively adjusted.

[0134] 4. Select the area of ​​interest in the sample.

[0135] In this embodiment, an area of ​​interest and having certain characteristics in the sample is observed on the above-mentioned differential contrast enhanced color polarization image, and the area is framed and selected, and the image information is recorded in a related server.

[0136] It should be noted that the selection of the region of interest (i.e., the region with biological cell structures or other microscopic structures, rather than the culture medium or other background) requires manual confirmation. For example, the region of interest marking the cell membrane is only used as an example.

[0137] 5. Continuously collect images.

[0138] In this embodiment, images of the living sample are continuously captured in real time.

[0139] 6. Three-dimensional image calculation.

[0140] In this embodiment, tomography is performed on the continuously captured images, and the tomography formula is as follows:

[0141]

[0142] Among them, T{} is the translation operation function, specifically:

[0143]

[0144] It means that for the zth image I z ,h=0 is the translation operation of the central focal plane (i.e., the central layer image); F{} and F -1 {} are Fourier transform and inverse Fourier transform respectively; (X, Y) are image space coordinates; (U, V) are image coordinates in Fourier domain; j is an imaginary unit, and K is the number of image pixels. Note that D in formula (5) represents the distance between the sample and the center image, which is the same as the value of h.

[0145] In this embodiment, when the pixel size L=4 um, the corresponding depth of field is calculated according to the depth of field formula to limit the maximum range of the layer. The specific formula is:

[0146]

[0147] It should be noted that the meanings of the corresponding parameters on the right side of equation 7 refer to the previous text and will not be repeated here. Specifically, according to the formula, the depth of field under blue light (450nm), green light (530nm), and red light (630nm) are 0.93um, 1.08um, and 1.25um, respectively. In practical applications, if an apochromatic objective is used, the basic depth of field is determined by blue light (i.e., the shortest wavelength), which is 0.93um; according to equations (5) and (6), any number of computed tomography images within a depth range of at least 3um can be obtained through the tomography algorithm; if a non-achromatic objective is used, the three wavelength imaging phases can be separated by 1um, and the depth of field directly reaches 3um. Using the tomography algorithm of equations (5) and (6), any number of computed tomography images within a depth range of at least 5um can be obtained. In summary, no matter which objective is used, a three-dimensional image of the test sample within a certain range can be generated from the computed tomography image.

[0148] 7. Sample positioning.

[0149] In this embodiment, template matching and gradient, maximum and other methods are used to obtain and locate the spatial position of the region of interest in the sample in the three-dimensional image.

[0150] 8. Displacement of the motion platform.

[0151] In this embodiment, the three-dimensional motion table is moved to perform dynamic displacement according to the positioning guidance.

[0152] In summary, the embodiments of the present invention have the following advantages.

[0153] 1. Due to the use of a color polarization combined light source combined with a non-corrected chromatic aberration objective lens structure, this embodiment can obtain sufficient image data for computed tomography through a single image acquisition.

[0154] 2. Due to the use of the computational tomography method, this embodiment can quickly extract the three-dimensional information of the sample from the oblique illumination microscopic image.

[0155] 3. Due to the use of a system solution combining image analysis and motion control, this embodiment can further achieve sample following, focus tracking and tracking shooting.

[0156] The present invention also provides a sample positioning device, see Fig.11 , Fig.11 is a schematic diagram of the structure of the sample positioning device provided in an optional embodiment of the present invention, such as Fig.11 As shown, the sample positioning device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the sample positioning device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output system (such as a display sample positioning device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple sample positioning devices can be connected, and each sample positioning device provides part of the necessary operations (for example, determined as a server array, a group of blade servers, or a multi-processor system). Fig.11 A processor 10 is taken as an example.

[0157] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0158] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0159] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created according to the use of the sample positioning device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the sample positioning device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0160] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0161] The sample positioning device further comprises a communication interface 30, which is used for the main control chip to communicate with other sample positioning devices or a communication network.

[0162] A computer-readable storage medium is also provided in an embodiment of the present invention. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor main control chip or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0163] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A sample positioning method for a microscope system, characterized in that: The method comprises: Using the light source of the microscope system to illuminate the sample to be tested; Acquire a color reference image of the sample to be tested including a region of interest mark; Continuously collecting multiple frames of color polarization images of the sample to be tested; Perform tomography on all color polarization images to generate corresponding tomographic images; Using the color reference image to locate in all tomographic images, to obtain the target position of the region of interest; A corresponding displacement adjustment is performed based on the target position to maintain dynamic focus on the region of interest.

2. The sample positioning method of a microscope system according to claim 1, characterized in that: The step of obtaining a color reference image of the sample to be tested containing a region of interest mark comprises: Synchronously collect multiple groups of polarization images corresponding to four polarization angles of each monochromatic light of the sample to be tested; Acquire all polarization images corresponding to the same monochromatic light respectively, and calculate the target polarization images of the corresponding monochromatic light respectively according to all polarization images of each monochromatic light; Based on preset marking features, the regions of interest are marked in each target polarization image of monochromatic light, and a corresponding color reference image is obtained, wherein the preset marking features at least include regions corresponding to biological cell structures and microscopic structures.

3. The sample positioning method of a microscope system according to claim 2, characterized in that: The step of calculating the target polarization images of the corresponding monochromatic lights respectively according to all polarization images of the monochromatic lights comprises: For each monochromatic light, the contrast enhancement process is performed using the corresponding azimuth distribution of each corresponding polarization image, and the target polarization image of the monochromatic light is generated accordingly.

4. The sample positioning method of a microscope system according to claim 1, characterized in that: The step of performing tomography on all color polarization images to generate corresponding tomographic images includes: All color polarization images are three-dimensionally reconstructed based on a preset tomography algorithm to generate corresponding tomography images of the sample to be tested, wherein the preset tomography algorithm is implemented by calling a function including Fourier transform, inverse Fourier transform and translation operations.

5. The sample positioning method of a microscope system according to claim 1, characterized in that: The method of using the color reference image to locate in all tomographic images to obtain the target position of the region of interest includes: Selecting a positioning image that matches the color reference image from all tomographic images; The marked position of the region of interest in the color reference image is obtained, and positioning is performed in the positioning image according to the marked position, and accordingly the target position of the region of interest is obtained.

6. The sample positioning method of a microscope system according to claim 5, characterized in that: Before performing corresponding displacement adjustment based on the target position to maintain dynamic tracking focus on the region of interest, the method further includes: Calculate the current depth of field of the positioning image; Determining whether the current depth of field meets a preset depth of field range; If the conditions are met, the step of performing corresponding displacement adjustment based on the target position to maintain dynamic focus on the region of interest is performed; If not, the process returns to the step of performing tomography on all color polarization images to generate corresponding tomographic images.

7. The sample positioning method of a microscopic system according to any one of claims 1 to 6, characterized in that: Before irradiating the sample to be tested with the light source of the microscope system, the method further comprises: Preset preliminary preparations are performed on the microscope system, and the preset preliminary preparations include a first setting operation of incident angles of red light and blue light corresponding to the light sources included in the annular light source, and a second setting operation of the pixel size in the image collected by the polarization camera; wherein the incident angle of the red light source is determined according to the numerical aperture of the objective lens and the ratio of the wavelength of red light to the wavelength of green light; the incident angle of the blue light source is determined according to the numerical aperture of the objective lens and the ratio of the wavelength of blue light to the wavelength of green light; and the pixel size is determined according to the color filter, objective lens magnification, green light wavelength and objective lens numerical aperture configured by the polarization camera.

8. A microscopic system, characterized in that: The microscope system includes: a stage, a three-dimensional motion stage, an industrial computer server, a light source, an objective lens, a displacement stage, a tube lens and a polarization camera; wherein the three-dimensional motion stage is a three-axis linkage mechanism including an x-axis, a y-axis and a z-axis; the light source uses an annular light source with three primary colors of red, green and blue and four polarizations; the annular light source includes a plurality of annular polarized light sources composed of monochromatic light, and each annular polarized light source is a 360° annular light source surface; each annular polarized light source is nested in sequence from the inside to the outside according to the size of the spectral wavelength; each annular polarized light source includes four polarization angles of 0°, 45°, 90° and 135°; the four light sources with different polarization angles are arranged around the center of a circle at a certain angle interval, and repeated for several groups; wherein the light sources of the 0° and 90° polarization angle channels are arranged relative to each other, and the light sources of the 45° and 135° polarization angle channels are arranged relative to each other.

9. A sample positioning device, characterized in that: The sample positioning device comprises the microscopic system according to claim 8 and a sample to be tested, wherein the microscopic system is used to perform the sample positioning method of the microscopic system according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the sample positioning method for a microscope system according to any one of claims 1 to 7.

Images