Multi-spectrum and information fused ultra-depth-of-field microscopic image acquisition system

Abstract

The invention provides a multi-spectrum and information fused super-field-depth microscopic image acquisition system, which is characterized in that on the basis of a traditional microscope system, a multi-spectrum illumination system (400-1000nm) capable of being freely adjusted is adopted for illumination by utilizing the principle that the positions of focusing layers are different when light in different wavebands irradiates an object and passes through a lens, so that the image acquisition accuracy is improved. The invention develops a multispectral light splitting imaging device composed of a plurality of semi-reflecting and semi-transmitting mirrors / galvanometers, filters and imaging elements, realizes light splitting of a multispectral illumination system and can obtain sample images of all wavebands only through one-time imaging, and the collected sample images are calculated by using a calculation module to obtain super-depth-of-field microscopic images. According to the system and the method, the depth of field can be effectively enlarged to about five times of the original microscope, the operation is simple and convenient, the tedious image acquisition step is avoided, and the dynamic measurement of an object with slight vibration can be realized.

Description

Technical Field

[0001] The invention belongs to the technical field of optical microscopes, and in particular relates to a multi-spectral and information-fused ultra-depth-of-field microscopic image acquisition system. Background Art

[0002] At present, the main ways to increase the depth of field are: reducing the aperture method, optical apodization method, wavefront coding method and zoom lens technology. The reducing aperture method refers to reducing the aperture of the aperture stop; the optical apodization method refers to modulating the transmittance and phase of the aperture stop to increase the depth of field of the system; in the Chinese patent with the publication number CN 118329852A, a large-image fluorescence microscopy imaging system based on wavefront coding is disclosed. This patent aims at the problem of insufficient field of view of the fluorescence microscopy imaging system when detecting dPCR fluorescent gene chips, and develops a depth of field extension device. The development process is to optimize the phase plate parameters through an algorithm, and make the phase plate according to the optimized parameters. The imaging system for dPCR fluorescent gene chip detection includes a receiving device for receiving images located on the same optical axis, a microscopic imaging system for imaging the sample chip, a filtering device and a sample for separating the excitation light and the fluorescence, an illumination device for exciting the fluorescence below the filtering device in the direction perpendicular to the optical axis, and an extension device for extending the depth of field placed in the microscopic imaging system. The extension device can meet the defocus invariance within the range of ±200μm and generate a consistent intermediate blurred image, which can generate a clear image after subsequent algorithm processing. In the Chinese patent with publication number CN 118433552 A, a method and system for collecting object images are disclosed. The patent is based on an existing liquid lens. According to the minimum step size, the liquid lens is zoomed successively, and images are collected in sequence. The collected images are partitioned one by one, the region of interest is extracted, and the gradient is calculated, and finally a large depth of field image is obtained.

[0003] The depth of field is proportional to the wavelength of the incident light, but the resolution of the microscope is inversely proportional to the wavelength of the incident light. Traditional microscopes generally make a trade-off between depth of field and spatial resolution. The pursuit of high resolution will cause a certain loss in depth of field, and the reduction in depth of field will cause only part of the image to be clear, while other areas of the image are blurred. Among the methods of increasing the depth of field, the aperture reduction method sacrifices resolution and cannot take into account both high resolution and large depth of field; the optical toe cutting method generally loses a certain amount of light flux and may cause uneven image brightness distribution in the imaging system; the development of additional optical elements such as the phase plate in the wavefront coding method requires complex algorithms and parameter optimization processes, and the developed phase plate is limited in application scenarios, and the collected images need to be processed again; the method of expanding the depth of field based on continuous zoom technology has many steps, and the process of taking images is time-consuming.

[0004] Therefore, inventing a universal and simple microscope system and method for obtaining ultra-depth-of-field images without sacrificing resolution is a problem that urgently needs to be solved in the field of microscopic optical technology. Summary of the invention

[0005] To solve the above problems, the present invention provides a multi-spectral and information fused ultra-depth-of-field microscopic image acquisition system, which can realize the splitting of the multi-spectral illumination system and obtain sample images of all bands through only one imaging. It is easy to operate and avoids the cumbersome steps of collecting images.

[0006] A multi-spectral and information fused ultra-depth microscopic image acquisition system, comprising an image receiving and microscopic imaging component 2 and a computing component 3;

[0007] When the system is used to photograph the surface morphology of a sample, the image receiving and microscopic imaging component 2 is used to photograph a multispectral image of a sample 1 having a height difference on the surface, and send the multispectral image to a computing device 3;

[0008] The computing device 3 is used to perform image processing on the multispectral image to obtain a super-depth-of-field microscopic image of the sample 1 .

[0009] Furthermore, the image receiving and microscopic imaging assembly 2 includes a sample stage 6 for placing the sample 1, a microscope objective lens 7, a prism group 8, a multi-spectral illumination system 9 located at the same horizontal plane as the prism group 8, a tube lens 10, and a multi-spectral spectroscopic imaging device 11; wherein the microscope objective lens 7 is arranged above the sample stage 6; the prism group 8 is arranged above the microscope objective lens 7; the tube lens 10 is arranged above the prism group 8; and the multi-spectral spectroscopic imaging device 11 is located above the tube lens 10;

[0010] The multi-spectral illumination system 9 emits a multi-spectral light source with a wavelength of 400 to 1000 nm, which is reflected to the microscope objective lens 7 via the prism group 8 and incident to the surface of the sample 1 through the microscope objective lens 7; the reflected light of the sample 1 passes through the microscope objective lens 7, the prism group 8, and the tube lens 10 in sequence and then is incident to the multi-spectral spectroscopic imaging device 11; the multi-spectral spectroscopic imaging device 11 is used to generate an image corresponding to each monochromatic light in the multi-spectral light source according to the reflected light carrying the surface information of the sample 1.

[0011] Furthermore, the multi-spectral spectroscopic imaging device 11 comprises N imaging elements, N filters with different filtering bands, and a spectroscopic module; wherein N is the number of bands of interest selected from the multi-spectral light source, and a filter is arranged in front of each imaging element;

[0012] The spectroscopic module is used to split the reflected light into N sub-beams, and the N sub-beams are respectively incident on N filters, each filter generates monochromatic light of different bands according to its own filtering band; the N monochromatic lights of different bands are respectively incident on an imaging element, and the imaging element generates an image corresponding to each monochromatic light.

[0013] Furthermore, when N is an odd number, the spectroscopic module is composed of N-1 half-reflecting half-mirrors; wherein, every two half-reflecting half-mirrors are arranged in a group left and right, and each group of half-reflecting half-mirrors is arranged up and down according to the position of an alternating half-reflecting half-mirror, forming a stepped arrangement; at the same time, each half-reflecting half-mirror is numbered in sequence from left to right and from top to bottom, and the half-reflecting half-mirror located in the middle of the stepped arrangement is coaxial with the tube lens 10; N groups of imaging elements and filters are respectively arranged on the left side of the left side half-reflecting half-mirror, the upper side of the upper side half-reflecting half-mirror, and the right side of the right side half-reflecting half-mirror in the stepped arrangement.

[0014] Further, when N=5, and the spectroscopic module is composed of four half-reflecting half-mirrors; wherein, every two half-reflecting half-mirrors are arranged in a group left and right, and the two groups of half-reflecting half-mirrors are arranged up and down according to the position of an alternating half-reflecting half-mirror; at the same time, the four half-reflecting half-mirrors are numbered in sequence from left to right and from top to bottom as half-reflecting half-mirror I, half-reflecting half-mirror II, half-reflecting half-mirror III, and half-reflecting half-mirror IV, then the half-reflecting half-mirror II and the half-reflecting half-mirror III are coaxial with the tube lens 10; five groups of imaging elements and filters are respectively arranged on the left and upper side of the half-reflecting half-mirror I, the upper side of the half-reflecting half-mirror II, and the upper and right sides of the half-reflecting half-mirror IV.

[0015] Furthermore, the light splitting module is a rotating micro-galvanometer; wherein each rotation angle of the micro-galvanometer corresponds to a group of imaging elements and filters, and N groups of imaging elements and filters are arranged around the micro-galvanometer with the micro-galvanometer as the center. Then, the reflected light emitted by the tube lens 10 is divided into N paths by the micro-galvanometer rotated to N different angles, and then respectively enters the N groups of filters and imaging elements.

[0016] Furthermore, when N=5, and the spectroscopic module is a rotating micro-galvanometer; wherein each rotation angle of the micro-galvanometer corresponds to a group of imaging elements and filters, and five groups of imaging elements and filters are arranged around the micro-galvanometer with the micro-galvanometer as the center. Then, the reflected light emitted by the tube lens 10 is divided into five paths by the micro-galvanometer rotated to five different angles, and then respectively incident on the five groups of filters and imaging elements.

[0017] Furthermore, the computing device 3 includes an image segmentation module, a decision generation module, a data fusion module and an image enhancement module;

[0018] The image segmentation module segments the clear area and the blurred area in each monochromatic light image in the multispectral image based on the characteristic gradient method, and obtains the activity level of each pixel point of the segmented image;

[0019] The decision generation module is used to obtain a decision graph corresponding to each monochromatic light image according to the activity level on each monochromatic light image:

[0020]

[0021] Among them, D i(x,y) is the decision diagram corresponding to the monochromatic image corresponding to the i-th band, n represents the number of bands of the monochromatic image, SF i(x,y) Indicates the activity level of the monochromatic image corresponding to the i-th band, SF j(x,y) represents the activity level corresponding to the jth band; if SF i(x,y) Not less than SF j(x,y) , then D i(x,y) =1, if SF i(x,y) Smaller than any SF j(x,y) , then D i(x,y) =0;

[0022] The data fusion module is used to perform pixel-level weighted fusion on each decision graph and the monochromatic light image corresponding to each decision graph to obtain a fused image;

[0023] The image enhancement module is used to visually enhance the fused image by using a brightness enhancement method, a contrast enhancement method, and a sharpness enhancement method to obtain a final super-depth-of-field microscopic image.

[0024] Furthermore, the activity level SF calculated pixel by pixel for any segmented image is (x,y) for:

[0025]

[0026] Among them, CF (x,y) is the gradient of the pixel at position (x, y) in the column direction, RF F(x,y) is the gradient of the pixel at position (x, y) in the row direction, r is the radius of the range of interest selected when calculating the activity level of the pixel at position (x, y), and F (x+a,y+b) is the region (x+a, y+b) centered at the pixel at position (x, y) on the segmented image output by the image segmentation neural network based on the feature gradient method, F (x+a-1,y+b) It is the area (x+a-1,y+b) centered on the pixel at position (x,y) on the segmented image output by the image segmentation neural network based on the feature gradient method.

[0027] Furthermore, the fused image M (x,y) The method to obtain is:

[0028]

[0029] Among them, D i(x,y) Img is the decision graph corresponding to the monochromatic image corresponding to the i-th band, i (x, y) is the monochromatic light image corresponding to the i-th band.

[0030] Beneficial effects:

[0031] 1. The present invention provides a multi-spectral and information-fused ultra-depth-of-field microscopic image acquisition system. On the basis of a traditional microscope system, the present invention utilizes the principle that when light of different wavelength bands illuminates an object and passes through a lens, the focusing plane positions are different. A freely adjustable multi-spectral illumination system (400-1000nm) is used for illumination. A multi-spectral spectroscopic imaging device composed of multiple semi-reflective and semi-mirror mirrors / galvanometer mirrors, filters and imaging elements is developed. The spectroscopic imaging device of the multi-spectral illumination system is realized, and sample images of all wavelength bands can be obtained by only one imaging. The operation is simple and the cumbersome steps of collecting images are avoided.

[0032] 2. The present invention provides a multi-spectral and information-fused ultra-depth-of-field microscopic image acquisition system, which processes the images of each band obtained using various modules of a deep learning algorithm based on artificial intelligence, and finally obtains an ultra-depth-of-field high-resolution image, avoiding the process of optimizing parameters and additionally designing components. The artificial intelligence algorithm can be used to achieve the effect of obtaining ultra-depth-of-field images without sacrificing image resolution. The depth of field of the present invention can be increased by about 5 times; by utilizing the characteristics of the ultra-depth of field of the present invention, a clear presentation of a curved surface with a certain curvature can be achieved, and image acquisition of dynamic samples with slight amplitude can also be achieved, broadening the scope of use of traditional microscopes. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of the principle of a multi-spectral and information fusion ultra-depth-of-field microscopic image acquisition system provided by the present invention;

[0034] Figure 2 A schematic diagram of the principle of the image receiving and microscopic imaging assembly provided by the present invention;

[0035] Figure 3 A block diagram of the computing components provided by the present invention;

[0036] Figure 4 A method for implementing the multi-spectral spectroscopic imaging device provided by the present invention;

[0037] Figure 5 Another implementation of the multi-spectral spectroscopic imaging device provided by the present invention. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0039] Figure 1 A schematic diagram of the structure of a multi-spectral and information-fused super-depth high-resolution image acquisition system provided by an embodiment of the present invention. This embodiment provides a multi-spectral and information-fused super-depth high-resolution image acquisition system, which includes an image receiving and microscopic imaging component 2 and a computing component 3, which are electrically connected; when the system is used to shoot the surface morphology of a sample, the image receiving and microscopic imaging component 2 shoots an image of a sample 1 with a height difference on the surface, and the computing component 3 performs data processing to finally obtain a super-depth high-resolution image of the sample 1. The magnification of the image receiving and microscopic imaging device is 60 times or more.

[0040] The image receiving and microscopic imaging component 2 shows the principle of imaging in the multi-spectral spectroscopic imaging component 5 by the microscopic system 4 composed of a multi-spectral illumination system, a microscope objective, a prism and a tube lens group, where the reflected light from the sample 1 is imaged. In the multi-spectral spectroscopic imaging component 5, it can be seen that light of different wavelengths is focused to different positions A, B, and C to collect information at different depth levels of the sample 1. It can be seen that light of different wavelengths corresponds to different focusing levels and has different depths of field. If A, B, and C correspond to the focusing level positions of the red, green, and blue bands, respectively, the depth of field formed is d1d2. The depth of field of a microscope has a physical depth of field: And geometric depth of field: Where e is the limiting resolution of the imaging element, n is the object side refractive index of the objective lens, usually taken as 1, ε is the limiting resolution of the human eye, usually taken as 1, M is the magnification of the microscope, NA is the numerical aperture of the microscope, λ is the wavelength of the incident light, and the larger value between the physical depth of field and the geometric depth of field is usually taken as the depth of field of the microscope.

[0041] like Figure 2 As shown, the image receiving and microscopic imaging assembly 2 includes a sample stage 6 for placing a sample 1, a microscope objective lens 7, a prism group 8, a multi-spectral illumination system 9 located at the same horizontal plane as the prism group 8, a tube lens 10, and a multi-spectral spectroscopic imaging device 11; wherein the microscope objective lens 7 is arranged above the sample stage 6; the prism group 8 is arranged above the microscope objective lens 7; the tube lens 8 is arranged above the prism group 7; and the multi-spectral spectroscopic imaging device 11 is located above the tube lens 10;

[0042] The multi-spectral illumination system 9 emits a multi-spectral light source with a freely adjustable wavelength of 400 to 1000 nm, for example, composed of 1 to N monochromatic lights such as red, yellow, green, and blue; the multi-spectral light source is reflected to the microscope objective 7 via the prism group 8, and is incident on the surface of the sample 1 through the microscope objective 7; the reflected light of the sample 1 passes through the microscope objective 7, the prism group 8, and the tube lens 10 in sequence and then is incident on the multi-spectral spectroscopic imaging device 11; the multi-spectral spectroscopic imaging device 11 is used to generate an image corresponding to each monochromatic light in the multi-spectral light source according to the reflected light carrying the surface information of the sample 1.

[0043] Furthermore, the multi-spectral spectroscopic imaging device 11 includes N imaging elements, N filters with different filtering bands, and a spectroscopic module; wherein N is the number of bands of interest selected from the multi-spectral light source, and a filter is arranged in front of each imaging element; the filter allows the light wavelength range to pass within the range of ±10nm of a monochromatic light wavelength; and each imaging element is a CCD image sensor or a CMOS photosensitive element.

[0044] The spectroscopic module is used to split the reflected light into N sub-beams, and the N sub-beams are incident on N filters respectively, and each filter generates monochromatic light of different bands according to its own filtering band; the N monochromatic lights of different bands are incident on an imaging element respectively, and the imaging element generates an image corresponding to each monochromatic light. In other words, the present invention splits the full-band light through filters to obtain monochromatic lights of different bands, and each imaging element collects the sample information carried by each monochromatic light to obtain multiple images corresponding to different colored lights.

[0045] The first implementation method of the optical splitting module is introduced below.

[0046] When N is an odd number, the spectroscopic module is composed of N-1 half-reflecting half-mirrors 12; wherein, every two half-reflecting half-mirrors 12 are arranged in a group left and right, and each group of half-reflecting half-mirrors 12 is arranged up and down according to the position of an alternating half-reflecting half-mirror, forming a stepped arrangement; at the same time, the four half-reflecting half-mirrors are numbered in sequence from left to right and from top to bottom, and the half-reflecting half-mirror located in the middle of the stepped arrangement is coaxial with the tube lens 10; N groups of imaging elements and filters are respectively arranged on the left side of the left side half-reflecting half-mirror, on the upper side of the upper side half-reflecting half-mirror, and on the right side of the right side half-reflecting half-mirror in the stepped arrangement.

[0047] For example, when N=2, the light splitting module is composed of 1 half-reflective half-mirror; when N=3 or 4, the light splitting module is composed of 3 half-reflective half-mirrors. On the basis of N=2, two half-reflective half-mirrors are added in the upward (downward) and left (right) directions. When N continues to increase, half-reflective half-mirrors are added in the upward (downward) and left (right) directions to realize more light beam imaging; Figure 4As shown, N=5, and the light splitting module is composed of four half-reflecting half-mirrors; wherein, every two half-reflecting half-mirrors are arranged in a group left and right, and the two groups of half-reflecting half-mirrors are arranged up and down according to the position of staggered half-reflecting half-mirrors; at the same time, the four half-reflecting half-mirrors are numbered in sequence from left to right and from top to bottom as half-reflecting half-mirrors I, half-reflecting half-mirrors II, half-reflecting half-mirrors III, and half-reflecting half-mirrors IV, and the half-reflecting half-mirrors II and the half-reflecting half-mirrors III are coaxial with the tube lens 10; five groups of imaging elements and filters are arranged on the left and upper side of the half-reflecting half-mirrors I, the upper side of the half-reflecting half-mirrors II, and the upper side and right side of the half-reflecting half-mirrors IV, respectively, and the magnification of the entire microscope optics is 100 times. .

[0048] The second implementation method of the light splitting module is introduced below.

[0049] The spectroscopic module is a rotating micro-galvanometer 13; wherein each rotation angle of the micro-galvanometer 13 corresponds to a group of imaging elements and filters, and N groups of imaging elements and filters are arranged around the micro-galvanometer 13 with the micro-galvanometer as the center. Then, the reflected light emitted by the tube lens 10 is divided into N paths by the micro-galvanometer 13 rotated to N different angles, and then respectively enters the N groups of filters and imaging elements.

[0050] For example, Figure 5 As shown, N=5, five groups of imaging elements and filters are arranged around the micro-galvanometer with the micro-galvanometer as the center. The reflected light emitted by the tube lens 10 is divided into five paths by the micro-galvanometer rotated to five different angles, and then respectively enters the five groups of filters and imaging elements. The magnification of the entire microscope optics is 100 times.

[0051] It should be noted that the wavelength band that can be freely adjusted for irradiation in the present invention is 400-1000nm. When five different wavelength bands of light are used to irradiate the sample, images with depth of field at five different focal lengths can be obtained respectively. Combined with the images of different focusing point planes obtained by the above-mentioned different imaging elements, after processing by the computing component 3, a clear image with a depth of field within a certain range can be obtained, which is about 5 times greater than the original depth of field (usually the light is in the 550nm wavelength band).

[0052] Furthermore, if Figure 3 As shown, the computing device 3 includes an image segmentation module 101, a decision generation module 102, a data fusion module 103 and an image enhancement module 104;

[0053] The image segmentation module 101 segments the clear area and the blurred area in each monochromatic light image in the multispectral image based on the feature gradient method, and obtains the activity level of each pixel point of the segmented image; wherein the activity level SF calculated pixel by pixel of any segmented image is (x,y) for:

[0054]

[0055] Among them, CF (x,y) is the gradient of the pixel at position (x, y) in the column direction, RF F(x,y) is the gradient of the pixel at position (x, y) in the row direction, r is the radius of the range of interest (i.e., kernel radius) selected when calculating the activity level of the pixel at position (x, y), F (x+a,y+b) is the region (x+a, y+b) centered at the pixel at position (x, y) on the segmented image output by the image segmentation neural network based on the feature gradient method, F (x+a-1,y+b) The area (x+a-1, y+b) centered on the pixel at position (x, y) on the segmented image output by the image segmentation neural network based on the feature gradient method; it can be seen that the core idea of ​​the image segmentation module 101 is to sharpen the imaging area within the depth of field (i.e., the clear area), and moderately blur the imaging area outside the depth of field (i.e., the blurred area), and obtain the activity level of each pixel point of the processed image; the activity level of each pixel point can provide an accurate decision basis for image fusion.

[0056] The decision generation module 102 is used to obtain a decision graph corresponding to each monochromatic image according to the activity level on each monochromatic image:

[0057]

[0058] Among them, D i(x,y) is the decision diagram corresponding to the monochromatic image corresponding to the i-th band, n represents the number of bands of the monochromatic image (i.e. the number of source images), SF i(x,y) Indicates the activity level of the monochromatic image corresponding to the i-th band, SF j(x,y) represents the activity level corresponding to the jth band; if SF i(x,y) Not less than SF j(x,y) , then D i(x,y) =1, if SF i(x,y) Smaller than any SF j(x,y) , then D i(x,y)=0; for example, assuming that there are 5 bands, the activity levels corresponding to the pixels in the first row and first column of the monochromatic image corresponding to the first band are not less than the activity levels corresponding to the pixels in the first row and first column of the monochromatic images corresponding to the other four bands, then the pixel value of the pixel in the first row and first column of the decision diagram corresponding to the first band is 1, if any one of the activity levels corresponding to the pixels in the first row and first column of the monochromatic image corresponding to the other four bands is greater than the activity level corresponding to the pixel in the first row and first column of the monochromatic image corresponding to the first band, then the pixel value of the pixel in the first row and first column of the decision diagram corresponding to the first band is 0; similarly, the pixel values ​​of the pixels in the other decision diagrams can be obtained, which will not be elaborated in the present invention.

[0059] The data fusion module 103 is used to perform pixel-level weighted fusion on each decision graph and the monochromatic light image corresponding to each decision graph to obtain a fused image M (x,y) :

[0060]

[0061] Among them, D i(x,y) Img is the decision graph corresponding to the monochromatic image corresponding to the i-th band, i (x, y) is the monochromatic light image corresponding to the i-th band.

[0062] The image enhancement module 104 is used to enhance the visual quality of the fused image by using a brightness enhancement method, a contrast enhancement method, and a sharpness enhancement method to obtain a final super-depth-of-field microscopic image.

[0063] It can be seen that when the image receiving and microscopic imaging component successfully collects images corresponding to light of different bands, these images are processed by the image segmentation module 101 and the decision generation module 102 of the computing component to obtain a decision graph corresponding to each image. In the data fusion module 103, each image is fused according to its corresponding decision graph to obtain an ultra-depth of field image, and finally, after the sharpness / contrast / brightness is enhanced in the image enhancement module 104, a clear image with a depth of field within a certain range is finally obtained.

[0064] In summary, the present invention utilizes the principle that when light of different wavelength bands illuminates an object and passes through a lens, the focusing plane positions are different. When the magnification and numerical aperture of the microscope objective are constant, when a multi-spectral illumination system is used to illuminate the object, the light is reflected by a prism group and then passes through the microscope objective to the surface of the sample to be measured. After the microscope objective receives the reflected light from the sample, it passes through a tube lens and a prism group to a multi-spectral spectroscopic imaging device. In the multi-spectral spectroscopic imaging device, the light carrying the sample surface information passes through a semi-reflective mirror / galvanometer and a filter to reach an imaging element, and finally generates an image corresponding to each monochromatic light in the multi-spectral illumination system.

[0065] At the same time, the present invention can also use the laser ranging method to measure the focal length. For the images of different bands of the sample to be tested, relying on the deep learning network of artificial intelligence, the image segmentation module is first applied to segment the images corresponding to each band, and the clear part of the image is extracted. Secondly, the decision diagram generation module is applied to generate the decision diagram of the image corresponding to each band, and the data fusion module is used again to fuse the decision diagrams. Finally, the image enhancement module is applied to enhance the visual sense of the image to obtain the final ultra-depth of field and high-resolution image.

[0066] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may certainly make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A multi-spectral and information fusion ultra-depth microscopic image acquisition system, characterized in that: It includes an image receiving and microscopic imaging component (2) and a computing component (3); When the system is used to photograph the surface morphology of a sample, the image receiving and microscopic imaging component (2) is used to photograph a multispectral image of a sample (1) having a height difference on the surface, and send the multispectral image to a computing device (3); The computing device (3) is used to perform image processing on the multispectral image to obtain a super-depth-of-field microscopic image of the sample (1).

2. The multi-spectral and information fusion ultra-depth microscopic image acquisition system according to claim 1, characterized in that: The image receiving and microscopic imaging component (2) comprises a sample stage (6) for placing a sample (1), a microscope objective lens (7), a prism group (8), a multi-spectral illumination system (9) located at the same level as the prism group (8), a tube lens (10), and a multi-spectral spectroscopic imaging device (11); wherein the microscope objective lens (7) is arranged above the sample stage (6); the prism group (8) is arranged above the microscope objective lens (7); the tube lens (10) is arranged above the prism group (8); and the multi-spectral spectroscopic imaging device (11) is located above the tube lens (10); The multi-spectral illumination system (9) emits a multi-spectral light source with a wavelength of 400 to 1000 nm. The multi-spectral light source is reflected to the microscope objective lens (7) via a prism group (8), and incident on the surface of the sample (1) through the microscope objective lens (7). The reflected light of the sample (1) passes through the microscope objective lens (7), the prism group (8), and the tube lens (10) in sequence and then is incident on the multi-spectral spectroscopic imaging device (11). The multi-spectral spectroscopic imaging device (11) is used to generate an image corresponding to each monochromatic light in the multi-spectral light source according to the reflected light carrying the surface information of the sample (1).

3. A multi-spectral and information fusion ultra-depth microscopic image acquisition system as claimed in claim 2, characterized in that: The multi-spectral spectroscopic imaging device (11) comprises N imaging elements, N filters with different filtering bands, and a spectroscopic module; wherein N is the number of bands of interest selected from the multi-spectral light source, and a filter is arranged in front of each imaging element; The spectroscopic module is used to split the reflected light into N sub-beams, and the N sub-beams are respectively incident on N filters, each filter generates monochromatic light of different bands according to its own filtering band; the N monochromatic lights of different bands are respectively incident on an imaging element, and the imaging element generates an image corresponding to each monochromatic light.

4. A multi-spectral and information fusion ultra-depth microscopic image acquisition system as claimed in claim 3, characterized in that: When N is an odd number, the light splitting module is composed of N-1 half-reflecting mirrors; wherein, every two half-reflecting mirrors form a group and are arranged left and right, and each group of half-reflecting mirrors is arranged up and down according to the position of a staggered half-reflecting mirror, forming a stepped arrangement; at the same time, each half-reflecting mirror is numbered in sequence from left to right and from top to bottom, and the half-reflecting mirror located in the middle of the stepped arrangement is coaxial with the tube lens (10); N groups of imaging elements and filters are respectively arranged on the left side of the left side half-reflecting mirror, on the upper side of the upper side half-reflecting mirror, and on the right side of the right side half-reflecting mirror in the stepped arrangement.

5. The multi-spectral and information fusion ultra-depth microscopic image acquisition system according to claim 3, characterized in that: When N=5, and the light splitting module is composed of four half-reflecting half-mirrors; wherein, every two half-reflecting half-mirrors form a group and are arranged left and right, and the two groups of half-reflecting half-mirrors are arranged up and down according to the position of one half-reflecting half-mirror being staggered; at the same time, the four half-reflecting half-mirrors are numbered in sequence from left to right and from top to bottom as half-reflecting half-mirror I, half-reflecting half-mirror II, half-reflecting half-mirror III, and half-reflecting half-mirror IV, and the half-reflecting half-mirror II and the half-reflecting half-mirror III are coaxial with the tube lens (10); and five groups of imaging elements and filters are respectively arranged on the left side and upper side of the half-reflecting half-mirror I, the upper side of the half-reflecting half-mirror II, and the upper side and right side of the half-reflecting half-mirror IV.

6. The multi-spectral and information fusion ultra-depth microscopic image acquisition system according to claim 3, characterized in that: The light splitting module is a rotating micro-vibrator; wherein each rotation angle of the micro-vibrator corresponds to a group of imaging elements and filters, and N groups of imaging elements and filters are arranged around the micro-vibrator with the micro-vibrator as the center. Then, the reflected light emitted by the tube lens (10) is divided into N paths by the micro-vibrator rotated to N different angles, and then respectively enters the N groups of filters and imaging elements.

7. The multi-spectral and information fusion ultra-depth microscopic image acquisition system according to claim 3, characterized in that: When N=5, and the light splitting module is a rotating micro-galvanometer; wherein each rotation angle of the micro-galvanometer corresponds to a group of imaging elements and filters, and five groups of imaging elements and filters are arranged around the micro-galvanometer with the micro-galvanometer as the center, then the reflected light emitted by the tube lens (10) is divided into five paths by the micro-galvanometer rotated to five different angles, and then respectively enters the five groups of filters and imaging elements.

8. The multi-spectral and information fusion ultra-depth microscopic image acquisition system according to claim 1, characterized in that: The computing device (3) comprises an image segmentation module, a decision generation module, a data fusion module and an image enhancement module; The image segmentation module segments the clear area and the blurred area in each monochromatic light image in the multispectral image based on the characteristic gradient method, and obtains the activity level of each pixel point of the segmented image; The decision generation module is used to obtain a decision graph corresponding to each monochromatic light image according to the activity level on each monochromatic light image: Among them, D i(x,y) is the decision diagram corresponding to the monochromatic image corresponding to the i-th band, n represents the number of bands of the monochromatic image, SF i(x,y) Indicates the activity level of the monochromatic image corresponding to the i-th band, SF j(x,y) represents the activity level corresponding to the jth band; if SF i(x,y) Not less than SF j(x,y) , then D i(x,y) =1, if SF i(x,y) Smaller than any SF j(x,y) , then D i(x,y) =0; The data fusion module is used to perform pixel-level weighted fusion on each decision graph and the monochromatic light image corresponding to each decision graph to obtain a fused image; The image enhancement module is used to visually enhance the fused image by using a brightness enhancement method, a contrast enhancement method, and a sharpness enhancement method to obtain a final super-depth-of-field microscopic image.

9. The multi-spectral and information fusion ultra-depth microscopic image acquisition system according to claim 8, characterized in that: The activity level SF calculated pixel by pixel for any segmented image (x,y) for: Among them, CF (x,y) is the gradient of the pixel at position (x, y) in the column direction, RF F(x,y) is the gradient of the pixel at position (x, y) in the row direction, r is the radius of the range of interest selected when calculating the activity level of the pixel at position (x, y), and F (x+a,y+b) is the region (x+a, y+b) centered at the pixel at position (x, y) on the segmented image output by the image segmentation neural network based on the feature gradient method, F (x+a-1,y+b) It is the area (x+a-1,y+b) centered on the pixel at position (x,y) on the segmented image output by the image segmentation neural network based on the feature gradient method.

10. The multi-spectral and information fusion ultra-depth microscopic image acquisition system according to claim 8, characterized in that: Fusion image M (x,y) The method to obtain is: Among them, D i(x,y) Img is the decision graph corresponding to the monochromatic image corresponding to the i-th band, i (x, y) is the monochromatic light image corresponding to the i-th band.

Images