Microscopic imaging apparatus and method having depth reconstruction effect

Abstract

A microscopic imaging apparatus and method having a depth reconstruction effect, the method comprising: by means of an image acquisition component, performing image acquisition on an object to be measured to obtain a first image and a second image of the object to be measured; according to inverse filters of a first photographing component and a second photographing component corresponding to a plurality of preset object distances, and a transformation matrix between the first photographing component and the second photographing component at each object distance, acquiring an offset loss amount of any pixel point at a corresponding object distance; then according to offset loss amounts of the any pixel point at all object distances, obtaining a depth value of the pixel point, thereby completing depth reconstruction and obtaining an ultra-depth-of-field image and a depth map. By adding a specially designed phase mask piece, the optical ultra-depth-of-field effect is achieved; and since the depth reconstruction process does not require multiple times of photographing or the use of a displacement platform, the ultra-depth-of-field image and the depth image can be obtained by means of only one photographing by means of dual cameras, which is simpler, more efficient, and easy to implement.

Description

Microscopic imaging device and method with depth reconstruction effect TECHNICAL FIELD

[0001] The present application relates to the field of microscopic three-dimensional imaging, in particular to a microscopic imaging device and method with depth reconstruction effect. BACKGROUND

[0002] With the development of mold design, precision machining, product detection and industrial measurement, the traditional two-dimensional image sensor technology is increasingly unable to meet the needs of production scenarios, and the demand for three-dimensional measurement technology is growing rapidly. Compared with two-dimensional image sensor technology, three-dimensional measurement technology adds depth information, and because depth itself is not sensitive to factors such as posture and lighting, three-dimensional measurement technology has greater advantages in object recognition and measurement tasks.

[0003] In the field of microscopic three-dimensional imaging, due to the extremely small depth of field of the objective lens, it is often necessary to use a motion structure and image synthesis to obtain the effect of super depth of field and the depth information of the object surface. The current mainstream microscopic three-dimensional reconstruction scheme is an interferometer or a multi-focal surface focus change. In reconstruction, a displacement platform is used to move along the depth of field direction, and the target image is synthesized by multiple shootings at different positions. However, the use of the displacement platform and the time division sampling will introduce system disturbance error, resulting in error in the obtained measurement results. SUMMARY

[0004] The technical problem solved by the present application is how to realize depth reconstruction without using a displacement platform and multiple sampling.

[0005] According to a first aspect, in an embodiment, a microscopic imaging device with depth reconstruction effect is provided, comprising a microscopic objective lens, a tube lens, a light source, a lifting platform and a plurality of light splitting devices, further comprising:

[0006] A phase mask is located between the microscopic objective lens and one light splitting device.

[0007] An image acquisition component comprises at least a first shooting component and a second shooting component. The first shooting component and the second shooting component have their corresponding inverse filters at a plurality of preset object distances, and at each object distance, the first shooting component and the second shooting component have a corresponding transformation matrix.

[0008] A processor is configured to:

[0009] Control the lifting platform to move in the vertical direction according to a preset movement step length.

[0010] obtaining inverse filters corresponding to a plurality of preset object distances for the first and second photographing components, and a transformation matrix between the first and second photographing components at each object distance;

[0011] controlling the first and second photographing components to simultaneously perform image acquisition on the object to be measured to obtain a first image and a second image;

[0012] for any one object distance: performing filter processing on the first image according to the inverse filter corresponding to the object distance for the first photographing component to obtain a third image corresponding to the object distance; performing filter processing on the second image according to the inverse filter corresponding to the object distance for the second photographing component to obtain a fourth image corresponding to the object distance; performing rigid transformation on the fourth image according to the transformation matrix between the first and second photographing components at the object distance to obtain a fifth image corresponding to the object distance; and obtaining an offset loss amount of any one pixel at the object distance according to pixel values corresponding to the pixel in the obtained third and fifth images;

[0013] obtaining a depth value of the any one pixel according to offset loss amounts corresponding to all object distances of the pixel, and obtaining a depth map corresponding to the object to be measured according to depth values of all pixels, thereby completing depth reconstruction.

[0014] According to a second aspect, in an embodiment, a microscopic imaging method with depth reconstruction effect is provided, comprising:

[0015] performing image acquisition on an object to be measured by an image acquisition component, wherein the image acquisition component comprises at least a first photographing component and a second photographing component, and the first and second photographing components simultaneously perform image acquisition to obtain a first image and a second image of the object to be measured; wherein the first and second photographing components each have an inverse filter corresponding to a plurality of preset object distances, and a transformation matrix corresponding to the first and second photographing components at each object distance;

[0016] for any one object distance: performing filter processing on the first image according to the inverse filter corresponding to the object distance for the first photographing component to obtain a third image corresponding to the object distance; performing filter processing on the second image according to the inverse filter corresponding to the object distance for the second photographing component to obtain a fourth image corresponding to the object distance; performing rigid transformation on the fourth image according to the transformation matrix between the first and second photographing components at the object distance to obtain a fifth image corresponding to the object distance; and obtaining an offset loss amount of any one pixel at the object distance according to pixel values corresponding to the pixel in the obtained third and fifth images;

[0017] The depth value of each pixel point is obtained according to the offset loss amount of the pixel point corresponding to all object distances, and the depth values of all pixel points constitute a depth map corresponding to the object to be measured.

[0018] According to the microscopic imaging device and method with depth reconstruction effect, the specially designed phase mask is added in the optical system, so that the point spread function after imaging remains unchanged within a certain range, the depth of field of the optical system is greatly expanded, the super depth of field effect is realized, and the phenomenon that the depth of field of the ordinary microscope is extremely small due to the expansion of the aperture is improved; in the depth reconstruction process, first, the corresponding relationship between the offset amount difference of the point spread functions corresponding to different object distances and different shooting components is implicitly established based on the inverse filters corresponding to different shooting components obtained in the calibration process and the transformation matrix between different shooting components, so that the change of the corresponding offset amount of the point spread function caused by the object distance change of different shooting components is recorded; then, in the actual use process, the real depth of the surface of the object to be measured is inversely deduced according to the corresponding offset loss amount of the object to be measured at different object distances, so that the depth reconstruction is completed, and the corresponding super depth of field image is obtained; since the depth reconstruction process no longer needs to shoot multiple times or use a displacement platform, the super depth of field image and the depth image can be obtained through only one shooting of the double cameras, which is more simple and efficient; by filtering the image through the inverse filter corresponding to different object distances, the image is restored to the original image, so that the triangular ghosting caused by any one point passing through the phase mask can be eliminated, thereby improving the imaging artifact problem existing in the reconstruction of the traditional phase mask; in addition, since the application does not need special auxiliary lighting, and all other optical devices except the phase mask have standard products, the application is also easier to implement. BRIEF DESCRIPTION OF DRAWINGS

[0019] Fig. 1 is a process flow diagram of a processor in a microscopic imaging device with depth reconstruction effect;

[0020] Fig. 2 is a method flow diagram of a microscopic imaging method with depth reconstruction effect;

[0021] Fig. 3 is a structural schematic diagram of a microscopic imaging device with depth reconstruction effect;

[0022] Fig. 4 is a shape diagram of a PSF function after inserting a phase mask;

[0023] Fig. 5 is a diagram showing the change of the offset amount of different shooting components at different object distances;

[0024] Fig. 6 is an image corresponding to a part of a standard object in a first standard image;

[0025] Fig. 7 is a diagram showing the depth reconstruction result;

[0026] Fig. 8 is a diffraction pattern of a certain pinhole of the standard object.

[0027] Reference numerals: 11 - micro-objective; 12 - tube lens; 13 - light source; 14 - lifting platform; 15 - light splitting device; 16 - phase mask; 17 - image acquisition component. DETAILED DESCRIPTION

[0028] The application will be further described below in connection with specific embodiments with reference to the drawings. Like numbers in different figures refer to like elements throughout. In the following description, numerous specific details are described to provide a thorough understanding of the application. However, it will be apparent to one skilled in the art that the application can be practiced without these specific details. In other instances, well-known structures have not been described in detail in order to avoid obscuring the application. Some operations are not shown or described in detail in the description since they are known to one skilled in the art, and they are not necessary for understanding the application.

[0029] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various embodiments. Meanwhile, the steps or actions in the method description can also be sequentially adjusted or changed in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment, and do not mean that the sequence is necessary, unless otherwise stated that a certain sequence must be followed.

[0030] The serial numbers of components in this paper, such as "first", "second", etc., are only used to distinguish the described objects, and do not have any sequence or technical meaning. Unless otherwise specified, the "connection" and "coupling" in this application include direct and indirect connection (coupling).

[0031] The phase mask belongs to the wavefront coding technology, which extends the depth of field of the optical system by modulating the pupil phase distribution function. The point spread function at different object distances has a certain shift, but the shape, size and blur distribution are almost completely consistent.

[0032] The embodiment of the present application aims to design a special optical element (i.e. phase mask) to make the shape of modulation transfer function and point spread function of the optical system insensitive to object distance, greatly expand the depth of field of the optical system, and based on the imaging characteristics of the microscopic imaging system, the depth information of the object surface can be obtained after one shooting of the dual camera, so that the result with super depth of field effect and depth reconstruction effect can be obtained without using displacement platform and multiple sampling.

[0033] Some embodiments provide a microscopic imaging device with depth reconstruction effect, and a structural schematic diagram thereof is shown in FIG. 3, which comprises a microscopic objective 11, a tube lens 12, a light source 13, a lifting platform 14, and a plurality of light splitting devices 15, and further comprises a phase mask 16, an image acquisition component 17, and a processor; in the microscopic imaging device of the embodiment, the light generated by the incoherent light source 13 is irradiated on the object to be measured, passes through the microscopic objective 11, is modulated by the phase mask 16, and then passes through the light splitting device 15, and is imaged by two image acquisition components 17 of the same model with different image distances; the following is specifically explained:

[0034] The phase mask 16 in the embodiment is a bi-cubic phase mask, which is located between the microscopic objective 11 and one light splitting device 15, and by placing the special surface type phase mask 16 at the pupil of the microscopic objective 11, the phase distribution function at the pupil is modulated, and the shape of the modulation transfer function and the point spread function of the optical system is insensitive to the object distance, and at the same time, the wavefront coding technology is used to greatly expand the depth of field of the optical system, wherein the bi-cubic constraint satisfied by the phase distribution function at the pupil after optical optimization is:

[0035] wherein, represents the phase distribution function, u and v are the position coordinates on the pupil, and a is the gain coefficient for controlling the bending strength of the surface, which is obtained by optical software (such as Zemax).

[0036] The image acquisition component 17 in the embodiment at least comprises a first shooting component and a second shooting component; wherein the first shooting component and the second shooting component have corresponding inverse filters at a plurality of preset object distances, and at each object distance, there is also a corresponding transformation matrix between the first shooting component and the second shooting component, which is used to convert the images simultaneously collected by the first shooting component and the second shooting component to the same coordinate system.

[0037] It should be noted that the optical system after adding the phase mask has the following characteristics: on the one hand, the modulation transfer function (MTF function) of the optical system imaging will not decrease to 0, so that the inverse transform of image degradation can be performed using deconvolution, that is, the output image of the optical system is restored to the original input image; on the other hand, the shape of the light intensity distribution corresponding to the point spread function (PSF) of the optical system will not change in a large object distance range (super depth of field range), and the shape is a triangular diffraction fringe with a shifted symmetric center; and in the super depth of field range, although the shape of the light intensity distribution does not change, the position of the imaging spot of a single point light source after passing through the phase mask will appear a displacement, that is, the center position of the spot will appear a shift, and the shift amount will change with the change of the working distance (or object distance), wherein the curve shape of the function δ(w) formed by the shift amount and the working distance is approximately a parabola, and the change of the shift amount corresponding to different shooting components at different object distances is shown in FIG. 5, wherein δ represents the shift amount, and w represents the working distance / object distance.

[0038] According to the imaging characteristics of the dual-cubic phase mask, if two shooting components are fixedly placed at different image distances, so as to keep the difference of the fixed back focal length, the difference between the shift amounts corresponding to the two shooting components at the same object distance and the working distance forms a monotonic continuous function Δδ(w) as shown in FIG. 5, and at this time, if the corresponding relationship between the shift amount difference and the working distance can be obtained in advance, the corresponding working distance can be restored according to the calculated shift amount difference in real-time shooting.

[0039] The process of obtaining the corresponding relationship between the shift amount difference and the working distance in the embodiment is called a calibration process, and in the calibration process, the inverse filter corresponding to the first shooting component and the second shooting component at a plurality of preset object distances, and the transformation matrix between the first shooting component and the second shooting component at each object distance are obtained.

[0040] In the calibration process, the processor controls the lifting platform 14 to move in the vertical direction according to a preset moving step, and after each movement of the lifting platform 14, the image acquisition component 17 is controlled to perform image acquisition on the pre-set standard object to obtain the images of the standard object corresponding to a plurality of preset object distances, wherein the standard object is placed on the lifting platform and is an opaque plane plate with equally spaced ultra-fine pinholes, and a backplane light source is arranged on the opaque plane plate to assist lighting to illuminate the ultra-fine pinholes on the opaque plane plate; and the specific calibration process of the embodiment is as follows:

[0041] Firstly, a non-transparent plane plate with ultra-fine pinholes distributed at equal intervals is placed on a lifting platform 14 equipped with a stepping motor, and a back light source is set to make the pinholes on the non-transparent plane plate bright, wherein the lifting direction of the lifting platform 14 is perpendicular to the optical axis of the objective lens; then, starting from the position with a nominal working distance of 0, the lifting platform 14 is controlled to move in the vertical direction according to a preset moving step, for example, the preset moving step can be set to 1 μm, corresponding to the lifting platform 14 moving upward by 1 μm each time; after each movement of the lifting platform 14, the pre-set standard object is imaged by the image acquisition component 17 until the range of the working distance is exceeded, and the two images of the standard object acquired by the first and second shooting components at each movement and the corresponding lifting platform position are recorded, thereby obtaining two images of the standard object corresponding to different object distances, wherein for any object distance: the two images obtained by the first and second shooting components after imaging the standard object at the object distance are respectively referred to as the first and second standard images.

[0042] In the present embodiment, after the pinholes on the non-transparent plane plate are illuminated by the back light source, the originally concentrated light will form an extended light spot on the image plane, as shown in FIG. 4, the light intensity distribution of a point after passing through the phase mask appears as a triangular tailing, at this time, the light intensity distribution of the light spot is the PSF function; in the present embodiment, the corresponding PSF functions are calculated by optical software (for example, Zemax) according to the diffusion patterns of the pinholes on the non-transparent plane plate in the first and second standard images, wherein the corresponding image of the part of the standard object in the first standard image is shown in FIG. 6, and FIG. 8 is a partial enlarged view of FIG. 6, which shows the diffusion pattern formed by a pinhole of the standard object.

[0043] Since the modulation transfer function of the system has no zero point and the shape of the PSF function is not sensitive to the field position, the inverse transform thereof can be solved by Fourier transform, and then the inverse filter corresponding to the obtained PSF function is constructed by Wiener filtering, that is, for any object distance, the inverse filter corresponding to the first shooting component at the object distance is obtained according to the PSF function corresponding to the first standard image; the inverse filter corresponding to the second shooting component at the object distance is obtained according to the PSF function corresponding to the second standard image; thereby obtaining the inverse filters corresponding to the first and second shooting components at the object distance.

[0044] Then, the first standard image and the second standard image are filtered by the obtained inverse filter to restore the degraded image containing the point spread function noise to a clear original image; wherein the first standard image is filtered by the inverse filter corresponding to the object distance through the first shooting component to obtain a third standard image; the second standard image is filtered by the inverse filter corresponding to the object distance through the second shooting component to obtain a fourth standard image; thus the first standard image and the second standard image are restored to the original image containing the standard object super-fine pinhole array, that is, the third standard image and the fourth standard image contain an array of equally spaced dots at this time, and each dot corresponds to a super-fine pinhole on the standard object.

[0045] The center point coordinates of each dot in the third standard image and the fourth standard image are obtained respectively, such as the center point coordinates of each dot in the third standard image and the fourth standard image obtained by a dot extraction algorithm; since each super-fine pinhole on the standard object has a corresponding dot in the third standard image and the fourth standard image, a transformation matrix can be obtained according to the center point coordinates of the corresponding dots in the third standard image and the fourth standard image by a linear least square method, that is:

[0046] Wherein, cam1 represents the first shooting component; cam2 represents the second shooting component; is a matrix composed of the center point coordinates of all dots in the third standard image corresponding to the first shooting component when the object distance is w; is a matrix composed of the center point coordinates of all dots in the fourth standard image corresponding to the second shooting component when the object distance is w; w is a transformation matrix between the first shooting component and the second shooting component when the object distance is w, which is a rigid transformation matrix containing rotation and translation.

[0047] The change of the offset value caused by the object distance transformation can also be obtained through the center point coordinates of the corresponding dots in the images captured by the dual-camera, so that the transformation matrix implicitly contains the offset difference between different shooting components in the process of converting the images corresponding to the first shooting component and the second shooting component to the same coordinate system, that is, the transformation matrix corresponding to different object distances can implicitly establish a corresponding relationship between different object distances and the corresponding offset difference.

[0048] In the embodiment, the processor is further configured to perform depth reconstruction to obtain a depth map and a super-depth-of-field image, and the depth reconstruction process performed by the processor is shown in FIG. 1, which includes:

[0049] Step S100: obtaining inverse filters corresponding to a plurality of preset object distances of the first shooting component and the second shooting component, and a transformation matrix between the first shooting component and the second shooting component at each object distance.

[0050] obtaining the inverse filters corresponding to the plurality of preset object distances between the first photographing component and the second photographing component, and the transformation matrix between the first photographing component and the second photographing component at each object distance.

[0051] Step S110: controlling the first photographing component and the second photographing component to simultaneously perform image acquisition on the measured object, to obtain a first image and a second image.

[0052] The measured object is placed on a plane with an object distance m, where m is unknown, that is, the embodiment only needs to ensure that the measured object is fixed and immovable within the field of view of the photographing component, without needing to know the specific object distance at which the measured object is placed.

[0053] Step S120: obtaining the offset loss of any pixel point at each object distance according to the obtained first image and second image.

[0054] Since the function Δδ(w) formed by the offset difference and the corresponding working distance is a monotonic and continuous function, for any pixel point p within the imaging range of the photographing component, there should be a unique object distance z that satisfies:

[0055] where I cam1 is the image collected by the first photographing component, corresponding to the first image in the embodiment; I cam2 is the image collected by the second photographing component, corresponding to the second image in the embodiment; denotes the inverse filter corresponding to the first photographing component at the object distance z; denotes the inverse filter corresponding to the second photographing component at the object distance z; T z denotes the transformation matrix between the first photographing component and the second photographing component at the object distance z; ‖‖ denotes the L2 norm; · denotes dot product; and * denotes convolution operation.

[0056] At this time, z is the depth value corresponding to the pixel point p; the embodiment traverses the plurality of preset object distances stored in advance, performs image transformation based on the inverse filters and the transformation matrixes corresponding to different object distances, the difference between the image transformation results is the difference caused by the offset between the point spread functions corresponding to different photographing components, the embodiment refers to the difference degree between the corresponding pixel points in the image transformation results as the offset loss of the pixel point, and then obtains the depth value of the pixel point by minimizing the difference degree corresponding to the pixel point, that is, the object distance that minimizes the offset loss of the pixel point is taken as the depth value of the pixel point.

[0057] Then, for any object distance, the first image is filtered according to the inverse filter corresponding to the object distance for the first shooting component to obtain a third image corresponding to the object distance; and the second image is filtered according to the inverse filter corresponding to the object distance for the second shooting component to obtain a fourth image corresponding to the object distance.

[0058] Since the transformation matrix in the embodiment is used to convert the image filtered by the second shooting component into the coordinate system of the image filtered by the first shooting component, the fourth image is rigidly transformed according to the transformation matrix between the first shooting component and the second shooting component at the object distance to obtain a fifth image corresponding to the object distance.

[0059] At this time, the fifth image and the third image are in the same coordinate system, and the difference between the two is the difference caused by the offset between the point spread functions corresponding to different shooting components. The difference between the third image and the fifth image for any pixel point is evaluated according to the pixel values corresponding to the pixel point in the third image and the fifth image, and the difference obtained is the offset loss corresponding to the object distance for the pixel point.

[0060] For example, for any pixel point, the gray values of the pixel point in different channels in the third image and the fifth image are obtained, and the norm of the difference between the corresponding gray values in different channels for the pixel point is calculated, such as the L2 norm, and the norm obtained is taken as the offset loss corresponding to the object distance for the pixel point.

[0061] Step S130: The depth value of any pixel point is obtained according to the offset loss corresponding to all object distances, and the depth values of all pixel points constitute the depth map corresponding to the object to be measured, and the depth reconstruction is completed.

[0062] For any pixel point, the minimum value of the offset loss corresponding to all preset object distances for the pixel point is obtained, and the object distance corresponding to the minimum value is taken as the depth value of the pixel point, and the depth values of all pixel points constitute the depth map corresponding to the object to be measured, and the depth reconstruction is completed; the result obtained after the depth reconstruction of a block-shaped object in the embodiment is shown in FIG. 7.

[0063] In addition, the embodiment can also obtain a super-depth-of-field image, wherein for any pixel point in the super-depth-of-field image, the third image and the fifth image corresponding to the depth value of the pixel point as the object distance are obtained, and the average value of the corresponding gray values of the pixel point in each channel in the obtained third image and the fifth image is calculated, and the average value obtained is taken as the gray value of the pixel point in the corresponding channel in the super-depth-of-field image.

[0064] The embodiment adds a specially designed phase mask in an optical system, so that the point spread function after imaging remains unchanged within a certain range, greatly expands the depth of field of the optical system, realizes the optical super depth of field effect, and improves the phenomenon that the depth of field of an ordinary microscope is extremely small due to expansion of the aperture; in the depth reconstruction process, first, based on the inverse filters corresponding to different shooting components obtained in the calibration process and the transformation matrix between different shooting components, a corresponding relationship between the offset difference of different object distances and the corresponding point spread functions of different shooting components is implicitly established, thereby recording the change of the corresponding offset of the point spread function caused by the change of the object distance of the different shooting components; then in the actual use process, the real depth of the object surface is inversely deduced according to the corresponding offset loss of the object at different object distances, so as to complete the depth reconstruction and obtain the corresponding super depth of field image; since the embodiment does not need to shoot multiple times or use a displacement platform in the depth reconstruction process, the super depth of field image and the depth image can be obtained through only one shooting of the double cameras, so the embodiment is more simple and efficient; by filtering the image through the inverse filter corresponding to the different object distances, the image can be restored to the original image, and the triangular ghosting caused by any one point passing through the phase mask can be eliminated, that is, by considering the imaging offset corresponding to all object distances, the imaging artifact problem existing in the reconstruction by the traditional phase mask can be improved; in addition, the embodiment does not need special auxiliary lighting, and all other optical devices except the phase mask have standard products, so the whole scheme is also easier to implement.

[0065] Referring to FIG. 2, some embodiments provide a microscopic imaging method with a depth reconstruction effect, specifically comprising:

[0066] Step S200: image acquisition of the object to be measured is performed by the image acquisition component to obtain a first image and a second image of the object to be measured.

[0067] In the embodiment, the image acquisition component at least includes a first shooting component and a second shooting component, and the first shooting component and the second shooting component simultaneously perform image acquisition to obtain a first image and a second image of the object to be measured; wherein the first shooting component and the second shooting component have corresponding inverse filters at a plurality of preset object distances, and at each object distance, there is a corresponding transformation matrix between the first shooting component and the second shooting component, which is used to convert the images corresponding to the first shooting component and the second shooting component to the same coordinate system.

[0068] Step S210: according to the first image and the second image of the object to be measured, the offset loss of any one pixel point at each object distance is obtained.

[0069] For any one object distance: filtering the first image according to the inverse filter corresponding to the object distance of the first shooting component to obtain a third image corresponding to the object distance; filtering the second image according to the inverse filter corresponding to the object distance of the second shooting component to obtain a fourth image corresponding to the object distance; and performing rigid transformation on the fourth image according to the transformation matrix between the first shooting component and the second shooting component at the object distance to obtain a fifth image corresponding to the object distance; obtaining the offset loss amount of any one pixel point at the object distance according to the pixel values corresponding to the pixel point in the obtained third image and fifth image.

[0070] Step S220: obtaining the depth value of any one pixel point according to the offset loss amounts corresponding to the pixel point at all object distances, and the depth values of all pixel points constitute the depth map corresponding to the object to be measured.

[0071] It should be noted that the method steps in the embodiment correspond to the processing steps of the processor in the above embodiment, and the specific implementation manner has been described in the above embodiment, which will not be described here again.

[0072] The embodiment first obtains the first image and the second image by image acquisition component for image acquisition of the object to be measured placed at an unknown position; then filters the corresponding images according to the inverse filter corresponding to different shooting components at different object distances to eliminate the point spread function noise contained in the images, so as to restore the images to clear original images, thereby obtaining third images and fourth images corresponding to different object distances; and then performs image transformation according to the transformation matrix of different shooting components at corresponding object distances to obtain fifth images corresponding to different object distances, so as to convert the images corresponding to different shooting components to the same coordinate system, i.e. converting the fourth image to the coordinate system corresponding to the third image. At this time, the difference between the fifth image and the third image is the difference caused by the offset of the point spread function. Therefore, the offset loss amount of the same pixel point at different object distances is obtained according to the difference between the third image and the fifth image, and the depth map corresponding to the pixel point is obtained by minimizing the offset loss amount, so as to complete the depth reconstruction and obtain the corresponding super depth of field image. Since the embodiment does not need to be photographed multiple times or use a displacement platform, the super depth of field image and the depth image can be obtained by only one shooting of the double camera, which is more simple and efficient.

[0073] Those skilled in the art can understand that all or part of the functions of various methods in the above embodiments can be realized by hardware or by a computer program. When all or part of the functions in the above embodiments are realized by a computer program, the program can be stored in a computer readable storage medium, which can include a read-only memory, a random access memory, a magnetic disk, an optical disk, a hard disk, and the like. The above functions are realized by executing the program by a computer. For example, the program is stored in a memory of a device, and the above functions are realized by executing the program in the memory by a processor. In addition, when all or part of the functions in the above embodiments are realized by a computer program, the program can also be stored in a storage medium such as a server, another computer, a disk, an optical disk, a flash disk, or a mobile hard disk, and is saved in a memory of a local device by downloading or copying, or the system of the local device is updated, and the above functions are realized by executing the program in the memory by a processor.

[0074] The above application of specific examples to the present application is described, which is only used to help understand the present application and does not limit the present application. For those skilled in the art, according to the idea of the present application, a number of simple deductions, deformations or substitutions can be made.

Claims

1. A microscopic imaging apparatus having a depth reconstruction effect, comprising a microscope objective, a tube lens, a light source, a stage, and a plurality of light splitting devices, characterized in that, Also comprising: a phase mask between the objective lens and a light splitting device; an image acquisition component comprising at least a first camera and a second camera, wherein each of the first camera and the second camera has a corresponding inverse filter at each of a plurality of preset object distances, and a corresponding transformation matrix between the first camera and the second camera at each of the plurality of preset object distances; and a processor configured to: control the lifting platform to move in the vertical direction according to a preset moving step; obtain the inverse filter corresponding to each of the plurality of preset object distances for each of the first camera and the second camera, and the transformation matrix between the first camera and the second camera at each of the plurality of preset object distances; control the first camera and the second camera to simultaneously acquire images of the object to be measured to obtain a first image and a second image; for any one of the plurality of preset object distances: filter the first image according to the inverse filter corresponding to the object distance to obtain a third image corresponding to the object distance; filter the second image according to the inverse filter corresponding to the object distance to obtain a fourth image corresponding to the object distance; and perform rigid transformation on the fourth image according to the transformation matrix between the first camera and the second camera at the object distance to obtain a fifth image corresponding to the object distance; and obtain a depth value of any one pixel point according to the pixel values of the pixel point in the third image and the fifth image, wherein the depth values of all pixel points constitute a depth map corresponding to the object to be measured, and depth reconstruction is completed. The phase mask comprises a bicubic phase mask.

2. The microscopic imaging apparatus of claim 1, wherein, The processor is further configured to control the image acquisition component to acquire images of a preset standard object after each movement of the lifting platform to obtain images of the standard object corresponding to the plurality of preset object distances, wherein the standard object is placed on the lifting platform.

3. The microscopic imaging apparatus of claim 1, wherein, The preset standard object comprises an opaque flat plate with equally spaced ultra-fine pinholes, and the opaque flat plate is provided with a backlight source for auxiliary illumination.

4. The microimaging device of claim 3, wherein, The processor obtains the inverse filter corresponding to each of the plurality of preset object distances for each of the first camera and the second camera by:

5. The microscopic imaging apparatus of claim 1, wherein, acquiring images of the preset standard object at the plurality of preset object distances by the first camera and the second camera respectively; for any one of the plurality of preset object distances: obtaining the inverse filter corresponding to the object distance for the first camera according to the image acquired by the first camera; and obtaining the inverse filter corresponding to the object distance for the second camera according to the image acquired by the second camera. The processor obtains the transformation matrix between the first camera and the second camera at each of the plurality of preset object distances by:

6. The microscopic imaging apparatus of claim 1, wherein, for any one of the plurality of preset object distances: referring to the images of the preset standard object acquired by the first camera and the second camera at the object distance as a first standard image and a second standard image respectively; ​ filtering the first standard image by the inverse filter corresponding to the object distance through the first photographing component to obtain a third standard image; filtering the second standard image by the inverse filter corresponding to the object distance through the second photographing component to obtain a fourth standard image; wherein the third standard image and the fourth standard image contain an array of equally spaced dots, each dot corresponding to a super-fine pinhole on the standard object; center point coordinates of each dot in the third standard image and the fourth standard image are obtained respectively; a transformation matrix is obtained according to the center point coordinates of the corresponding dots in the third standard image and the fourth standard image, the transformation matrix being a rigid transformation matrix containing rotation and translation.

7. The microscopic imaging apparatus of claim 1, wherein, The processor obtains the offset loss amount of the arbitrary pixel point at the object distance according to the pixel values of the arbitrary pixel point in the obtained third image and the fifth image, including: For an arbitrary pixel point, the gray values of the pixel point on different channels in the third image and the fifth image are obtained respectively, and the norm is calculated according to the difference between the corresponding gray values of the pixel point on different channels, and the obtained norm is taken as the offset loss amount of the pixel point at the object distance.

8. The microscopic imaging apparatus of claim 1, wherein, The processor obtains the depth value of the arbitrary pixel point according to the offset loss amount of the pixel point at all object distances, including: for an arbitrary pixel point, the minimum value of the offset loss amount of the pixel point at all object distances is obtained, and the object distance corresponding to the minimum value is taken as the depth value of the pixel point.

9. The microscopic imaging apparatus of claim 1, wherein, The processor further includes: an ultra-deep-of-field image is obtained, wherein for an arbitrary pixel point in the ultra-deep-of-field image, the third image and the fifth image corresponding to the depth value of the pixel point as the object distance are obtained, the average value of the corresponding gray values of the pixel point on each channel in the obtained third image and the fifth image is calculated, and the obtained average value is taken as the gray value of the pixel point on the corresponding channel in the ultra-deep-of-field image.

10. A method of microscopic imaging with depth reconstruction effect, characterized in that, including: an image acquisition component is used to acquire images of a to-be-measured object, the image acquisition component at least including a first photographing component and a second photographing component, and the first photographing component and the second photographing component simultaneously acquire images to obtain a first image and a second image of the to-be-measured object; wherein the first photographing component and the second photographing component have corresponding inverse filters at a plurality of preset object distances, and at each object distance, the first photographing component and the second photographing component have a corresponding transformation matrix therebetween; For any one object distance: filtering the first image according to the inverse filter corresponding to the object distance of the first shooting component to obtain a third image corresponding to the object distance; filtering the second image according to the inverse filter corresponding to the object distance of the second shooting component to obtain a fourth image corresponding to the object distance; and performing rigid transformation on the fourth image according to the transformation matrix between the first shooting component and the second shooting component at the object distance to obtain a fifth image corresponding to the object distance; obtaining the offset loss amount of the arbitrary pixel point at the object distance according to the pixel values of the arbitrary pixel point in the third image and the fifth image. Obtaining the depth value of the arbitrary pixel point according to the offset loss amount of the arbitrary pixel point at all object distances, and the depth values of all pixel points constitute a depth map corresponding to the object to be measured.

11. The method of microscopic imaging according to claim 10, wherein, The first shooting component and the second shooting component have corresponding inverse filters at a plurality of preset object distances, including: Collecting images of a standard object at a plurality of preset object distances through the first shooting component and the second shooting component respectively. For any one object distance: obtaining the inverse filter corresponding to the object distance of the first shooting component according to the image collected by the first shooting component; and obtaining the inverse filter corresponding to the object distance of the second shooting component according to the image collected by the second shooting component.

12. The method of microscopic imaging of claim 11, wherein, The standard object includes an opaque plane plate with ultra-fine pinholes arranged at equal intervals.

13. The method of microscopic imaging of claim 10, wherein, The first shooting component and the second shooting component have corresponding transformation matrices at each object distance, including: For any one object distance: the images of the standard object collected by the first shooting component and the second shooting component at the object distance are respectively referred to as a first standard image and a second standard image; Filtering the first standard image through the inverse filter corresponding to the object distance of the first shooting component to obtain a third standard image; and filtering the second standard image through the inverse filter corresponding to the object distance of the second shooting component to obtain a fourth standard image; wherein the third standard image and the fourth standard image contain an array of circular dots arranged at equal intervals, and each circular dot corresponds to an ultra-fine pinhole on the standard object; Obtaining the center point coordinates of each circular dot in the third standard image and the fourth standard image respectively; and obtaining a transformation matrix according to the center point coordinates of the corresponding circular dots in the third standard image and the fourth standard image, the transformation matrix being a rigid transformation matrix containing rotation and translation.

14. The method of microscopic imaging of claim 10, wherein, The method for obtaining the offset loss amount of the arbitrary pixel point at the object distance according to the pixel values of the arbitrary pixel point in the third image and the fifth image includes: For any one pixel point, obtaining the gray values of the pixel point on different channels in the third image and the fifth image respectively, calculating the norm of the difference between the corresponding gray values of the pixel point on different channels, and taking the obtained norm as the offset loss amount of the pixel point at the object distance.

15. The method of microscopic imaging of claim 10, wherein, The depth value of the pixel point is obtained according to the offset loss amount corresponding to all object distances of the arbitrary pixel point, and the method comprises the following steps: for an arbitrary pixel point, obtaining the minimum value in the offset loss amount corresponding to all object distances of the pixel point, and taking the object distance corresponding to the minimum value as the depth value of the pixel point.

16. The method of microscopic imaging of claim 10, wherein, Further comprising: An ultra-deep-of-field image is obtained, wherein for an arbitrary pixel point in the ultra-deep-of-field image, a third image and a fifth image corresponding to the depth value of the pixel point as an object distance are obtained, the average value of the gray value corresponding to the pixel point on each channel in the obtained third image and fifth image is calculated, and the average value is taken as the gray value of the pixel point on the corresponding channel in the ultra-deep-of-field image.

17. A computer-readable storage medium, characterized in that, The medium stores a computer program, and the computer program can be executed by the processor to implement the method in any one of claims 10-16.

Images