Microscopic imaging device and method with depth reconstruction effect

Abstract

The invention discloses a microscopic imaging device and method with a depth reconstruction effect, and the method comprises the steps: carrying out the image collection of a to-be-detected object through an image collection part, and obtaining a first image and a second image of the to-be-detected object; according to inverse filters corresponding to a plurality of preset object distances of the first shooting component and the second shooting component and a transformation matrix between the first shooting component and the second shooting component under each object distance, obtaining an offset loss amount of any pixel point under the corresponding object distance; and the depth value of the pixel point is obtained according to the offset loss amount corresponding to all the object distances of any pixel point, so that depth reconstruction is completed, and an ultra-depth-of-field image and a depth map are obtained. By adding the specially designed phase mask sheet, the optical super depth-of-field effect is realized; multiple times of shooting or use of a displacement platform is not needed in the depth reconstruction process, the ultra-depth-of-field image and the depth image can be obtained only through one-time shooting of the double cameras, and the method is simpler, more efficient and easy to implement.

Description

Technical Field

[0001] The present invention relates to the field of microscopic three-dimensional imaging, and in particular to a microscopic imaging device and method with a depth reconstruction effect. Background Art

[0002] With the development of mold design, precision machining, product inspection, industrial measurement and other fields, traditional two-dimensional image sensor technology is increasingly unable to meet the needs of production scenarios, and the demand for three-dimensional measurement technology is growing rapidly. Compared with two-dimensional image sensor technology, three-dimensional measurement technology has an additional dimension of depth information, and because depth itself is not very sensitive to factors such as posture and lighting, three-dimensional measurement technology has greater advantages in object recognition and measurement tasks.

[0003] In the field of microscopic three-dimensional imaging, due to the extremely small depth of field of the objective lens, it is often necessary to use motion structures and image synthesis to obtain the super depth of field effect and the depth information of the object surface. The current mainstream microscope three-dimensional reconstruction solutions are interferometers or multi-focal plane focusing changes. During reconstruction, a displacement platform is required to move along the depth of field direction, and the target image is synthesized through multiple shots at different positions; however, the use of the displacement platform and time-divided sampling will introduce system disturbance errors, resulting in errors in the obtained measurement results. Summary of the invention

[0004] The main technical problem solved by the present invention is how to achieve depth reconstruction without using a displacement platform and multiple sampling.

[0005] According to the first aspect, an embodiment provides a microscopic imaging device with a depth reconstruction effect, including a microscope objective lens, a tube lens, a light source, a lifting platform, and a plurality of light splitting devices, and further comprising: A phase mask, wherein the phase mask is located between a microscope objective and a light splitting device; An image acquisition component, the image acquisition component at least comprising a first shooting component and a second shooting component; the first shooting component and the second shooting component have corresponding inverse filters at multiple preset object distances, and at each object distance, there is also a corresponding transformation matrix between the first shooting component and the second shooting component; and a processor for: Control the lifting platform to move in the vertical direction according to the preset moving step length; Acquire inverse filters corresponding to the first photographing component and the second photographing component at a plurality of preset object distances, and a transformation matrix between the first photographing component and the second photographing component at each object distance; Controlling the first photographing component and the second photographing component to simultaneously capture images of the object to be measured to obtain a first image and a second image; For any object distance: filtering the first image according to the inverse filter corresponding to the object distance by the first photographing component to obtain a third image corresponding to the object distance; filtering the second image according to the inverse filter corresponding to the object distance by the second photographing component to obtain a fourth image corresponding to the object distance; and performing a rigid body transformation on the fourth image according to the transformation matrix between the first photographing component and the second photographing component at the object distance to obtain a fifth image corresponding to the object distance; obtaining the offset loss amount corresponding to the arbitrary pixel point at the object distance according to the pixel value corresponding to the arbitrary pixel point in the obtained third image and the fifth image; The depth value of any pixel is obtained according to the offset loss corresponding to all object distances, and the depth values ​​of all pixels constitute a depth map corresponding to the object to be measured, thereby completing depth reconstruction.

[0006] According to the second aspect, an embodiment provides a microscopic imaging method with a depth reconstruction effect, comprising: The image acquisition component is used to acquire an image of the object to be measured, and the image acquisition component at least includes a first shooting component and a second shooting component, and the first shooting component and the second shooting component simultaneously acquire images to obtain a first image and a second image of the object to be measured; wherein the first shooting component and the second shooting component have corresponding inverse filters at multiple preset object distances, and at each object distance, there is also a corresponding transformation matrix between the first shooting component and the second shooting component; For any object distance: filtering the first image according to the inverse filter corresponding to the object distance by the first photographing component to obtain a third image corresponding to the object distance; filtering the second image according to the inverse filter corresponding to the object distance by the second photographing component to obtain a fourth image corresponding to the object distance; and performing a rigid body transformation on the fourth image according to the transformation matrix between the first photographing component and the second photographing component at the object distance to obtain a fifth image corresponding to the object distance; obtaining the offset loss amount corresponding to the arbitrary pixel point at the object distance according to the pixel value corresponding to the arbitrary pixel point in the obtained third image and the fifth image; A depth value of any pixel is obtained according to the offset loss corresponding to the arbitrary pixel at all object distances, and the depth values ​​of all the pixels constitute a depth map corresponding to the object to be measured.

[0007] According to a microscopic imaging device and method with a depth reconstruction effect of the above-mentioned embodiment, by adding a specially designed phase mask to the optical system, the point spread function after imaging maintains the same shape within a certain range, thereby greatly expanding the depth of field of the optical system, achieving an optical super-depth of field effect, and improving the phenomenon of extremely small depth of field of ordinary microscopes due to the expansion of the aperture; in the depth reconstruction process, firstly, based on the inverse filters corresponding to different shooting components obtained in the calibration process and the transformation matrix between different shooting components, an implicit correspondence between different object distances and the offset difference between the point spread functions corresponding to different shooting components is established, thereby recording the change in the corresponding offset of the point spread functions of different shooting components caused by the change in the object distance; and then in actual use, According to the corresponding offset loss amount of the object to be measured at different object distances, the real depth of the surface of the object to be measured is inferred, so as to complete the depth reconstruction and obtain the corresponding super depth of field image; since the depth reconstruction process no longer requires multiple shots or the use of a displacement platform, a super depth of field image and a depth image can be obtained through only one shot of a dual camera, which is simpler and more efficient; and the image is filtered by an inverse filter corresponding to different object distances and restored to the original image, which can eliminate the triangular smear generated by any point passing through the phase mask, thereby improving the problem of imaging artifacts existing in the reconstruction of the traditional phase mask; in addition, since the present application does not require special auxiliary lighting, and other optical devices except the phase mask have standard products, the present application is also easier to implement. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A processing flow chart of a processor in a microscopic imaging device with a depth reconstruction effect; Figure 2 A method flow chart of a microscopic imaging method with a deep reconstruction effect; Figure 3 It is a structural schematic diagram of a microscopic imaging device with a depth reconstruction effect; Figure 4 It is a schematic diagram of the shape of the PSF function after inserting the phase mask; Figure 5 It is a schematic diagram of the change of the offset corresponding to different shooting components at different object distances; Figure 6 is an image corresponding to a partial area of ​​the standard object in the first standard image; Figure 7 This is a schematic diagram of the depth reconstruction results; Figure 8 The diffusion pattern formed by a pinhole of the standard.

[0009] Figure numerals: 11 - microscope objective lens; 12 - tube lens; 13 - light source; 14 - lifting platform; 15 - spectrometer; 16 - phase mask; 17 - image acquisition component. DETAILED DESCRIPTION

[0010] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0011] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0012] The serial numbers assigned to the components herein, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0013] Phase mask belongs to wavefront coding technology. It extends the depth of field of the optical system by modulating the pupil phase distribution function. It has the effect of greatly expanding the depth of field of the optical system. Although its point spread function has a certain offset at different object distances, its shape, size, and blur distribution are almost completely consistent. The embodiment of the present invention aims to make the shapes of the modulation transfer function and the point spread function of the optical system insensitive to the object distance through specially designed optical elements (i.e., phase masks), while greatly expanding the depth of field of the optical system. Based on the imaging characteristics of the microscopic imaging system, the dual cameras can obtain the depth information of the object surface after one shooting, thereby obtaining results with super depth of field effect and depth reconstruction effect without using a displacement platform and multiple sampling.

[0014] Some embodiments provide a microscopic imaging device with a depth reconstruction effect, and the schematic diagram thereof is shown in FIG. Figure 3 , including a microscope objective lens 11, a tube lens 12, a light source 13, a lifting platform 14 and a plurality of spectroscopic devices 15, and also including a phase mask 16, an image acquisition component 17 and a processor; in the microscopic imaging device of this embodiment, the light generated by the incoherent light source 13 irradiates the object to be measured, passes through the microscope objective lens 11, is modulated by the phase mask 16, and then passes through the spectroscopic device 15, and is imaged by two image acquisition components 17 of the same model with different image distances; the following is a specific description: The phase mask 16 in this embodiment is a bicubic phase mask, which is located between the microscope objective 11 and a spectrometer 15. By placing a phase mask 16 with a special surface shape at the pupil of the microscope objective 11, the light phase distribution function at the pupil is modulated, and the shapes of the modulation transfer function and the point spread function of the optical system are made insensitive to the object distance. At the same time, the wavefront coding technology is used to greatly expand the depth of field of the optical system. The bicubic constraint satisfied by the phase distribution function at the pupil after optical optimization is: φ ( u , v )=2 πα ( u 3 + v 3 ) in, φ ( u , v ) represents the phase distribution function, u , v is the position coordinate on the pupil, α To control the gain factor of the surface bending strength, it is optimized by optical software (such as Zemax); The image acquisition component 17 in this embodiment includes at least a first shooting component and a second shooting component; wherein the first shooting component and the second shooting component have corresponding inverse filters at multiple preset object distances, and at each object distance, there is also a corresponding transformation matrix between the first shooting component and the second shooting component, which is used to convert the images simultaneously captured by the first shooting component and the second shooting component into the same coordinate system; It should be noted that the optical system after adding the phase mask will have the following characteristics: on the one hand, the modulation transfer function (MTF function) of the optical system imaging will not drop to 0, so deconvolution can be used to perform the inverse transformation of image degradation, that is, to restore the output image of the optical system to the original input image; on the other hand, the shape of the light intensity distribution corresponding to the point spread function (PSF) of the optical system will not change within a larger object distance range (super depth of field range), and its shape is a triangular diffraction stripe with an offset symmetry center; and within this super depth of field range, although the shape of the light intensity distribution will not change, the position of the imaging spot of a single point light source after passing through the phase mask will be displaced, that is, the center position of the spot will be offset, and the offset will change with the change of the working distance (or object distance), where the function formed by the offset and the working distance is δ ( w ) is approximately a parabola, and the corresponding offset changes of different shooting components at different object distances are shown in Figure 5 As shown, δ Indicates the offset, w Indicates working distance / object distance; According to the imaging characteristics of the bicubic phase mask, if the two photographing components are fixed at different image distances to maintain a fixed back focal length difference, the function Δ formed by the difference between the corresponding offsets of the two photographing components at the same object distance and the corresponding working distance is δ ( w ) is a monotone continuous function, such as Figure 5 As shown, at this time, if the corresponding relationship between the offset difference and the working distance can be obtained in advance, the corresponding working distance can be restored according to the calculated offset difference in real-time shooting; In this embodiment, the process of obtaining the corresponding relationship between the offset difference and the working distance is called a calibration process, in which the inverse filters corresponding to the first imaging component and the second imaging component at multiple preset object distances and the transformation matrix between the first imaging component and the second imaging component at each object distance are obtained; In this calibration process, the processor controls the lifting platform 14 to move in the vertical direction according to the preset moving step length, and controls the image acquisition component 17 to acquire images of the preset standard object after each movement of the lifting platform 14, so as to obtain images corresponding to the standard object at multiple preset object distances, wherein the standard object is placed on the lifting platform, which is an opaque flat plate with ultra-fine pinholes distributed at equal intervals, and a backlight surface light source is set on the opaque flat plate for auxiliary lighting to illuminate the ultra-fine pinholes on the opaque flat plate; then the specific calibration process of this embodiment is: First, an opaque flat plate with ultra-fine pinholes distributed at equal intervals is used as a pre-set standard object, and it is placed on a lifting platform 14 equipped with a stepping motor, and a backlight source is set to illuminate the pinholes on the opaque flat plate, wherein the lifting direction of the lifting platform 14 is perpendicular to the optical axis of the objective lens; then starting from the position where the nominal working distance is 0, the lifting platform 14 is controlled to move in the vertical direction according to the preset moving step length. For example, the preset moving step length can be set to 1 μm , corresponding to each upward movement of the lifting platform 14 μm After each movement of the lifting platform 14, the image acquisition component 17 acquires images of the preset standard object until it exceeds the range of the working distance, and records the two images acquired by the first shooting component and the second shooting component of the standard object at the same time after each movement and the corresponding lifting platform position, thereby obtaining two images corresponding to the standard object at different object distances, wherein, for any object distance: at the object distance, the two images acquired after the first shooting component and the second shooting component acquire images of the standard object are respectively referred to as the first standard image and the second standard image; In this embodiment, after the pinhole on the opaque plane plate is illuminated by the backlight source, the light originally concentrated at one point will form an extended light spot on the image plane, such as Figure 4 As shown in FIG. 1 , the light intensity distribution of a point after passing through the phase mask is a triangular smear. At this time, the light intensity distribution of the light spot is the PSF function. In this embodiment, the corresponding PSF functions are calculated by optical software (such as Zemax) according to the diffusion patterns formed by the pinhole on the opaque plane plate in the first standard image and the second standard image, respectively. The image corresponding to the partial area of ​​the standard object in the first standard image is as shown in FIG. Figure 6 As shown, Figure 8 for Figure 6 A partial enlargement of the standard, which shows the diffusion pattern formed by a pinhole of the standard; Since the modulation transfer function of the system has no zero point and the shape of the PSF function is insensitive to the position of the field of view, its inverse transform can be solved by Fourier transform, and then the inverse filter corresponding to the obtained PSF function can be constructed by Wiener filtering, that is, for any object distance, the inverse filter corresponding to the first shooting component at the object distance is obtained according to the PSF function corresponding to the first standard image; the inverse filter corresponding to the second shooting component at the object distance is obtained according to the PSF function corresponding to the second standard image; thus, the inverse filters corresponding to the first shooting component and the second shooting component at the object distance are obtained; Then, the first standard image and the second standard image are filtered by the obtained inverse filter pair, so as to restore the degraded image containing point spread function noise to a clear original image; wherein, the first standard image is filtered by the first shooting component using the inverse filter corresponding to the object distance to obtain the third standard image; the second standard image is filtered by the second shooting component using the inverse filter corresponding to the object distance to obtain the fourth standard image; thereby, the first standard image and the second standard image are restored to the original image containing the ultrafine pinhole array on the standard object, that is, at this time, the third standard image and the fourth standard image contain an array of dots distributed at equal intervals, and each dot corresponds to an ultrafine pinhole on the standard object; The center point coordinates of each dot in the third standard image and the fourth standard image are obtained respectively, such as by using a dot extraction algorithm to obtain the center point coordinates of each dot in the third standard image and the fourth standard image; since each ultrafine pinhole on the standard object has a corresponding dot in the third standard image and the fourth standard image, the linear least squares method can be used to obtain the transformation matrix according to the center point coordinates of the corresponding dots in the third standard image and the fourth standard image, that is: in, cam 1 represents a first shooting component; cam 2 represents a second shooting component; For the object distance w , a matrix composed of the center point coordinates of all the dots in the third standard image corresponding to the first shooting component; For the object distance w , a matrix composed of the center point coordinates of all the dots in the fourth standard image corresponding to the second shooting component; T w The object distance is w is a transformation matrix between the first shooting component and the second shooting component, the transformation matrix being a rigid body transformation matrix including rotation and translation; The coordinates of the center points of the corresponding dots in the images captured by the dual cameras can also be used to calculate the change in the offset value caused by the object distance transformation. Therefore, when the transformation matrix converts the images corresponding to the first shooting component and the second shooting component to the same coordinate system, it will also implicitly include the offset difference between different shooting components, that is, the transformation matrix corresponding to different object distances can implicitly establish the corresponding relationship between different object distances and the corresponding offset difference.

[0015] In this embodiment, the processor is also used to perform depth reconstruction to obtain a depth map and a super depth of field image, wherein the processing flow of the processor performing depth reconstruction is as follows: Figure 1 As shown, including: Step S100: Obtain inverse filters corresponding to the first photographing component and the second photographing component at multiple preset object distances, and a transformation matrix between the first photographing component and the second photographing component at each object distance.

[0016] The inverse filters corresponding to the first shooting component and the second shooting component at multiple preset object distances obtained in the calibration process are obtained, as well as the transformation matrix between the first shooting component and the second shooting component at each object distance.

[0017] Step S110: Control the first shooting component and the second shooting component to simultaneously capture images of the object to be measured, to obtain a first image and a second image.

[0018] The object to be measured is placed at a distance of m On the plane, m Unknown, that is, in this embodiment, it is only necessary to ensure that the object to be measured is fixed within the field of view of the shooting component, and it is not necessary to know the specific object distance where the object to be measured is placed.

[0019] Step S120: Obtain the offset loss amount corresponding to any pixel point at each object distance according to the obtained first image and second image.

[0020] Since the function Δ formed by the offset difference and the corresponding working distance δ ( w ) is a monotone continuous function, then any pixel point within the imaging range of the shooting component p , there should be a unique object distance z satisfy: in, I cam1 is an image captured by the first shooting component, corresponding to the first image in this embodiment; I cam2 is an image captured by the second shooting component, corresponding to the second image in this embodiment; Indicates that the object distance is z when an inverse filter corresponding to the first shooting component; Indicates that the object distance is z when an inverse filter corresponding to the second shooting component; T z Indicates that the object distance is z , the transformation matrix between the first shooting component and the second shooting component; “|| ||” represents the L2 norm; “·” represents the dot product; “*” represents the convolution operation; at this time, z That is the pixel pCorresponding depth value; This embodiment traverses a plurality of pre-stored preset object distances, and performs image transformation based on the inverse filters and transformation matrices corresponding to different object distances. At this time, the difference between the image transformation results is the difference caused by the offset between the corresponding point spread functions of different shooting components. In this embodiment, the degree of difference between the corresponding pixels in the image transformation result is called the offset loss of the pixel point, and then the depth value of the pixel point is obtained by minimizing the degree of difference corresponding to the pixel point, that is, the object distance that minimizes the offset loss of the pixel point is used as the depth value of the pixel point; For any object distance: the first image is filtered according to the inverse filter corresponding to the object distance of the first shooting component to obtain a third image corresponding to the object distance; the second image is filtered according to the inverse filter corresponding to the object distance of the second shooting component to obtain a fourth image corresponding to the object distance; Since the transformation matrix in this embodiment converts the filtered image corresponding to the second imaging component into the coordinate system of the filtered image corresponding to the first imaging component, the fourth image is rigidly transformed according to the transformation matrix between the first imaging component and the second imaging component at the object distance to obtain the fifth image corresponding to the object distance. At this time, the fifth image and the third image are in the same coordinate system, and the difference between the two is the difference caused by the offset between the corresponding point spread functions of different shooting components. According to the pixel value corresponding to any pixel point in the obtained third image and the fifth image, the difference degree between the third image and the fifth image is evaluated, and the obtained difference degree is the offset loss amount corresponding to the pixel point at the object distance; For example, for any pixel point, the grayscale values ​​of the pixel point in different channels in the third image and the fifth image are obtained respectively, and the norm is calculated according to the difference between the corresponding grayscale values ​​of the pixel point in different channels, such as calculating the L2 norm, and the obtained norm is used as the offset loss corresponding to the pixel point at the object distance.

[0021] Step S130: obtaining a depth value of any pixel according to the offset loss corresponding to all object distances. The depth values ​​of all pixels constitute a depth map corresponding to the object to be measured, thereby completing depth reconstruction.

[0022] For any pixel point, the minimum value of the offset loss corresponding to the pixel point under all preset object distances is obtained, and the object distance corresponding to the obtained minimum value is used as the depth value of the pixel point. The depth values ​​of all pixels constitute the depth map corresponding to the object to be measured, thereby completing the depth reconstruction. The result obtained after the depth reconstruction of a block object in this embodiment is as follows Figure 7 shown.

[0023] In addition, this embodiment can also obtain a super depth of field image, wherein, for any pixel point in the super depth of field image, a third image and a fifth image corresponding to the object distance are obtained when the depth value of the pixel point is used as the object distance. In the obtained third image and fifth image, the average value of the grayscale value corresponding to the pixel point in each channel is calculated, and the obtained average value is used as the grayscale value of the pixel point in the corresponding channel in the super depth of field image.

[0024] In this embodiment, a specially designed phase mask is added to the optical system, so that the point spread function after imaging maintains the same shape within a certain range, greatly expanding the depth of field of the optical system, achieving an optical super-depth of field effect, and improving the phenomenon of extremely small depth of field of ordinary microscopes due to the enlarged aperture; in the depth reconstruction process, firstly, based on the inverse filters corresponding to different shooting components obtained in the calibration process and the transformation matrix between different shooting components, the corresponding relationship between different object distances and the offset difference between the point spread functions corresponding to different shooting components is implicitly established, thereby recording the change of the corresponding offset of the point spread function of different shooting components caused by the change of object distance; then, in actual use, according to the corresponding offset loss of the object to be measured at different object distances, the object to be measured is inversely deduced. The real depth of the object surface is obtained to complete the depth reconstruction and obtain the corresponding super depth of field image. Since the present embodiment does not need to shoot multiple times or use a displacement platform in the process of depth reconstruction, the super depth of field image and the depth image can be obtained by only one shooting of the dual cameras, so it is simpler and more efficient. By filtering the image with the inverse filter corresponding to different object distances and restoring it to the original image, the triangular smear generated by any point passing through the phase mask can be eliminated, that is, by considering the imaging offset corresponding to all object distances, the imaging artifact problem existing in the reconstruction of the traditional phase mask can be improved. In addition, the present embodiment does not require special auxiliary lighting, and other optical devices except the phase mask are standard products, so the whole scheme is also easier to implement.

[0025] Please refer to Figure 2 In some embodiments, a microscopic imaging method with a depth reconstruction effect is provided, specifically comprising: Step S200: performing image acquisition on the object to be measured by an image acquisition component to obtain a first image and a second image of the object to be measured.

[0026] In this embodiment, the image acquisition component includes at least a first shooting component and a second shooting component, and the first shooting component and the second shooting component perform image acquisition simultaneously to obtain a first image and a second image of the object to be measured; wherein the first shooting component and the second shooting component have their corresponding inverse filters at multiple preset object distances, and at each object distance, there is also a corresponding transformation matrix between the first shooting component and the second shooting component, and the transformation matrix is ​​used to convert the images corresponding to the first shooting component and the second shooting component into the same coordinate system.

[0027] Step S210: Obtaining the offset loss corresponding to any pixel point at each object distance according to the first image and the second image of the object to be measured.

[0028] For any object distance: the first image is filtered according to the inverse filter corresponding to the first shooting component at the object distance to obtain the third image corresponding to the object distance; the second image is filtered according to the inverse filter corresponding to the second shooting component at the object distance to obtain the fourth image corresponding to the object distance; and the fourth image is rigidly transformed according to the transformation matrix between the first shooting component and the second shooting component at the object distance to obtain the fifth image corresponding to the object distance; the offset loss amount corresponding to any pixel point at the object distance is obtained according to the pixel value corresponding to any pixel point in the obtained third image and fifth image.

[0029] Step S220: obtaining a depth value of any pixel point according to the offset loss amount corresponding to the pixel point at all object distances, and the depth values ​​of all the pixels points constitute a depth map corresponding to the object to be measured.

[0030] It should be noted that the method steps in this embodiment correspond to the processing steps of the processor in the above embodiment, and the specific implementation methods have been specifically described in the above embodiment and will not be repeated here.

[0031] In this embodiment, the image acquisition component firstly acquires the image of the object to be measured placed at an unknown position to obtain the first image and the second image; then, the corresponding images are filtered according to the inverse filters corresponding to the different shooting components at different object distances to eliminate the point spread function noise contained in the images, so that the images are restored to clear original images, thereby obtaining the third image and the fourth image corresponding to the different object distances; then, the image transformation is performed according to the transformation matrix of the different shooting components at the corresponding object distances to obtain the fifth image corresponding to the different object distances, so that the images corresponding to the different shooting components are converted to the same coordinate system, that is, the fourth image is converted to the coordinate system corresponding to the third image. At this time, the difference between the fifth image and the third image is the difference caused by the offset of the point spread function, so the offset loss corresponding to the pixel point at different object distances is obtained according to the difference between the third image and the fifth image of the same pixel point, and the depth map corresponding to the pixel point is obtained by minimizing the offset loss, so as to complete the depth reconstruction and obtain the corresponding super depth of field image. Since this embodiment does not need to shoot multiple times or use a displacement platform, the super depth of field image and the depth image can be obtained by only one shooting of the dual cameras, which is simpler and more efficient.

[0032] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0033] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art, according to the concept of the present invention, some simple deductions, modifications or substitutions can be made.

Claims

1. A microscopic imaging device with a depth reconstruction effect, comprising a microscope objective lens, a tube lens, a light source, a lifting platform and a plurality of light splitting devices, characterized in that: Also includes: A phase mask, wherein the phase mask is located between a microscope objective and a light splitting device; An image acquisition component, the image acquisition component at least comprising a first shooting component and a second shooting component; the first shooting component and the second shooting component have corresponding inverse filters at multiple preset object distances, and at each object distance, there is also a corresponding transformation matrix between the first shooting component and the second shooting component; and a processor for: Control the lifting platform to move in the vertical direction according to the preset moving step length; Acquire inverse filters corresponding to the first photographing component and the second photographing component at a plurality of preset object distances, and a transformation matrix between the first photographing component and the second photographing component at each object distance; Controlling the first photographing component and the second photographing component to simultaneously capture images of the object to be measured to obtain a first image and a second image; For any object distance: filtering the first image according to the inverse filter corresponding to the object distance by the first photographing component to obtain a third image corresponding to the object distance; filtering the second image according to the inverse filter corresponding to the object distance by the second photographing component to obtain a fourth image corresponding to the object distance; and performing a rigid body transformation on the fourth image according to the transformation matrix between the first photographing component and the second photographing component at the object distance to obtain a fifth image corresponding to the object distance; obtaining the offset loss amount corresponding to the arbitrary pixel point at the object distance according to the pixel value corresponding to the arbitrary pixel point in the obtained third image and the fifth image; The depth value of any pixel is obtained according to the offset loss corresponding to all object distances, and the depth values ​​of all pixels constitute a depth map corresponding to the object to be measured, thereby completing depth reconstruction.

2. The microscopic imaging device according to claim 1, characterized in that: The phase mask comprises a bicubic phase mask.

3. The microscopic imaging device according to claim 1, characterized in that: The processor is also used to control the image acquisition component to acquire images of a preset standard object after each movement of the lifting platform, so as to obtain images corresponding to the standard object at multiple preset object distances, wherein the standard object is placed on the lifting platform.

4. The microscopic imaging device according to claim 3, characterized in that: The pre-set standard object comprises: an opaque flat plate with ultra-fine pinholes distributed at equal intervals, and the opaque flat plate is provided with a backlight surface light source for auxiliary lighting.

5. The microscopic imaging device according to claim 1, characterized in that: The processor obtains the inverse filters corresponding to the first shooting component and the second shooting component at multiple preset object distances, including: The first photographing component and the second photographing component respectively collect images of a preset standard object at a plurality of preset object distances; For any object distance: obtain the inverse filter corresponding to the first photographing component at the object distance based on the image captured by the first photographing component; obtain the inverse filter corresponding to the second photographing component at the object distance based on the image captured by the second photographing component.

6. The microscopic imaging device according to claim 1, characterized in that: The processor obtains the transformation matrix between the first shooting component and the second shooting component at each object distance, including: For any object distance: at the object distance, the images captured by the first photographing component and the second photographing component on the preset standard object are respectively referred to as the first standard image and the second standard image; The first standard image is filtered by the first shooting component using an inverse filter corresponding to the object distance to obtain a third standard image; the second standard image is filtered by the second shooting component using an inverse filter corresponding to the object distance to obtain a fourth standard image; wherein the third standard image and the fourth standard image contain arrays of dots distributed at equal intervals, and each dot corresponds to an ultrafine pinhole on the standard object; The center point coordinates of each dot in the third standard image and the fourth standard image are respectively obtained; a transformation matrix is ​​obtained according to the center point coordinates of corresponding dots in the third standard image and the fourth standard image, and the transformation matrix is ​​a rigid body transformation matrix including rotation and translation.

7. The microscopic imaging device according to claim 1, characterized in that: The processor obtains the offset loss amount corresponding to any pixel point at the object distance according to the pixel value corresponding to any pixel point in the obtained third image and the fifth image, including: For any pixel point, the grayscale value of the pixel point in different channels in the third image and the fifth image is obtained respectively, and the norm is calculated according to the difference between the corresponding grayscale values ​​of the pixel point in different channels, and the obtained norm is used as the offset loss corresponding to the pixel point at the object distance.

8. The microscopic imaging device according to claim 1, characterized in that: The processor obtains the depth value of any pixel point based on the offset loss amount corresponding to any pixel point at all object distances, including: for any pixel point, obtaining the minimum value of the offset loss amount corresponding to the pixel point at all object distances, and using the object distance corresponding to the obtained minimum value as the depth value of the pixel point.

9. The microscopic imaging device according to claim 1, characterized in that: The processor further comprises: A super depth of field image is obtained, wherein, for any pixel point in the super depth of field image, a third image and a fifth image corresponding to the object distance are obtained when the depth value of the pixel point is used as the object distance, and an average value of the grayscale value corresponding to the pixel point in each channel in the obtained third image and fifth image is calculated, and the obtained average value is used as the grayscale value of the pixel point in the corresponding channel in the super depth of field image.

10. A microscopic imaging method with depth reconstruction effect, characterized in that: include: The image acquisition component is used to acquire an image of the object to be measured, and the image acquisition component at least includes a first shooting component and a second shooting component, and the first shooting component and the second shooting component simultaneously acquire images to obtain a first image and a second image of the object to be measured; wherein the first shooting component and the second shooting component have corresponding inverse filters at multiple preset object distances, and at each object distance, there is also a corresponding transformation matrix between the first shooting component and the second shooting component; For any object distance: filtering the first image according to the inverse filter corresponding to the object distance by the first photographing component to obtain a third image corresponding to the object distance; filtering the second image according to the inverse filter corresponding to the object distance by the second photographing component to obtain a fourth image corresponding to the object distance; and performing a rigid body transformation on the fourth image according to the transformation matrix between the first photographing component and the second photographing component at the object distance to obtain a fifth image corresponding to the object distance; obtaining the offset loss amount corresponding to the arbitrary pixel point at the object distance according to the pixel value corresponding to the arbitrary pixel point in the obtained third image and the fifth image; A depth value of any pixel is obtained according to the offset loss corresponding to all object distances, and the depth values ​​of all pixels constitute a depth map corresponding to the object to be measured.

11. The microscopic imaging method according to claim 10, characterized in that: The first photographing component and the second photographing component each have a corresponding inverse filter at a plurality of preset object distances, including: The first photographing component and the second photographing component respectively collect images of a preset standard object at a plurality of preset object distances; For any object distance: obtain the inverse filter corresponding to the first photographing component at the object distance based on the image captured by the first photographing component; obtain the inverse filter corresponding to the second photographing component at the object distance based on the image captured by the second photographing component.

12. The microscopic imaging method according to claim 11, characterized in that: The pre-set standard object includes: an opaque plane plate with ultra-fine pinholes distributed at equal intervals.

13. The microscopic imaging method according to claim 10, characterized in that: At each object distance, there is also a corresponding transformation matrix between the first shooting component and the second shooting component, including: For any object distance: at the object distance, the images captured by the first photographing component and the second photographing component on the preset standard object are respectively referred to as the first standard image and the second standard image; The first standard image is filtered by the first shooting component using an inverse filter corresponding to the object distance to obtain a third standard image; the second standard image is filtered by the second shooting component using an inverse filter corresponding to the object distance to obtain a fourth standard image; wherein the third standard image and the fourth standard image contain arrays of dots distributed at equal intervals, and each dot corresponds to an ultrafine pinhole on the standard object; The center point coordinates of each dot in the third standard image and the fourth standard image are respectively obtained; a transformation matrix is ​​obtained according to the center point coordinates of corresponding dots in the third standard image and the fourth standard image, and the transformation matrix is ​​a rigid body transformation matrix including rotation and translation.

14. The microscopic imaging method according to claim 10, characterized in that: The step of obtaining the offset loss amount corresponding to any pixel point at the object distance according to the pixel value corresponding to any pixel point in the obtained third image and the fifth image comprises: For any pixel point, the grayscale value of the pixel point in different channels in the third image and the fifth image is obtained respectively, and the norm is calculated according to the difference between the corresponding grayscale values ​​of the pixel point in different channels, and the obtained norm is used as the offset loss corresponding to the pixel point at the object distance.

15. The microscopic imaging method according to claim 10, characterized in that: The method of obtaining the depth value of any pixel point according to the offset loss amount corresponding to any pixel point at all object distances includes: for any pixel point, obtaining the minimum value of the offset loss amount corresponding to the pixel point at all object distances, and using the object distance corresponding to the obtained minimum value as the depth value of the pixel point.

16. The microscopic imaging method according to claim 10, characterized in that: Also includes: A super depth of field image is obtained, wherein, for any pixel point in the super depth of field image, a third image and a fifth image corresponding to the object distance are obtained when the depth value of the pixel point is used as the object distance, and an average value of the grayscale value corresponding to the pixel point in each channel in the obtained third image and fifth image is calculated, and the obtained average value is used as the grayscale value of the pixel point in the corresponding channel in the super depth of field image.

17. A computer-readable storage medium, characterized in that: A computer program is stored on the medium, and the computer program can be executed by a processor to implement the method according to any one of claims 10 to 16.

Images