A binocular high-resolution digital image real-time stereo vision microscope system

Abstract

This invention provides a binocular high-resolution digital imaging real-time stereoscopic vision microscope system, comprising a binocular microscope body (consisting of a binocular zoom parfocal optical path and a microscope magnifying objective camera), a binocular imaging subsystem, and a stereoscopic image display. The binocular imaging subsystem includes: an image projection correction calculation module, which acquires original left and right image data through the binocular microscope body, performs manufacturing parameter consistency correction and image projection correction on the original left and right image data, and obtains a projection distortion-corrected image; and a stereoscopic image output calculation module, which, based on the distortion-corrected image, performs photographic spatial projection position and attitude correction to obtain a large depth-of-field digital stereoscopic image, and transmits the ultra-depth-of-field digital stereoscopic image to the stereoscopic image display for real-time display. This invention can obtain nanometer-level sampling resolution digital images, eliminate image projection distortion caused by manufacturing and installation errors of the microscope body, and achieve high-resolution, high-magnification microscopic stereoscopic image observation.

Description

Technical Field

[0001] This invention relates to the field of stereoscopic microscopy technology, and in particular to a binocular high-resolution digital imaging real-time stereoscopic microscopy system. Background Technology

[0002] In recent years, optical stereomicroscopes have been able to observe the spatial state of tiny objects in real time. However, high-magnification optical microscopes are limited by the principles of optical diffraction and physical depth of field, with magnifications of less than 300x, allowing only the observation of micron-level resolution optical microscopic stereoscopic images. Other types of high-magnification microscopes and confocal microscopes cannot observe stereoscopic images, only 2D planar microscopic images. Currently, high-magnification microscopes still struggle to achieve real-time high-magnification and high-resolution, large-depth-of-field stereoscopic visual image observation and detection. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a binocular high-resolution digital imaging real-time stereoscopic vision microscope system that can obtain nanometer-resolution sampled digital images, analyze and calculate to eliminate image projection distortion caused by errors in the manufacturing and installation of the microscope body, realize high-resolution, high-magnification, and large-depth-of-field microscopic stereoscopic image observation, long-term storage of microscopic stereoscopic image data, and real-time transmission and viewing of microscopic stereoscopic images via network.

[0004] To achieve the above objectives, the present invention provides the following solution: a binocular high-resolution digital imaging real-time stereo vision microscope system, comprising a binocular microscope body, a binocular imaging subsystem connected to the binocular microscope body, and a stereo image display connected to the binocular imaging subsystem; the binocular imaging subsystem includes: The image projection correction calculation module is used to acquire the original left and right image data through the binocular microscope, perform consistency correction and projection correction on the original left and right image data, and obtain the projection distortion corrected image. The stereoscopic image output calculation module is used to correct the rearranged image based on the projection distortion correction image and using spatial projection position and attitude correction parameters to complete the correction of manufacturing and installation consistency errors and image projection errors, output a large depth-of-field digital stereoscopic image, transmit the large depth-of-field digital stereoscopic image to the stereoscopic image display for real-time display and storage, and transmit it to the network platform in real time. The image projection correction calculation module and the stereoscopic image output calculation module are interconnected.

[0005] Optionally, the binocular microscope body is composed of a first zoom parfocal optical path and a second zoom parfocal optical path, which are symmetrically arranged.

[0006] Optionally, the first zoom parfocal optical path consists of a first zoom parfocal optical path objective lens and a first microscopic magnifying objective lens camera, wherein the first microscopic magnifying objective lens camera is installed and photographed perpendicularly to the optical image magnification imaging surface of the first zoom parfocal optical path objective lens; the second zoom parfocal optical path consists of a second zoom parfocal optical path objective lens and a second microscopic magnifying objective lens camera, wherein the second microscopic magnifying objective lens camera is installed and photographed perpendicularly to the optical image magnification imaging surface of the second zoom parfocal optical path objective lens.

[0007] Optionally, the first zoom parfocal optical path objective lens has the same distortion and the second zoom parfocal optical path objective lens has the same focal length and manufacturing parameters. The baseline of the mounting distance between the first zoom parfocal optical path objective lens and the second zoom parfocal optical path objective lens is set between 1 cm and 8 cm. The image overlap between the first zoom parfocal optical path objective lens and the second zoom parfocal optical path objective lens is set between 50% and 100%.

[0008] Optionally, the binocular microscope body is configured with a parallel light path mode and a cross-beam light path mode. In the cross-beam light path mode, the inward angle between the first zoom parfocal objective lens and the second zoom parfocal objective lens is set to between 0.5° and 15°.

[0009] Optionally, the image projection correction calculation module includes: The manufacturing parameter uniformity calculation unit is used to acquire original left and right image data using the binocular microscope body by employing a stacked stereo imaging method with multiple multi-distance focal planes, and to perform objective lens distortion calibration and uniformity correction and focal length calibration and uniformity correction based on the original left and right image data to obtain a binocular microscope body manufacturing parameter uniformity image. The objective lens projection center correction calculation unit is used to calculate the pixel positions of the central principal rays of the first zoom parfocal optical path objective lens and the second zoom parfocal optical path objective lens on the two sensors based on the binocular microscope body manufacturing parameter uniform image, and set the pixel positions as coordinate rotation points for photographic space attitude projection calculation, and perform image center projection correction according to the coordinate rotation points. Photographic projection correction calculation unit: Based on the deformation correction image of the binocular microscope body manufacturing parameters, it uses an intelligent matching algorithm to search for multiple corresponding projection points on the left and right images. Based on the corresponding projection points, it calculates and obtains the spatial attitude projection rotation angle correction value and scaling ratio correction value of the right image relative to the left image during photography, thus obtaining the spatial projection position attitude correction parameters. The images are rearranged using the projection correction parameters to obtain the projection deformation correction image. The projection deformation correction image is saved to the computer and converted to a display format for transmission to a stereoscopic display and network platform.

[0010] Optionally, the stereoscopic image display can be configured as an active stereoscopic display, a polarized stereoscopic display, or a glasses-free 3D display, and the carrier of the stereoscopic image display can include a glasses-free 3D laptop, a glasses-free 3D tablet computer, or a glasses-free 3D mobile phone.

[0011] This invention discloses the following technical advantages by providing a binocular high-resolution digital imaging real-time stereo vision microscope system: 1. Use computer calibration and analytical calculation to correct manufacturing parameter consistency errors: correct inconsistencies in precision manufacturing of the two optical systems, image projection distortion, reduce the precision manufacturing and installation requirements for pairing the two cameras, reduce the consistency requirements for objective lens focal length and distortion, reduce the installation error requirements for the main optical axis and sensor center, reduce the parallel installation error requirements for the two cameras, improve the performance of microscope equipment, and realize the digitization of stereoscopic images of stereo microscopes and expand the application range.

[0012] 2. Improve resolution and magnification and expand field of view: By combining objective lens magnification with digital image magnification, the resolution and magnification are improved while maintaining a longer optical depth of field, thus expanding the field of view for viewing stereoscopic microscopic images, overcoming the limitations of optical magnification diffraction and optical physical depth of field.

[0013] 3. More comfortable viewing and multi-mode display: naked-eye 3D, active, polarized light and other multi-mode stereoscopic display, avoids looking down and staring at the eyepiece, reduces fatigue during long-term viewing, and can be viewed on monitors of different sizes to view huge microscopic stereoscopic images.

[0014] 4. Super-depth-of-field stereoscopic image projection visual viewing capability: Multiple multi-distance focal plane stacking photography combined with super-depth-of-field synthesis to obtain super-depth-of-field stereoscopic image viewing. Improved operational adaptability, enabling functions such as multi-person viewing, saving, remote sharing, and navigation measurement.

[0015] 5. Improved the versatility of microscope equipment, enabling functions such as multi-person viewing, long-term storage, real-time remote sharing via network, and navigation measurement.

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the system architecture provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a binocular high-resolution digital image real-time stereo vision microscope system provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the optical lens distortion structure of a microscope objective provided in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the calculation and correction of the principal ray projection from the objective lens center onto the sensor position, provided in an embodiment of the present invention. Figure 5 A schematic diagram illustrating the principle of photographic spatial projection position correction provided in an embodiment of the present invention; Explanation of reference numerals in the attached figures: 11. First zoom parfocal optical path objective lens; 12. First microscopic magnification objective lens camera; 13. First zoom parfocal objective lens imaging plane; 21. Second zoom parfocal optical path objective lens; 22. Second microscopic magnification objective lens camera; 23. Second zoom parfocal objective lens imaging plane; 3. Stereoscopic image display; 4. Computer; 5. Large objective lens. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] like Figure 1 , Figure 2 As shown, the present invention provides a binocular high-resolution digital imaging real-time stereo vision microscope system, including a computer 4, a large objective lens 5, a binocular microscope body, a binocular imaging subsystem connected to the binocular microscope body, and a stereo image display 3 connected to the binocular imaging subsystem.

[0022] I. Binocular Microscope Body The binocular microscope body consists of a first zoom parfocal optical path and a second zoom parfocal optical path, which are symmetrically arranged. The binocular microscope body is mounted on a movable slide, allowing it to move at different heights or planar distances.

[0023] The first zoom parfocal optical path consists of a first zoom parfocal optical path objective lens S1 and a first microscopic magnifying objective lens camera L1. The first microscopic magnifying objective lens camera L12 is mounted perpendicularly to the optical image magnification imaging surface (first zoom parfocal objective lens imaging surface 13) of the first zoom parfocal optical path objective lens 11 for imaging. The second zoom parfocal optical path consists of a second zoom parfocal optical path objective lens S2 and a second microscopic magnifying objective lens camera L2. The second microscopic magnifying objective lens camera L22 is mounted perpendicularly to the optical image magnification imaging surface (second zoom parfocal objective lens imaging surface 23) of the second zoom parfocal optical path objective lens 21 for imaging. The two cameras simultaneously capture images to obtain nanometer-resolution digital images, and the captured digital images are transmitted in real time to a computer 4 or stored in a device.

[0024] The microscope objective cameras L1 and L2, depending on the application, use microscope lenses with different magnifications and sensors with different pixel sizes. The two cameras, L1 and L2, simultaneously capture images to obtain digital images with nanometer-level sampling resolution, and transmit the captured digital images to the computer 4 or store them in the device in real time.

[0025] The first zoom parfocal optical path objective S1 and the second zoom parfocal optical path objective S2 have the same focal length and manufacturing parameters. The mounting distance baseline M between the first zoom parfocal optical path objective S1 and the second zoom parfocal optical path objective S2 is set between 1cm and 8cm. The image overlap between the first zoom parfocal optical path objective S1 and the second zoom parfocal optical path objective S2 is set between 50% and 100%.

[0026] The binocular microscope body is equipped with a parallel light path mode and a cross-beam light path mode. In the parallel light path mode, the two objectives S1 and S2 are installed without any included angle. In the cross-beam light path mode, the inward included angle between the first zoom parfocal objective S1 and the second zoom parfocal objective S2 is set between 0.5° and 15°.

[0027] II. Binocular Imaging Subsystem 1. Image projection correction calculation module, such as Figure 3 , Figure 4 , Figure 5 As shown, the system is used to acquire original left and right image data through the binocular microscope, and to perform manufacturing and installation parameter consistency correction and photographic posture projection correction on the original left and right image data to obtain a projection distortion corrected image; the image projection correction calculation module includes: 1.1 Manufacturing parameter uniformity calculation unit, used to acquire original left and right image data using the binocular microscope body through a stacked stereoscopic imaging method with multiple multi-distance focal planes, and based on the original left and right image data, to perform objective lens distortion calibration and uniformity correction, and focal length... fCalibration and standardization corrections yielded a standardized image of the binocular microscope's manufacturing parameters.

[0028] 1.2 Objective lens projection center correction calculation unit, used to calculate the pixel positions of the principal rays projected from the center of the lenses of the first microscopic magnifying objective camera L1 and the second microscopic magnifying objective camera L2 onto the two sensors based on the binocular microscope body manufacturing parameter uniform image, and set the pixel positions as coordinate rotation points o1 and o2 for photographic space attitude projection calculation, and perform image center projection correction according to the coordinate rotation points.

[0029] 1.3 Photographic Projection Correction Calculation Unit: Based on the binocular microscope body manufacturing parameter distortion correction image, an intelligent matching algorithm is used to search for multiple corresponding projection points on the left and right images. According to the corresponding projection points, the spatial attitude projection rotation angle correction value and scaling ratio correction value of the right image relative to the left image during photography are calculated and obtained to obtain the spatial projection position attitude correction parameters. The images are rearranged using the projection correction parameters to obtain the projection distortion correction image. The projection distortion correction image is saved to the computer and the display format is converted for transmission to the stereoscopic display and network platform.

[0030] Specifically: the optical distortion of the two objectives S1 and S2 and the inconsistency of focal length f1≠f2 are calculated and corrected; the installation error of the principal optical axis of the microscope magnifying objective camera L1 and L2 and the sensor position is corrected; and the projection distortion caused by the installation position error of the two digital cameras is corrected.

[0031] Correction for optical distortion calculation of objectives: The optical lens manufacturing distortions of the two objectives S1 and S2 and the microscope magnifying camera lenses L1 and L2 are inconsistent. This optical distortion of the objectives leads to non-linear projection image deformation, severely affecting the viewing comfort of stereoscopic images. The distortion of the two objectives S1 and S2 is calibrated, and the distortion calculations of the two objectives S1 and S2 and the microscope magnifying camera lenses L1 and L2 are corrected to be consistent, with S1=S2.

[0032] Correction for objective lens focal length f1≠f2: If the optical manufacturing focal lengths f1≠f2 of the two objective lenses are inconsistent, the focal lengths of the two objective lenses will be corrected to be consistent, with f1=f2.

[0033] Correction calculation for the projection of the principal rays from the center of the microscope magnifying lens camera L1 and L2 onto the correct position of the sensor: Calculate the pixel position of the projection of the two principal rays from the center of the lens onto the electronic sensor, which are the coordinate rotation points o1 and o2 for the pose projection calculation in the photographic space.

[0034] The installation position error of the two cameras in the microscope causes projection distortion correction. The two cameras cannot be installed in an ideal parallel position, resulting in spatial attitude installation errors such as forward / backward, rotation, and pitch, which distort the projection of the captured left and right images. The solution is to calculate the projection correction parameters for the images captured by the two cameras at their respective installation positions, and then use these parameters to rearrange the distorted left and right projection images into projection images with the correct photographic attitude spatial position.

[0035] Microscope mounting attitude correction parameter calculation: Based on the deformed corrected image, an influence matching algorithm is used to search for multiple corresponding projection points A, B…N on the left and right images. According to the corresponding projection points, the spatial projection rotation angle correction value and scaling ratio correction value of the right image relative to the left image during photography are obtained, thus obtaining the spatial projection position attitude correction parameters. , ω, k, T.

[0036] 2. A stereoscopic image output calculation module is used to correct and rearrange the image based on the projection distortion correction image and using spatial projection position and attitude correction parameters to complete the correction of manufacturing and installation consistency errors and image projection errors, output a large depth-of-field digital stereoscopic image, transmit the large depth-of-field digital stereoscopic image to the stereoscopic image display for real-time display and storage, and transmit it to the network platform in real time.

[0037] The spatial projection distortion correction parameters are calculated by using an image matching algorithm to search for multiple corresponding projection points on the left and right digital images. Based on these multiple corresponding projection points, the spatial projection rotation angle and scaling correction values ​​of the right digital image relative to the left digital image during the photography are obtained, thus yielding the spatial projection position and attitude correction parameters.

[0038] Let S1 and S2 be the two projection centers, and let the images of object point A in S1 and S2 be α1 and α2, respectively. Then, the corresponding rays S1α1, S2α2 and S1S2 are coplanar, that is: Then we have: In the formula: x1, y1, f are the image coordinates of point A in S1, and x2, y2, z2 are the image coordinates of point A in S2. ω and k are the rotation angles of S2 relative to S1.

[0039] Based on the spatial projection position and attitude correction parameters, the projected images are rearranged to correct the projection distortion caused by the installation position errors of the two cameras.

[0040] The left and right image data after projection distortion calculation and correction are calculated and stored by computer 4 according to different stereoscopic image display format requirements, and simultaneously pushed to stereoscopic image display 3 and network platform.

[0041] 3. Super depth-of-field digital stereoscopic images: Using a digital imaging stereoscopic vision microscope system, a multi-stage, multi-distance focal plane stacking stereoscopic image photography method is employed. After computer processing and synthesis of four major depth-of-field projections, super depth-of-field digital stereoscopic vision images are obtained.

[0042] III. Stereoscopic Image Display Stereoscopic display 3 is a dedicated electronic display for viewing digital stereoscopic microscopic images. Types include active stereoscopic displays, polarized stereoscopic displays, and glasses-free 3D displays. Digital stereoscopic images are pushed to networks and network platforms, enabling viewing on mobile glasses-free 3D laptops, glasses-free 3D tablets, glasses-free 3D mobile phones, etc.

[0043] Therefore, this invention provides a binocular high-resolution digital image real-time stereoscopic vision microscope system. The analytical calculation method of this invention eliminates image projection distortion caused by errors in the manufacturing and installation of the microscope body, maintains a large stereoscopic scene depth, obtains high magnification, nanometer resolution sampling microscopic digital images, and realizes high-resolution, high-magnification microscopic digital image stereoscopic vision observation and detection.

[0044] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0045] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A binocular high-resolution digital imaging real-time stereo vision microscope system, characterized in that, It includes a binocular microscope body, a binocular imaging subsystem connected to the binocular microscope body, and a stereoscopic image display connected to the binocular imaging subsystem; The binocular microscope body is used for simultaneous imaging of binocular left and right magnified digital images; The binocular imaging subsystem includes: The image projection correction calculation module is used to acquire the original left and right image data through the binocular microscope, perform consistency correction and projection correction on the original left and right image data, and obtain the projection distortion corrected image. The stereoscopic image output calculation module is used to correct the rearranged image based on the projection distortion correction image and using spatial projection position and attitude correction parameters to complete the correction of manufacturing and installation consistency errors and image projection errors, output a large depth-of-field digital stereoscopic image, transmit the large depth-of-field digital stereoscopic image to the stereoscopic image display for real-time display and storage, and transmit it to the network platform in real time. The image projection correction calculation module and the stereoscopic image output calculation module are interconnected.

2. The binocular high-resolution digital imaging real-time stereo vision microscope system according to claim 1, characterized in that, The binocular microscope body is composed of a first zoom parfocal optical path and a second zoom parfocal optical path, which are symmetrically arranged.

3. The binocular high-resolution digital imaging real-time stereo vision microscope system according to claim 2, characterized in that, The first zoom parfocal optical path consists of a first zoom parfocal optical path objective lens and a first microscopic magnifying objective lens camera. The first microscopic magnifying objective lens camera is set and installed perpendicularly to the optical image magnification imaging surface of the first zoom parfocal optical path objective lens for imaging. The second zoom parfocal optical path consists of a second zoom parfocal optical path objective lens and a second microscopic magnifying objective lens camera. The second microscopic magnifying objective lens camera is set and installed perpendicularly to the optical image magnification imaging surface of the second zoom parfocal optical path objective lens for imaging. The first and second microscope magnifying objective cameras have the same manufacturing parameters, and different magnification microscope lenses and sensors with different pixel sizes are selected according to their uses.

4. A binocular high-resolution digital imaging real-time stereo vision microscope system according to claim 3, characterized in that, The focal length and distortion manufacturing parameters of the first zoom parfocal optical path objective and the second zoom parfocal optical path objective are the same. The baseline of the mounting distance between the first zoom parfocal optical path objective and the second zoom parfocal optical path objective is set between 1cm and 8cm. The image overlap between the first zoom parfocal optical path objective and the second zoom parfocal optical path objective is set between 50% and 100%.

5. A binocular high-resolution digital imaging real-time stereo vision microscope system according to claim 4, characterized in that, The binocular microscope body is equipped with a parallel light path mode and a cross-beam light path mode. In the cross-beam light path mode, the inward angle between the first zoom parfocal objective lens and the second zoom parfocal objective lens is set to between 0.5° and 15°.

6. A binocular high-resolution digital imaging real-time stereo vision microscope system according to claim 5, characterized in that, The image projection correction calculation module includes: The manufacturing parameter uniformity calculation unit is used to acquire the original left and right image data through the binocular microscope body, and perform objective lens distortion calibration and uniformity correction and focal length calibration and uniformity correction based on the original left and right image data to obtain the binocular microscope body manufacturing parameter uniformity image. The objective lens projection center correction calculation unit is used to calculate the pixel positions of the central principal rays of the first zoom parfocal optical path objective lens and the second zoom parfocal optical path objective lens on the two sensors based on the binocular microscope body manufacturing parameter uniform image, and set the pixel positions as coordinate rotation points for photographic space attitude projection calculation, and perform image center projection correction according to the coordinate rotation points. Photographic projection correction calculation unit: Based on the deformation correction image of the binocular microscope body manufacturing parameters, it uses an intelligent matching algorithm to search for multiple corresponding projection points on the left and right images. Based on the corresponding projection points, it calculates and obtains the spatial attitude projection rotation angle correction value and scaling ratio correction value of the right image relative to the left image during photography, thus obtaining the spatial projection position attitude correction parameters. The images are rearranged using the projection correction parameters to obtain the projection deformation correction image. The projection deformation correction image is saved to the computer and converted to a display format for transmission to a stereoscopic display and network platform.

7. A binocular high-resolution digital imaging real-time stereo vision microscope system according to claim 6, characterized in that, The stereoscopic image display is an active stereoscopic display, a polarized stereoscopic display, or a glasses-free 3D display, and the carrier of the stereoscopic image display includes glasses-free 3D laptops, glasses-free 3D tablets, or glasses-free 3D mobile phones.

Images