3D calibration method for 3D optical equipment and surgical microscope

Abstract

The present disclosure provides a 3D calibration method for a 3D optical device and a surgical microscope. The method comprises: obtaining a left-eye image and a right-eye image captured by a binocular camera in the 3D optical device; performing window adjustment based on the obtained left-eye image and right-eye image based on a preset window, wherein the window adjustment includes shifting at least one of a first segmentation window in the left-eye image and a second segmentation window in the right-eye image; segmenting the left-eye image and the right-eye image according to the adjusted first segmentation window and the adjusted second segmentation window, respectively, to obtain segmented left-eye images and right-eye images; enlarging the segmented left-eye image and right-eye image to a target image size, and outputting the enlarged left-eye image and right-eye image in a left-right arrangement to a 3D imaging display device. The disclosed technical solution can improve the efficiency and accuracy of 3D calibration.

Description

Technical Field

[0001] The present disclosure relates to the field of 3D display technology, and in particular to a 3D calibration method for 3D optical equipment and a surgical microscope. Background Art

[0002] When performing three-dimensional display based on a 3D optical device, the installation position of the binocular camera in the 3D optical device needs to be calibrated to meet the binocular camera shooting parallax and shooting angle required to present the best three-dimensional display effect.

[0003] Currently, the mounting position of binocular cameras in 3D optical equipment is typically adjusted using additional mechanical devices or manual methods. However, 3D calibration using mechanical devices significantly increases costs and limits the adjustment range. Manual calibration cannot precisely control the adjustment distance, making it easy to over- or under-adjust, requiring repeated adjustments and reducing 3D calibration efficiency. Summary of the Invention

[0004] The present disclosure provides a 3D calibration method for a 3D optical device and a surgical microscope, which can perform 3D calibration by adjusting the relative positions of left and right eye images without adjusting the installation position of a binocular camera, thereby improving the efficiency and accuracy of 3D calibration.

[0005] In a first aspect, an embodiment of the present disclosure provides a 3D calibration method for a 3D optical device, comprising:

[0006] Obtaining left-eye images and right-eye images captured by binocular cameras in a 3D optical device;

[0007] Based on a preset window, and according to the acquired left-eye image and right-eye image, performing window adjustment, wherein the window adjustment includes shifting at least one of a first segmentation window in the left-eye image and a second segmentation window in the right-eye image;

[0008] Segmenting the left-eye image and the right-eye image according to the first segmentation window and the second segmentation window after the window adjustment to obtain segmented left-eye image and right-eye image;

[0009] The segmented left-eye image and right-eye image are both enlarged to a target image size, and the enlarged left-eye image and right-eye image are output to a 3D imaging display device in a left-right arrangement format.

[0010] In a second aspect, the present disclosure also provides a surgical microscope, including a binocular camera, a processing module, and a 3D imaging display device; wherein,

[0011] The binocular camera is used to capture left-eye images and right-eye images;

[0012] The processing module is configured to implement the method described in any embodiment of the present disclosure according to the captured left-eye image and right-eye image;

[0013] The 3D imaging display device is used for performing 3D imaging display on input left-eye images and right-eye images.

[0014] In a third aspect, an embodiment of the present disclosure further provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to execute the method as described in any embodiment of the present disclosure.

[0015] The technical solution provided by the embodiments of the present disclosure adopts a method of performing window adjustment based on the left-eye image and the right-eye image respectively captured by the binocular camera in the 3D optical device: at least one of the first segmentation window in the left-eye image and the second segmentation window in the right-eye image is displaced, and the first segmentation window and the second segmentation window after the window adjustment are used to segment the left-eye image and the right-eye image respectively, and the segmented left-eye image and the right-eye image are both enlarged to the target image size, thereby achieving the adjustment of the relative positions of the left and right eye images, and outputting the enlarged left-eye image and right-eye image in a left-right arrangement format to the 3D imaging display device for presenting the 3D display effect, so as to achieve the best 3D display effect by adjusting the relative positions of the left and right eye images, avoiding the tediousness and uncertainty of manual operation, and not requiring the addition of additional mechanical devices, with the advantages of ease of use, high correction accuracy, time saving and no interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0017] Figure 1A The following is a flowchart illustrating a 3D calibration method for a 3D optical device according to an embodiment of the present disclosure;

[0018] Figure 1B An example diagram showing a captured image and a segmented window according to an embodiment of the present disclosure is exemplarily shown;

[0019] Figure 1C An exemplary diagram illustrating an image magnification process according to an embodiment of the present disclosure is shown;

[0020] Figure 1D An exemplary diagram showing a 3D display according to an embodiment of the present disclosure is shown;

[0021] Figure 1EAn example diagram showing left-eye and right-eye image correction according to an embodiment of the present disclosure is exemplarily shown;

[0022] Figure 1F An example diagram of pixel size calibration according to an embodiment of the present disclosure is exemplarily shown;

[0023] Figure 2A The following is a flowchart illustrating a 3D calibration method for a 3D optical device according to another embodiment of the present disclosure;

[0024] Figure 2B An exemplary diagram showing a window moving to the left according to an embodiment of the present disclosure is shown;

[0025] Figure 3A A flowchart of a 3D calibration method for a 3D optical device according to another embodiment of the present disclosure is exemplarily shown;

[0026] Figure 3B An example diagram of the optical path of a binocular camera according to an embodiment of the present disclosure is exemplarily shown;

[0027] Figure 4A A flowchart of a 3D calibration method for a 3D optical device according to another embodiment of the present disclosure is exemplarily shown;

[0028] Figure 4B An exemplary diagram showing a window moving downward according to an embodiment of the present disclosure is shown;

[0029] Figure 5 A schematic diagram of a surgical microscope according to an embodiment of the present disclosure is exemplarily shown. DETAILED DESCRIPTION

[0030] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0031] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0032] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0033] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0034] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0035] Figure 1A A flowchart illustrating a 3D calibration method for a 3D optical device according to an embodiment of the present disclosure is provided. This embodiment of the present disclosure is suitable for calibrating 3D display effects by adjusting the relative positions between left and right eye images captured by the 3D optical device. This method can be executed by a processing module within the 3D optical device, which can be implemented in software and / or hardware. Optionally, the 3D optical device can be any optical device capable of 3D display effects. For example, the 3D optical device can be, but is not limited to, a 3D surgical microscope.

[0036] like Figure 1A As shown, the 3D optical device 3D calibration method may include steps S110-S140.

[0037] In step S110 , a left-eye image and a right-eye image respectively captured by binocular cameras in a 3D optical device are acquired.

[0038] In the disclosed embodiment, the binocular cameras in step S110 include a left-eye camera and a right-eye camera. The 3D optical device uses the images captured by these two cameras to record them as left-eye images and right-eye images, respectively. Using these captured left-eye images and right-eye images, the binocular cameras in the 3D optical device are calibrated in 3D.

[0039] In step S120, based on the preset window and the acquired left-eye image and right-eye image, window adjustment is performed, where the window adjustment includes shifting at least one of the first segmentation window in the left-eye image and the second segmentation window in the right-eye image.

[0040] It should be understood that the preset window is a pre-set window for splitting the left and right eye images. The window size of the preset window is smaller than the image size of the left and right eye images. The preset window includes a first splitting window and a second splitting window. The first splitting window is a window for cutting out part of the image in the left eye image. The second splitting window is a window for cutting out part of the image in the right eye image. The window size of the first splitting window is smaller than the image size of the left eye image. The window size of the second splitting window is smaller than the image size of the right eye image. The window size of the first splitting window can be the same as the window size of the second splitting window so that images of the same size are cut out. The window sizes of the first splitting window and the second splitting window can be set based on business needs. The window adjustment in step S120 is to adjust the position of the window in the image. Different window positions result in different image areas being cut out. For example, see Figure 1B The image size of the left and right eye images (i.e., the real frame area) is 3840×2160, and the window size of the segmentation window (i.e., the virtual frame area) is 3600×2026. Therefore, a partial image of 3600×2026 can be cut out from the 3840×2160 complete image captured by the binocular camera through the segmentation window.

[0041] In step S120, based on the left-eye image and the right-eye image obtained in step S110, at least one of the first split window in the left-eye image and the second split window in the right-eye image is displaced, thereby changing the relative position between the images in the two windows and achieving movement of the images in the field of view. As an implementation method, only one of the split windows can be adjusted in position, while the position of the other split window remains fixed, thereby changing the relative position of the images in the two windows. As another implementation method, the positions of the two split windows can be adjusted simultaneously, that is, the positions of the first split window and the second split window are adjusted simultaneously, thereby also changing the relative positions of the images in the two windows. The specific implementation method can be set based on business needs and is not limited here.

[0042] In step S130 , the left-eye image and the right-eye image are segmented according to the adjusted first segmentation window and the adjusted second segmentation window to obtain segmented left-eye image and right-eye image.

[0043] It should be understood that, based on the position of the first segmentation window in the left-eye image after window adjustment, the image within the first segmentation window is segmented from the left-eye image, and the image within the first segmentation window is used as the segmented left-eye image. Similarly, based on the position of the second segmentation window in the right-eye image after window adjustment, the image within the second segmentation window is segmented from the right-eye image, and the image within the second segmentation window is used as the segmented right-eye image. By using the first segmentation window and the second segmentation window to segment the left and right eye images, partial left and right eye images can be obtained.

[0044] In step S140 , the segmented left-eye image and right-eye image are both enlarged to a target image size, and the enlarged left-eye image and right-eye image are output to a 3D imaging display device in a left-right arrangement format.

[0045] The 3D imaging display device in step S140 may be, but is not limited to, a 3D television. The target image size is an image size compatible with the 3D imaging display device so that the enlarged left-eye image and right-eye image can be displayed in 3D imaging on the 3D imaging display device. For example, if the 3D imaging display device is a 4K display device, the segmented left-eye image and right-eye image need to be enlarged to an image size of 3840×2160 so that the 3D imaging display device can use the input 3840×2160 left-eye and right-eye images for 3D imaging and display as 3D images on the 3D imaging display device.

[0046] For example, see Figure 1C , the image sizes of the left and right eye images taken by the binocular camera are both image sizes adapted to the 3D imaging display device, so that the image in the first split window can be enlarged to the same image size as the left eye image, and the image in the second split window can be enlarged to the same image size as the right eye image. It should be noted that in the image magnification process, the magnification factor required for the image is very small, and the negative impact of the interpolation algorithm can be ignored. This embodiment can adjust the phase position between the left and right eye images by splitting and enlarging the captured left and right eye images, thereby achieving the same effect as adjusting the installation position of the binocular camera, meeting the needs of 3D image display. The 3D calibration method provided by the disclosed embodiment is flexible in control, easy to use, has high correction accuracy, does not increase additional costs, and thus greatly improves the installation efficiency.

[0047] For example, see Figure 1D By adjusting the window displacement of the left and right eye images taken by the binocular camera, the relative position of the captured images is changed, thereby optimizing the 3D display effect. It can also replace the adjustment of the physical position of the binocular camera, solving the problems of low efficiency, poor accuracy and susceptibility to vibration of the traditional binocular camera position calibration method.

[0048] It should be noted that if the 3D image displayed on the 3D imaging display device achieves the optimal 3D display effect after this window adjustment, it indicates that the 3D calibration is complete. There is no need to adjust the window again during the next 3D imaging. The left and right eye images taken next time can be directly split and enlarged according to the split window after this adjustment, so that the 3D image displayed on the 3D imaging display device next time can achieve the optimal 3D display effect.

[0049] In the embodiment of the present disclosure, by shifting at least one of the first segmentation window in the left-eye image and the second segmentation window in the right-eye image, and using the adjusted first segmentation window and the second segmentation window to segment the left-eye image and the right-eye image respectively, and enlarging the segmented left-eye image and the right-eye image to the target image size, the relative positions of the left and right eye images are adjusted, and the enlarged left-eye image and the right-eye image are output to the 3D imaging display device in a left-right arrangement format for presenting a 3D display effect, so as to achieve the best 3D display effect by adjusting the relative positions of the left and right eye images, avoid the tediousness and uncertainty of manual operation, and do not require the addition of additional mechanical devices. The method has the advantages of ease of use, high correction accuracy, time saving, and no interference.

[0050] As an optional embodiment, before performing the window adjustment, the method further includes: correcting the left-eye image and the right-eye image obtained in step S110, and / or performing pixel calibration.

[0051] The calibration process includes at least one of the following calibration operations:

[0052] Hardware Installation and Calibration: Perform preliminary installation of the left and right eye cameras and secure them according to the structural and optical design requirements. The left and right cameras must be installed correctly, meaning the clockwise or counterclockwise rotation between the images captured by the left and right cameras cannot exceed 5 degrees. The left and right cameras must be installed on the same horizontal plane, ensuring that the images captured by the two cameras are not larger or smaller than each other. It is important to note that once the binocular cameras are installed, they should be fixed in place; there is no need to adjust their position during the subsequent 3D calibration process.

[0053] Software alignment correction: that is, automatically initialize and correct the left eye image and right eye image obtained in step S110. Under fixed parameters, such as focal length 10mm and working distance 20mm, shoot the calibration template. The software will obtain the rotation matrix and translation vector of the two cameras based on the captured checkerboard image, and then generate a mapping table to align the left and right images. After correction, according to the solution of this application, further 3D adaptive adjustment is performed to optimize the 3D display effect, see Figure 1E shown.

[0054] By performing the above correction operation on the left-eye image and the right-eye image acquired in step S110 , the 3D calibration efficiency and the 3D display effect can be further improved.

[0055] Pixel calibration is to determine the relationship between physical displacement and pixel displacement and perform calibration. For example: Figure 1F, set the magnification to 10x, and if the maximum image size within the field of view is 32mm × 18mm, then in 4K image (resolution 3840 × 2160), the pixel size Lpixel = 32 / 3840 = 8.3µm / pixel. The pixel size can be used to convert between length units and pixel counts.

[0056] Figure 2A The flowchart of a 3D calibration method for a 3D optical device according to another embodiment of the present disclosure is exemplarily shown.

[0057] like Figure 2A As shown, the 3D optical device 3D calibration method may include the following steps S210-S240:

[0058] In step S210 , a left-eye image and a right-eye image respectively captured by binocular cameras in a 3D optical device are acquired.

[0059] This step and Figure 1A The steps S110 shown are the same or similar and will not be described in detail in this embodiment of the present disclosure.

[0060] In step S220, based on the preset window and the acquired left-eye image and right-eye image, window adjustment is performed. The window adjustment includes shifting at least one window in the left-right direction according to the input movement distance of at least one window in the left-right direction.

[0061] The movement distance input in step S220 can be expressed in pixels or units of length. For example, the movement distance is 20 pixels or 2 mm. The movement direction is either leftward or rightward, thereby adjusting the parallax between the left and right eye images.

[0062] In step S220, by observing the relative position between the left eye image and the right eye image captured by the binocular camera, the movement distance of at least one of the first segmentation window and the second segmentation window in the left-right direction can be input, and then the at least one window can be displaced in the left-right direction according to the input movement distance, thereby achieving the left-right movement of the image in the field of view. For example, see Figure 2B , moving the window 20 pixels to the left on the captured image, which is equivalent to moving the image 20 pixels to the left in the field of view.

[0063] When only one of the split windows (i.e., the first split window or the second split window) is adjusted to the left or right position, the split window to be adjusted, the moving direction and moving distance of the split window can be input, and the split window can be displaced in the specified direction and specified distance according to the input moving direction and moving distance, so as to adjust the relative position between the left and right eye images in the left and right directions.

[0064] When simultaneously adjusting the left and right positions of two split windows (i.e., the first split window and the second split window), as an implementation method, the movement direction and movement distance of the first split window and the movement direction and movement distance of the second split window can be input. Based on the input movement direction and movement distance, the first split window and the second split window are controlled to shift in a specified direction and distance, thereby adjusting the relative position between the left and right eye images in the left and right directions. The movement directions of the first and second split windows can be the same or different. The movement distances of the first and second split windows can be the same or different, and are input based on specific needs.

[0065] When simultaneously adjusting the left and right positions of two split windows (i.e., the first and second split windows) for left-right position adjustment, another implementation method is to input only a positive or negative movement distance. Based on the input movement distance, the first and second split windows are controlled to move the same specified distance in opposite directions, thereby adjusting the relative position between the left and right eye images in the left-right direction. For example, when simultaneously adjusting the first and second split windows, the first and second split windows are symmetrically moved by the same distance in the left-right direction. The first and second split windows move in opposite directions but the same distance. A positive movement distance indicates that the first split window moves leftward and the second split window moves rightward, increasing the relative distance between the left and right eye images. A negative movement distance indicates that the first split window moves rightward and the second split window moves leftward, decreasing the relative distance between the left and right eye images.

[0066] It should be understood that the input movement distance information allows the left and right eye images to be adjusted in the left and right directions according to the pixel stepping, thereby improving the 3D calibration accuracy.

[0067] In step S230 , the left-eye image and the right-eye image are segmented according to the adjusted first segmentation window and the second segmentation window to obtain segmented left-eye images and right-eye images.

[0068] This step and Figure 1A The step S130 shown is the same or similar, and will not be described in detail in this embodiment of the present disclosure.

[0069] In step S240 , the segmented left-eye image and right-eye image are both enlarged to a target image size, and the enlarged left-eye image and right-eye image are output to a 3D imaging display device in a left-right arrangement format.

[0070] This step and Figure 1A The steps S140 shown are the same or similar, and will not be described in detail in this embodiment of the present disclosure.

[0071] In the disclosed embodiment, the left-right displacement of the left and right eye images is controlled by inputting the movement distance of at least one window in the left-right direction and displacing the at least one window in the left-right direction according to the input information. The use of the input control method can avoid the tediousness and uncertainty of manual calibration operations, greatly improving the calibration efficiency, while also avoiding the large size and additional cost of the mechanical device and the influence of the camera position offset caused by vibration.

[0072] Figure 3A The flowchart of a 3D calibration method for a 3D optical device according to another embodiment of the present disclosure is exemplarily shown.

[0073] like Figure 3A As shown, the 3D optical device 3D calibration method may include the following steps S310-S340:

[0074] In step S310 , a left-eye image and a right-eye image respectively captured by binocular cameras in a 3D optical device are acquired.

[0075] This step and Figure 1A The steps S110 shown are the same or similar and will not be described in detail in this embodiment of the present disclosure.

[0076] In step S320, based on the preset window, window adjustment is performed according to the acquired left-eye image and right-eye image. The window adjustment includes adjusting the distance according to the input left-eye and right-eye parallax, determining the movement distance of at least one window in the left-right direction, and shifting the at least one window in the left-right direction according to the movement distance in the left-right direction.

[0077] The left-eye disparity adjustment distance input in step S320 refers to the change in the relative distance between the left-eye image and the right-eye image. The input left-eye disparity adjustment distance can be a positive or negative value. A positive value indicates an increase in left-eye disparity, while a negative value indicates a decrease. The left-eye disparity adjustment distance can also be expressed in pixels or units of length. For example, the left-eye disparity adjustment distance is an increase of 20 pixels.

[0078] In step S320, when only one of the split windows (i.e., the first or second split window) is to be adjusted left and right, the split window to be adjusted and the left-eye parallax adjustment distance (with positive and negative values) can be input. Based on the input left-eye parallax adjustment distance, the split window's movement direction and distance are determined, and the corresponding displacement is performed to adjust the relative position of the left and right eye images in the left-right direction. For example, if the split window to be adjusted is the first split window and the left-eye parallax adjustment distance is +20, the first split window is moved 20 pixels to the left to increase the left-eye parallax by 20 pixels. If the split window to be adjusted is the second split window and the left-eye parallax adjustment distance is +20, the second split window is moved 20 pixels to the right to increase the left-eye parallax by 20 pixels.

[0079] In step S320, when the left-right positions of the two split windows (i.e., the first split window and the second split window) are adjusted simultaneously, the movement direction and distance of the first split window and the second split window are determined based on the input left-right disparity adjustment distance, which has positive and negative values. The first and second split windows are shifted accordingly to adjust the relative position between the left and right eye images in the left-right direction. Exemplarily, when the first and second split windows are adjusted simultaneously, the first and second split windows are shifted symmetrically by the same distance in the left-right direction. For example, when the input left-right disparity adjustment distance is a positive value, the first split window is shifted leftward and the second split window is shifted rightward to increase the relative distance between the left and right eye images. When the input left-right disparity adjustment distance is a negative value, the first split window is shifted rightward and the second split window is shifted leftward to decrease the relative distance between the left and right eye images. The input left-right eye disparity adjustment distance is divided by 2, and the obtained result is used as the movement distance of the first segmentation window and the second segmentation window, so that the first segmentation window and the second segmentation window move symmetrically by the same distance in the left-right direction.

[0080] It should be understood that the input left-eye and right-eye disparity adjustment distance allows the left-eye and right-eye images to be positioned in the left-right direction according to the pixel adjustment granularity, thereby improving the 3D calibration accuracy.

[0081] The left-eye and right-eye parallax adjustment distance input in step S320 can be determined by at least the following two implementation methods:

[0082] As an implementation method, the relative position between the left-eye image and the right-eye image captured by the binocular camera can be observed, and the left-eye and right-eye parallax adjustment distance can be manually input. Then, according to the input left-eye and right-eye parallax adjustment distance, at least one window can be displaced in the left-right direction, thereby realizing the left-right position movement of the image in the field of view.

[0083] As another implementation method, the left and right eye parallax adjustment distance can be automatically determined and input based on the parameter information of the binocular camera and the left and right eye target parallax, so that the window can be displaced in the left and right directions more quickly and accurately based on the automatically input left and right eye parallax adjustment distance, thereby realizing the left and right position movement of the image in the field of view.

[0084] For example, according to the formula Determine the effective left-eye and right-eye disparity under the specific parameters of the binocular camera, and determine the input left-eye and right-eye disparity adjustment distance according to the left-eye and right-eye target disparity and the effective left-eye and right-eye disparity. is the effective parallax of the left and right eyes of the binocular camera, is the baseline length of the binocular camera, is the focal length of the binocular camera, is the working distance of the binocular camera.

[0085] See also Figure 3B , baseline length Refers to the middle distance between the two lenses, in millimeters. It refers to the distance between the lens and the object A, and the unit can be millimeters. The object A can be the patient's eye, etc. ,focal length and working distance Substitute into the formula The effective parallax of the left and right eyes of the binocular camera can be determined . Effective parallax between left and right eyes It can be used to characterize the left-eye and right-eye parallax of a binocular camera under specific parameters. The left-eye parallax is set based on business needs and needs to be adjusted. The left-eye target parallax is the effective left-eye parallax that conforms to human observation. For example, the actual observer's interocular distance can be determined as the left-eye target parallax. , so that the actual observer can see the best 3D display effect. Subtract the effective parallax of the left and right eyes The obtained difference is determined as the input left-eye disparity adjustment distance, and the window is adjusted according to the left-eye disparity adjustment distance, that is, the first split window and the second split window have a relative displacement of the difference, so that the disparity between the left-eye and right-eye images after adjustment is equal to the left-eye target disparity. , so that the optimal 3D display effect can be achieved more quickly through a single window adjustment. It should be noted that, according to the calibrated pixel size, the left and right eye parallax adjustment distance can be converted from length units to pixel numbers. In addition, when the focal length of the binocular camera is and working distance When changes occur, it is necessary to re-determine and re-enter the parallax adjustment distance between the left and right eyes, and then re-perform 3D calibration to ensure the optimal display effect of the 3D image.

[0086] It should be understood that when the parallax is moderate, the brain can fuse the two images and produce a sense of three-dimensionality. If the parallax is too large, it may cause double vision (double images) or visual fatigue; if the parallax is too small, it may not be able to effectively perceive depth. Therefore, according to the principle of 3D imaging of the human eye, the effective parallax B of the left and right eyes needs to be guaranteed to be within the target parallax standard range, that is, B min ≤B≤B max 3D effect can only be produced within.

[0087] For example, the maximum value B in the target parallax standard range is max and minimum value B min According to the formula and formula And the minimum field of view and maximum field of view To determine, among them, is the field of view angle. Among them, the minimum field of view angle and maximum field of view It is determined based on the optimal horizontal range of the human eye. For example, in the application scenario of surgical microscope, the minimum field of view angle is Set to 0.5 degrees, maximum field of view Set to 2 degrees.

[0088] Specifically, the given baseline length and minimum field of view Substitute into the formula The minimum working distance can be determined in , the given baseline length ,focal length and minimum working distance Substitute into the formula The maximum value B in the target parallax standard range can be determined max Similarly, the given baseline length and minimum field of view Substitute into the formula The maximum working distance can be determined in , the given baseline length ,focal length and maximum working distance Substitute into the formula The minimum value B in the target disparity representation range can be determined min It should be noted that when the baseline length of the binocular camera is and focal length When the change occurs, the minimum value B in the target parallax standard range needs to be re-determined. min and the maximum value B max .

[0089] For example, if the left and right eye target disparity is set Within target parallax standard range B min ≤B≤B max If the target parallax is set, it means the left and right eyes are In order to present the best 3D display effect, the parallax of the left and right eye targets can be directly The difference between the effective left and right eye disparity B is subtracted to determine the left and right eye disparity adjustment distance, and the window is adjusted according to the difference so that the left and right eye images after the window adjustment can present the best 3D display effect.

[0090] Exemplarily, there is also an adjustment coefficient k, and the left and right eye target disparity is adjusted according to the adjustment coefficient to update the left and right eye target disparity so that the left and right eye target disparity is within the target disparity standard range. Specifically, if the left and right eye target disparity Not within target parallax standard range B min ≤B≤B max Within, the left and right eye target disparity is adjusted according to the adjustment coefficient k Adjustment is performed so that the updated left and right eye target disparity is within the target disparity standard range. It can be a coefficient between 0 and 1. and adjustment coefficient The multiplication result is within the target disparity standard range, and the multiplication result is used as the adjusted left and right eye target disparity. ,Right now . According to the updated left and right eye target disparity Determine the distance for adjusting the left and right eye parallax. For example, adjust the updated left and right eye target parallax The difference between the effective left-eye and right-eye disparity B is subtracted to determine the left-eye and right-eye disparity adjustment distance. Window adjustment is performed based on this difference to ensure that the left-eye and right-eye images after window adjustment can also present the optimal 3D display effect. It should be noted that this adjustment coefficient can be a fixed value or a mapping table based on specific parameters. The adjustment parameter k is not limited to a fixed value. Furthermore, this adjustment coefficient setting is only a preferred setting for the target disparity.

[0091] For example, the adjustment coefficient This can be determined in at least three ways:

[0092] As the first implementation method: according to the left and right eye target parallax , minimum effective parallax B min and the maximum effective parallax B max Determine the adjustment coefficient For example, if the left and right eye target disparity Less than the minimum effective parallax B min , then the minimum effective disparity B min Divided by the left and right eye target disparity The result is used as the adjustment coefficient ,Right now =B min / ; If the left and right eye target disparity Greater than the maximum effective parallax B max , then the maximum effective disparity B max Divided by the left and right eye target disparity The result is used as the adjustment coefficient ,Right now =B max / .

[0093] As a second implementation method: adjust the coefficient Set to Sigmoid function, normalization function or hyperbolic tangent function, and use Sigmoid function, normalization function or hyperbolic tangent function to convert the left and right eye target disparity Mapping to target parallax standard range B min ≤B≤B max Inside.

[0094] As a third implementation method: pre-set different left and right eye target disparities , baseline length ,focal length and working distance Conduct multiple experiments under different conditions to determine the adjustment coefficient for each case. And store it in the table, and then you can determine the adjustment coefficient by looking up the table

[0095] In step S330 , the left-eye image and the right-eye image are segmented according to the adjusted first segmentation window and the second segmentation window to obtain segmented left-eye images and right-eye images.

[0096] This step and Figure 1A The step S130 shown is the same or similar, and will not be described in detail in this embodiment of the present disclosure.

[0097] In step S340 , the segmented left-eye image and right-eye image are both enlarged to a target image size, and the enlarged left-eye image and right-eye image are output to a 3D imaging display device in a left-right arrangement format.

[0098] This step and Figure 1A The steps S140 shown are the same or similar, and will not be described in detail in this embodiment of the present disclosure.

[0099] In the disclosed embodiment, a left-right eye parallax adjustment distance is input, and at least one window is displaced in the left-right direction according to the input left-right eye parallax adjustment distance, thereby controlling the left-right displacement of the left-right eye image by utilizing the input left-right eye parallax adjustment distance. The use of the input control method can avoid the tediousness and uncertainty of manual calibration operations, greatly improving the calibration efficiency, while also avoiding the large size and additional cost of the mechanical device and the influence of the camera position offset caused by vibration.

[0100] Figure 4A The following exemplarily shows a flowchart of a 3D calibration method for a 3D optical device according to yet another embodiment of the present disclosure.

[0101] like Figure 4A As shown, the 3D optical device 3D calibration method may include the following steps S410-S440:

[0102] In step S410 , a left-eye image and a right-eye image respectively captured by binocular cameras in a 3D optical device are acquired.

[0103] This step and Figure 1A The steps S110 shown are the same or similar and will not be described in detail in this embodiment of the present disclosure.

[0104] In step S420, based on the preset window and according to the acquired left-eye image and right-eye image, the window adjustment is performed, and the window adjustment includes shifting at least one of the first segmentation window in the left-eye image and the second segmentation window in the right-eye image in the left-right direction and the up-down direction.

[0105] The window adjustment in step S420 is: on the basis of shifting at least one of the first split window and the second split window in the left-right direction, shifting in the up-down direction is also performed, so as to achieve the movement of the image in the upper and lower positions in the field of view. Figure 4B , moving the window down 20 pixels on the captured image, which is equivalent to moving the image down 20 pixels in the field of view.

[0106] The process of shifting the window up and down based on the left and right shifting can be achieved in at least the following ways:

[0107] As a first implementation, the vertical window movement distance is a fixed value, which can be determined based on parameter information of the 3D optical device. For example, the minimum vertical window movement distance of at least one window is 10 pixels, and the maximum is 30 pixels. Based on conventional 4K display devices, the vertical window movement distance can be set within a range of 10 to 30 pixels. Adjusting the vertical pixel displacement based on the left and right window adjustments can enhance the stereoscopic perception of the image while minimizing the misjudgment of depth information.

[0108] As a second implementation, the vertical movement distance of at least one window is determined based on the horizontal movement distance of at least one window. After the horizontal movement distance of the window is determined based on input parameters, the vertical movement distance of the window can be automatically determined based on the horizontal movement distance, thereby achieving coordinated window displacement. For example, the horizontal movement distance of the window can be the same as the vertical movement distance, so that the image moves the same distance in both the vertical and horizontal directions. Alternatively, the vertical movement distance of at least one window is a preset multiple of the corresponding horizontal displacement distance, with the minimum value of the preset multiple being 1 / 10 and the maximum value being 1 / 5. The correspondence between the vertical and horizontal directions can be set based on business needs, for example, a leftward movement corresponds to an upward movement, and a rightward movement corresponds to a downward movement. If a window moves n pixels to the left, it also moves a×n pixels upward, where a is a preset multiple less than 1, for example, a range of values ​​from 1 / 10 to 1 / 5, thereby highlighting the horizontal adjustment of the image.

[0109] As a third implementation, at least one window is displaced vertically based on an input vertical movement distance of at least one window. By observing the relative position between the left-eye image and the right-eye image captured by the binocular camera and inputting a vertical movement distance of at least one of the first and second split windows, the at least one window is displaced vertically based on the input movement distance, thereby achieving flexible vertical movement of the image in the field of view to meet the user's personalized needs.

[0110] In step S430 , the left-eye image and the right-eye image are segmented according to the adjusted first segmentation window and the second segmentation window to obtain segmented left-eye images and right-eye images.

[0111] This step and Figure 1A The steps S110 shown are the same or similar and will not be described in detail in this embodiment of the present disclosure.

[0112] In step S440 , the segmented left-eye image and right-eye image are both enlarged to a target image size, and the enlarged left-eye image and right-eye image are output to a 3D imaging display device in a left-right arrangement format.

[0113] This step and Figure 1A The steps S110 shown are the same or similar and will not be described in detail in this embodiment of the present disclosure.

[0114] In the embodiment of the present disclosure, by shifting at least one window in the first split window and the second split window in the left-right direction, a shift is also performed in the up-down direction, thereby achieving the movement of the image in the field of view, thereby better adjusting the relative position between the left and right eye images, and further improving the 3D calibration effect.

[0115] As an optional embodiment, the method further includes: when performing image segmentation, reducing and adjusting the segmentation range of the preset window according to at least one window displacement information.

[0116] It should be noted that the adjustment of the window position will cause visual distortion of the edge of the image. For example, after the window is adjusted, no image is displayed on the edge of the image. Therefore, when the image is segmented, the distorted pixels on the edge are also segmented by shrinking the window, thereby ensuring that the segmented image is all valid pixels, eliminating the visual impact of the distorted image on the overall image.

[0117] It should be understood that when performing image segmentation, a window cropping range is determined based on at least one piece of window displacement information, and the first and second segmentation windows are reduced in size based on the window cropping range. The window reduction range depends on the window displacement information. For example, if the window is moved 50 pixels to the left, the window length needs to be reduced by 50 pixels to ensure that all pixels within the window are valid pixels. Exemplarily, when the segmentation window is square in shape, the window cropping range may include the window cropping length and cropping width. The window displacement information may include the window movement direction and the window movement distance. Whether the window length or width is to be cropped can be determined based on the window movement direction, and the specific window cropping value can be determined based on the window movement distance. A specific implementation method may be: determining the relative movement distance between the first segmentation window and the second segmentation window in the left-right direction as the segmentation length of the first segmentation window and the second segmentation window, determining the relative movement distance between the first segmentation window and the second segmentation window in the up-down direction as the segmentation width of the first segmentation window and the second segmentation window, and reducing the segmentation length in the left-right direction of the first segmentation window and the second segmentation window according to the determined segmentation length, and reducing the segmentation width in the up-down direction of the first segmentation window and the second segmentation window according to the determined segmentation width, thereby reducing the first segmentation window and the second segmentation window in the left-right direction and the up-down direction, and using the reduced first segmentation window and the second segmentation window to perform image segmentation, so that the segmented left and right eye images do not contain distorted parts caused by displacement, thereby improving the 3D imaging quality. For example, if the first split window and the second split window are shifted by m pixels in the left-right direction and n pixels in the up-down direction, then the first split window and the second split window are reduced by m pixels in the left-right direction and n pixels in the up-down direction, so that the distortion caused by the window displacement can be cropped out, thereby improving the 3D imaging quality.

[0118] It should be understood that after determining the window displacement information, the window can be first displaced according to the window displacement information and then reduced in size; alternatively, the window can be first reduced in size according to the window displacement information and then displaced. Both implementations can achieve the same technical effect, namely, all pixels in the resulting segmented left and right eye images are valid pixels, thereby eliminating the visual impact of the distorted portion of the image on the overall image. This embodiment does not limit this.

[0119] Figure 5 A schematic diagram of a surgical microscope according to another embodiment of the present disclosure is exemplarily shown.

[0120] like Figure 5 As shown, the surgical microscope may include: a binocular camera 510, a processing module 520 and a 3D imaging display device 530.

[0121] The binocular camera 510 is used to capture left-eye images and right-eye images; the processing module 520 is used to implement the method of any embodiment of the present disclosure based on the captured left-eye images and right-eye images; and the 3D imaging display device 530 is used to perform 3D imaging display on the input left-eye images and right-eye images.

[0122] The surgical microscope in the disclosed embodiment uses a 3D imaging display device instead of an eyepiece, allowing the surgeon to perform surgical procedures in front of a screen, shortening surgical time and improving patient survival rates. Alternatively, spectators can obtain real-time 3D images of the surgical procedure through various 3D perspectives, such as 3D glasses or naked-eye 3D. The surgical microscope in the disclosed embodiment uses software control, replacing the traditional manual or mechanical adjustment of the binocular camera installation position. It is simple to operate, highly accurate, and efficient, effectively resolving the problems of low efficiency, poor accuracy, and susceptibility to vibration that exist in existing technologies.

[0123] In the surgical microscope of the disclosed embodiment, the processing module 520 performs window adjustment based on the left-eye image and the right-eye image respectively captured by the binocular camera 510: at least one of the first segmentation window in the left-eye image and the second segmentation window in the right-eye image is displaced, and the first segmentation window and the second segmentation window after the window adjustment are used to segment the left-eye image and the right-eye image respectively, and the segmented left-eye image and right-eye image are both enlarged to the target image size, thereby achieving the adjustment of the relative positions of the left and right eye images, and outputting the enlarged left-eye image and right-eye image in a left-right arrangement format to the 3D imaging display device 530 for presentation of the 3D display effect, so as to achieve the best 3D display effect by adjusting the relative positions of the left and right eye images, avoiding the tediousness and uncertainty of manual operation, and without adding additional mechanical devices, and having the advantages of ease of use, high correction accuracy, time saving, and no interference.

[0124] An embodiment of the present disclosure provides a computer storage medium having a computer program stored thereon. When the program is executed by a processor, the 3D calibration method for a 3D optical device provided in the above embodiment is implemented.

[0125] It should be noted that the computer-readable medium described above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable storage media may include, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wire, optical cable, RF (radio frequency), or any suitable combination thereof.

[0126] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0127] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0128] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0130] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0131] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0132] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0133] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions of this disclosure can be achieved, and this document is not limited here.

[0134] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A 3D optical device 3D calibration method, characterized in that: include: Obtaining left-eye images and right-eye images captured by binocular cameras in a 3D optical device; Based on a preset window, and according to the acquired left-eye image and right-eye image, performing window adjustment, the window adjustment comprising adjusting a distance according to an input left-eye and right-eye parallax, determining a left-right movement distance of at least one of a first segmentation window in the left-eye image and a second segmentation window in the right-eye image, and shifting the at least one window in the left-right direction according to the left-right movement distance; performing a vertical displacement on at least one of the first segmentation window in the left-eye image and the second segmentation window in the right-eye image; Segmenting the left-eye image and the right-eye image according to the first segmentation window and the second segmentation window after the window adjustment to obtain segmented left-eye image and right-eye image; Enlarging the split left-eye image and right-eye image to a target image size, and outputting the enlarged left-eye image and right-eye image to a 3D imaging display device in a left-right arrangement format; According to the formula Determine the effective left-eye and right-eye parallax under the specific parameters of the binocular camera, and determine the input left-eye and right-eye parallax adjustment distance according to the left-eye and right-eye target parallax and the effective left-eye and right-eye parallax; wherein, is the effective parallax of the left and right eyes of the binocular camera, is the baseline length of the binocular camera, is the focal length of the binocular camera, is the working distance of the binocular camera; the left-eye effective parallax is used to characterize the left-eye parallax of the binocular camera under specific parameters, and the left-eye target parallax is the left-eye parallax that needs to be adjusted; The left and right eye target disparities are adjusted according to the adjustment coefficient to update the left and right eye target disparities so that the left and right eye target disparities are within the target disparity standard range; wherein the maximum value and the minimum value in the target disparity standard range are calculated according to the formula and formula And the minimum and maximum field of view angles are determined, where is the field of view angle.

2. The method according to claim 1, characterized in that When the first segmentation window and the second segmentation window are adjusted simultaneously, the first segmentation window and the second segmentation window are symmetrically moved by the same distance in the left-right direction.

3. The method according to claim 1, wherein: The minimum viewing angle is set to 0.5 degrees, and the maximum viewing angle is set to 2 degrees.

4. The method according to claim 1, wherein: The minimum value of the moving distance of the at least one window in the up and down directions is 10 pixels, and the maximum value is 30 pixels.

5. The method according to claim 1, wherein: The moving distance of the at least one window in the up-down direction is determined according to the moving distance of the at least one window in the left-right direction.

6. The method according to claim 5, characterized in that: The moving distance of the at least one window in the up-down direction is a preset multiple of the corresponding displacement distance in the left-right direction, and the minimum value of the preset multiple is 1 / 10 and the maximum value is 1 / 5.

7. The method according to claim 1, wherein: When performing image segmentation, the segmentation range of the preset window is reduced and adjusted according to the at least one window displacement information.

8. The method according to claim 1, wherein: Before making window adjustments, also include: Correction and / or pixel calibration are performed on the acquired left-eye image and right-eye image.

9. A surgical microscope, characterized in that: It includes binocular cameras, processing modules and 3D imaging display equipment; among which, The binocular camera is used to capture left-eye images and right-eye images; The processing module is used to implement the method according to any one of claims 1 to 8 according to the captured left-eye image and right-eye image; The 3D imaging display device is used for performing 3D imaging display on input left-eye images and right-eye images.

Images