3D imaging video recording method and system

Abstract

The invention discloses a 3D imaging video recording method and system, and belongs to the technical field of 3D imaging. The system comprises a dual-camera synchronous acquisition system, a high-precision XY adjusting mechanism, a focal length adjusting function, an aperture adjusting function, a real-time depth-of-field adjusting function, a low-delay coding technology and the like, high-quality 3D operation video recording is realized through the technologies, and depth information of an operation area can be displayed in real time in an operation. The system not only can efficiently record a 3D operation video, but also can support later playback, restores a real operation visual angle through VR / AR equipment, and improves the medical teaching effect. Moreover, the recorded video file is compressed and stored in an efficient coding mode, so that more space is saved in data storage, subsequent archiving, sharing and playback are facilitated, the accuracy, safety and teaching efficiency of the operation are effectively improved, the limitation of traditional operation video is broken through, and the operation experience is improved. And powerful support is provided for real-time navigation, doctor training and operation quality control in an operation.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of 3D imaging technology, and particularly to a 3D imaging video recording method and system. Background Art

[0002] At present, the application of medical imaging technology in surgery has made remarkable progress. Common imaging technologies include 2D imaging systems and 3D imaging systems. Existing 2D imaging systems usually equip microscopes with planar cameras to transmit two-dimensional images. However, this method lacks depth perception, cannot display the three-dimensional spatial structure during surgery, and cannot provide doctors with complete visual information. Although some high-end microscopes achieve limited 3D effects by superimposing multiple focal lengths or dual-path beam splitting, these systems usually require dedicated 3D display devices, and still store in 2D format during recording and cannot record independently.

[0003] With the progress of technology, some improved 3D imaging systems have emerged. These systems can provide multi-dimensional images / videos, but there are still several technical deficiencies. Although some microscopes provide stereoscopic vision, the surgical video will still be compressed into a 2D image during recording, resulting in the lack of key depth information during surgical playback and unable to restore the real surgical perspective; existing 3D video recording systems usually require supporting dedicated display devices, and the video file size is huge, making storage and transmission difficult; most microscope 3D systems cannot be fused with preoperative CT / MRI data in real time, resulting in a deviation between intraoperative images and the actual situation, and re-registration is required, affecting surgical efficiency.

[0004] To make up for these technical deficiencies, the present invention proposes a new 3D imaging video recording method and system. By directly collecting the original 3D video stream through an embedded binocular sensor, it can provide clear, real-time, three-dimensional surgical videos without additional display devices, improving surgical accuracy and teaching effects. Summary of the Invention

[0005] The embodiments of the present application provide a 3D imaging video recording method and system. The technical solutions are as follows:

[0006] According to one aspect of the present application, a 3D imaging video recording system is provided. The system includes:

[0007] A group of dual cameras including camera A and camera B, used for synchronously collecting 3D images;

[0008] A high-precision XY adjustment mechanism, disposed between camera A and camera B, used for precisely adjusting the X-axis and Y-axis positions of each camera to ensure the precise alignment of the binocular parallax baseline;

[0009] A control system for managing the focal length adjustment and aperture control of each camera to ensure real-time adjustment and synchronous acquisition of image data;

[0010] A 3D display unit, including a 3D display and 3D glasses, for presenting the captured 3D images;

[0011] An integrated processor for processing 3D image data for real-time recording and playback.

[0012] Optionally, the dual cameras are equipped with corresponding dual-camera adapters and connected to the microscope, and the dual cameras are fixed below the microscope;

[0013] An XY adjustment knob for finely adjusting the position of the dual cameras. Under the fine adjustment, the centers of the dual cameras are aligned with the cross calibration image.

[0014] Optionally, the dual cameras work synchronously, and the focus of the dual cameras is observed and adjusted through the eyepiece of the microscope;

[0015] The dual cameras support real-time focal length adjustment and plane switching for adjusting different depth-of-field requirements.

[0016] Optionally, the system generates a synchronous pulse signal through FPGA technology, and the synchronous pulse signal is used to ensure that the exposure time difference between camera A and camera B is less than 1 millisecond, thereby avoiding motion blur.

[0017] Optionally, the system is equipped with a high-precision optical module, and the high-precision optical module is configured with a focal length adjustment function and an aperture adjustment function.

[0018] On the other hand, a 3D imaging and recording method is also provided. The method is used for the 3D imaging and recording system according to any one of claims 1 to 6, and the method includes:

[0019] Using the dual cameras to synchronously acquire 3D images, and precisely aligning the parallax baselines of camera A and camera B by observing through the eyepiece of the microscope;

[0020] Using the high-precision XY adjustment mechanism to adjust the position of the dual cameras until the alignment accuracy of the dual cameras reaches ±0.1 mm;

[0021] Adjusting the focal plane required for the surgery through the control system to quickly switch the focus and clearly present different depth-of-field parts during the surgery;

[0022] Adjusting the aperture setting to dynamically adjust the light input amount, and the dynamic adjustment is based on ensuring the image clarity and color restoration under different lighting conditions;

[0023] Adjust the aperture setting through the high-precision optical module equipped in the system to dynamically adjust the light input volume;

[0024] Monitor and adjust the display of the 3D image in real time through the integrated processor, wherein the adjustment of the integrated processor is based on the standard of maintaining the best clarity and depth perception of the images observed during the surgical procedure.

[0025] Optionally, the method further includes:

[0026] During the surgical procedure, according to the needs of the surgery, adjust the depth of field of the dual cameras in real time through the control system, and ensure the clarity of the images through dynamic focal plane switching;

[0027] Adjust the dual cameras in real time through a synchronous pulse signal, and the synchronous pulse signal is used to ensure that the exposure time difference between camera A and camera B is less than 1 millisecond.

[0028] Optionally, the method further includes:

[0029] During the video recording process, through the integrated processor combined with OCT optical coherence tomography or fluorescence angiography data, superimpose the intraoperative real-time imaging and other image data on the 3D image to form comprehensive visual information different from a single imaging modality;

[0030] Utilize low-latency coding technology to ensure that the time delay from image acquisition to display is less than 50 ms, and the 50 ms meets the high-speed operation requirements of microsurgery.

[0031] Optionally, the method further includes:

[0032] Store the imaging video data in an efficient compression format and optimize the data transmission size while storing the video.

[0033] Play back the 3D video recording through a VR / AR device for the analysis and learning of surgical operations.

[0034] On the other hand, a computer-readable storage medium is provided, and the storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the 3D imaging video recording method as described in the above aspect.

[0035] The present application discloses a 3D imaging and video recording method and system, belonging to the field of 3D imaging technology. The system includes a dual-camera synchronous acquisition system, a high-precision XY adjustment mechanism, a focal length adjustment function, an aperture adjustment function, real-time depth of field adjustment, a low-latency encoding technology, etc. Through these technologies, high-quality 3D surgical videos are achieved, and the depth information of the surgical area can be displayed in real time during the surgery. The system can not only efficiently record 3D surgical videos, but also support later playback, restore the real surgical perspective through VR / AR devices, and improve the effect of medical teaching. Moreover, the recorded video files adopt an efficient encoding method for compressed storage, making data storage more space-saving, facilitating subsequent archiving, sharing, and playback, effectively improving the accuracy, safety, and teaching efficiency of surgeries, breaking through the limitations of traditional surgical videos, and providing strong support for real-time navigation during surgeries, doctor training, and surgical quality control. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Shows a schematic plan view of the 3D imaging and video recording system;

[0037] Figure 2 Corresponding to Figure 1 Side schematic view;

[0038] Figure 3 Is a schematic solid view corresponding to 1;

[0039] Figure 4 Corresponding to Figure 2 Solid schematic view;

[0040] Figure 5 And Figure 6 Are actual scene pictures of the implementation scenario applied to a surgical microscope. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0042] As used herein, "a plurality of" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0043] Embodiment 1

[0044] A 3D imaging and video recording system is provided, and the system includes:

[0045] A group of dual cameras including Camera A and Camera B for synchronously acquiring 3D images;

[0046] A high-precision XY adjustment mechanism, disposed between the camera A and the camera B, is used to precisely adjust the X-axis and Y-axis positions of each camera to ensure the precise alignment of the binocular parallax baseline;

[0047] A control system, used to manage the focal length adjustment and aperture control of each camera, ensures real-time adjustment and synchronous acquisition of image data;

[0048] A 3D display unit, including a 3D monitor and 3D glasses, is used to present the captured 3D images;

[0049] An integrated processor, used to process 3D image data for real-time video recording and playback.

[0050] As Figure 1 and 2 shown, Figure 1 shows a schematic plan view of the 3D imaging and video recording system, Figure 2 which is a schematic side view corresponding to Figure 1 . Figure 1 The positions of the camera A and the camera B and their structural schematics are marked at the position, where the mark 02 is the aperture adjustment position, Figure 2 in which the mark 03 is the X-axis and Y-axis adjustment points of the example camera A. In addition, Figure 2 the mark 01 in it indicates the focal length adjustment point of the example camera A.

[0051] Correspondingly, Figure 3 is a schematic diagram of the entity corresponding to 1, Figure 4 which is a schematic diagram of the entity corresponding to Figure 2 .

[0052] Thus, it can be seen that the system synchronously acquires 3D images by using dual cameras (camera A and camera B), and the dual cameras are precisely adjusted relative to the surgical area. The adjustment of the cameras is achieved through the XY adjustment mechanism (horizontal and vertical positions) to ensure that each camera is aligned with the target and guarantees an accuracy of ±0.1 mm. The system further includes an integrated processor, which is used to process the acquired image data and convert it into 3D images in real time for surgical personnel or trainees to view through a 3D monitor and 3D glasses.

[0053] By adopting a high-precision dual-camera synchronous acquisition system, it can provide real-time 3D image display, making the depth information clearly visible during the operation, effectively solving the problem that the traditional 2D imaging system cannot present a sense of depth, and improving the surgical precision and teaching efficiency.

[0054] Optionally, the dual cameras are equipped with corresponding dual-camera adapters and connected to the microscope. The dual cameras are fixed below the microscope. The XY adjustment knobs are used to finely adjust the positions of the dual cameras. Under the fine adjustment, the centers of the dual cameras are aligned with the cross-calibration images.

[0055] As Figure 5 and Figure 6 shown, it is a real-scene picture of the implementation scenario applied to the surgical microscope. In a possible implementation manner, the system precisely fixes the cameras through the dual-camera adapters installed on the microscope and uses the XY adjustment knobs for fine-tuning. The image center during the adjustment process is aligned with the cross-calibration images to ensure the alignment accuracy of the system. By adjusting the positions of the cameras, the images are ensured to be aligned and the misalignment problem is avoided.

[0056] The introduction of the XY adjustment mechanism enables the dual cameras to be precisely aligned, eliminates the image misalignment that may be caused by mechanical installation errors, ensures the accuracy of the 3D images, and improves the reliability of intraoperative navigation.

[0057] Optionally, the dual cameras work synchronously. The foci of the dual cameras are observed and adjusted through the eyepiece of the microscope.

[0058] The dual cameras support real-time adjustment of the focal length and plane switching to adjust for different depth-of-field requirements.

[0059] In a possible implementation manner, first, the dual cameras work synchronously under the microscope and adjust the XY positions to ensure alignment. Secondly, adjust the focal length to meet the requirements of different depth regions during the operation. Then, adjust the aperture according to the ambient light conditions to optimize the image quality. Finally, monitor and adjust the display of the 3D images in real time to ensure a stable and clear display effect.

[0060] This method realizes the real-time adjustment of the 3D images during the operation, ensuring that doctors can always obtain a clear field of view under different lighting and depth conditions. By quickly adjusting the focal length and aperture, it reduces visual fatigue during the operation and improves the comfort and accuracy of the operation.

[0061] Optionally, the system generates a synchronous pulse signal through FPGA technology. The synchronous pulse signal is used to ensure that the exposure time difference between camera A and camera B is less than 1 millisecond, thereby avoiding motion blur.

[0062] Optionally, the system is equipped with a high-precision optical module. The high-precision optical module is configured with a focal length adjustment function and an aperture adjustment function.

[0063] The system generates a synchronous pulse signal through FPGA technology to ensure that the exposure time difference between dual cameras A and B is less than 1 millisecond. In this way, even in fast dynamic scenarios, the synchronization between the cameras can be guaranteed, avoiding motion blur.

[0064] By precisely synchronizing the exposure times of the dual cameras, it is ensured that even during dynamic surgical procedures, the images can remain clear, avoiding image deviations caused by different exposure times and improving the quality of 3D surgical videos.

[0065] Embodiment 2

[0066] A 3D imaging video recording method is also provided. The method is used for the 3D imaging video recording system according to any one of claims 1 to 6. The method includes:

[0067] Synchronously acquiring 3D images using the dual cameras, and precisely aligning the parallax baselines of camera A and camera B by observing through the eyepiece of the microscope;

[0068] Adjusting the positions of the dual cameras using the high-precision XY adjustment mechanism until the alignment accuracy of the dual cameras reaches ±0.1 mm;

[0069] Adjusting the focal plane required for the surgery through the control system to quickly switch the focus and clearly present parts with different depths of field during the surgery;

[0070] Adjusting the aperture setting to dynamically adjust the light input. The dynamic adjustment is based on ensuring image clarity and color restoration under different lighting conditions;

[0071] Adjusting the aperture setting to dynamically adjust the light input through the high-precision optical module equipped in the system;

[0072] Real-time monitoring and adjusting the display of the 3D images through the integrated processor. Among them, the adjustment of the integrated processor is based on keeping the images observed during the surgical process in the best clarity and depth perception in real time.

[0073] In one example, first, synchronously acquire 3D images through the dual cameras, and adjust the XY positions of the cameras to ensure alignment; second, adjust the focal plane according to the requirements of the surgical area through the focal length adjustment function; then, adjust the aperture setting to optimize the image effects under different lighting environments; finally, ensure that the 3D images are stably displayed on the 3D monitor in real time.

[0074] Through this method, doctors can obtain 3D images in real time during the surgery, precisely control the focal length and aperture, improving visual accuracy. At the same time, real-time adjustment and stable display keep the images during the surgical process clear, enhancing the doctor's operation perception and decision-making ability.

[0075] Optionally, the method further includes:

[0076] During the operation, according to the needs of the operation, the depth of field of the dual cameras is adjusted in real time through the control system, and the clarity of the image is ensured by switching the dynamic focal plane;

[0077] The dual cameras are adjusted in real time synchronization through a synchronization pulse signal, and the synchronization pulse signal is used to ensure that the exposure time difference between camera A and camera B is less than 1 millisecond.

[0078] During the operation, the doctor quickly adjusts the depth of field according to different requirements of the operation area. For example, in ophthalmic surgery, the system can quickly switch the focal length to adapt to different depths of field between the cornea and the retina, and at the same time maintain high-precision image display during retinal surgery.

[0079] This dynamic depth of field adjustment method enables the system to adapt to the rapid changes during the operation, ensuring that the doctor always obtains clear and accurate 3D images during the rapid focus switching process, significantly improving the accuracy and fluency of the surgical operation.

[0080] Optionally, the method further includes:

[0081] During the video recording process, through the integrated processor, combining OCT (Optical Coherence Tomography) or fluorescence angiography data, the intraoperative real-time imaging is superimposed with other imaging data and displayed in the 3D image to form comprehensive visual information different from a single imaging modality; using low-latency encoding technology to ensure that the time delay from image acquisition to display is less than 50 ms, and the 50 ms meets the high-speed operation requirements of microsurgery.

[0082] This method supports the real-time superimposition of OCT (Optical Coherence Tomography) or fluorescence angiography data during the operation, and can fuse the data of different imaging modalities into the 3D view in real time, providing more comprehensive visual information of the surgical area.

[0083] Through this dynamic multi-modal data fusion, the doctor can obtain more information about tissues, blood flow and depth based on the real-time 3D image, enhancing the understanding and judgment of complex surgical procedures and improving the accuracy and safety of the surgery.

[0084] Optionally, the method further includes:

[0085] The imaging video data is stored in an efficient compression format, and the data transmission size is optimized while the video is stored; the 3D video is played back through VR / AR devices for analysis and learning of surgical operations.

[0086] During the surgical video recording process, all image data is compressed in the efficient H.265 encoding format to ensure that the size of the 4K / 3D video file per hour is less than 50GB. Moreover, these video files can be directly stored in the hospital's PACS system for convenient later playback and analysis.

[0087] Through efficient compression technology, the video files can not only greatly save storage space but also be compatible with different platforms such as PCs and VR devices, facilitating doctors and trainees to review the surgical process for analysis and learning, while also improving the efficiency of data management and storage.

[0088] The embodiment of the present application also provides a computer-readable medium that stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the 3D imaging video recording method described in each of the above embodiments.

[0089] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A 3D imaging and video recording system, characterized in that, The system includes: A pair of dual cameras including camera A and camera B, which are used to synchronously capture 3D images; A high-precision XY adjustment mechanism, which is arranged between camera A and camera B, and is used to accurately adjust the X-axis and Y-axis positions of each camera; A control system, which is used to manage the focal length adjustment and aperture control of each camera; A 3D display unit, which includes a 3D monitor and 3D glasses, and is used to present the captured 3D images; An integrated processor, which is used to process 3D image data for real-time recording and playback.

2. The system according to claim 1, wherein The dual cameras are equipped with corresponding dual-camera adapters and are connected to the microscope. The dual cameras are fixed under the microscope; An XY adjustment knob, which is used to finely adjust the position of the dual cameras. Under the fine adjustment, the centers of the dual cameras are aligned with the cross calibration image.

3. The system according to claim 1, wherein The dual cameras work synchronously, and the foci of the dual cameras are observed and adjusted through the eyepiece of the microscope; The dual cameras support real-time adjustment of the focal length and plane switching to adjust for different depth-of-field requirements.

4. The system according to claim 1, wherein The system generates a synchronous pulse signal through FPGA technology. The synchronous pulse signal is used to ensure that the exposure time difference between camera A and camera B is less than 1 millisecond, so as to avoid motion blur.

5. The system according to any one of claims 1 to 4, characterized in that, The system is equipped with a high-precision optical module, and the high-precision optical module is configured with a focal length adjustment function and an aperture adjustment function.

6. A 3D imaging and video recording method, characterized in that, The method is used for the 3D imaging and recording system according to any one of claims 1 to 5. The method includes: Using the dual cameras to synchronously capture 3D images, and accurately aligning the parallax baselines of camera A and camera B by observing through the eyepiece of the microscope; Using the high-precision XY adjustment mechanism to adjust the positions of the dual cameras until the alignment accuracy of the dual cameras reaches ±0.1 mm; Adjusting the focal plane required for the surgery through the control system to quickly switch the focus and clearly present the parts with different depths of field during the surgery; Adjusting the aperture setting to dynamically adjust the light input amount. The dynamic adjustment is based on ensuring the image clarity and color restoration under different lighting conditions; Adjusting the aperture setting through the high-precision optical module equipped with the system to dynamically adjust the light input amount; Real-time monitoring and adjusting the display of 3D images through the integrated processor. Among them, the adjustment of the integrated processor is based on the standard of maintaining the best clarity and depth perception for the images observed during the surgical process.

7. The method according to claim 6, wherein The method further includes: During the surgery, according to the needs of the surgery, the control system is used to adjust the depth of field of the dual cameras in real time, and the clarity of the image is ensured through dynamic focal plane switching; Real-time synchronously adjusting the dual cameras through the synchronous pulse signal. The synchronous pulse signal is used to ensure that the exposure time difference between camera A and camera B is less than 1 millisecond.

8. The method according to claim 6, characterized in that The method further includes: During the recording process, through the integrated processor, combining OCT optical coherence tomography or fluorescence angiography data, the intraoperative real-time imaging and other image data are superimposed and displayed in the 3D image to form comprehensive visual information different from a single imaging modality. Using low-latency encoding technology to ensure that the time delay from image acquisition to display is less than 50 ms, and the 50 ms meets the high-speed operation requirements of microsurgery.

9. The method according to claim 6, wherein The method further includes: Storing the imaging video data in an efficient compression format and optimizing the data transmission size while storing the video; Playing back the 3D video through VR / AR devices for analysis and learning of surgical operations.

Images