An intelligent surgical exoscopic system

Abstract

The present invention discloses an intelligent surgical exoscopic system, comprising a work trolley, a pneumatic robotic arm, an intelligent multi-optical imaging system located at the end of the pneumatic robotic arm, a 3D display device, and auxiliary control equipment; the work trolley comprises a housing, a display rotating bracket, a four-star base, and universal casters; the pneumatic robotic arm comprises a pneumatic mechanical arm, a pneumatic mechanical arm, a pneumatic camera arm, and a spherical joint; the intelligent multi-optical imaging system comprises multiple integrated zoom imaging optical paths, an intelligent focus controller, and a ring integrated chip; the 3D display device comprises a polarized 3D main screen, a polarized 3D secondary screen, and a head-mounted display device; the head-mounted display device comprises VR glasses and AR glasses; and the auxiliary control device comprises a pressure-sensitive foot pedal. The present invention meets the needs of modern surgical operations, providing more intelligent services, more efficient signal transmission, and a more comfortable user experience.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical devices and relates to an intelligent surgical exoscopic system. Background Art

[0002] In recent years, with the rapid development of medical imaging technology, surgical procedures have gradually moved towards minimally invasive and precision surgery. As an important auxiliary tool, exoscopic systems have played a vital role in minimally invasive surgery, open surgery, and remote collaboration. Exoscopic systems use cameras to transmit real-time images of the surgical area to a monitor for the doctor to observe and operate. This allows doctors to perform precise surgical procedures with minimal exposure to the patient's internal tissues, thereby reducing surgical trauma, shortening postoperative recovery time, and improving surgical outcomes.

[0003] While a wide variety of surgical exoscopic systems are currently available on the market, some significant challenges remain. For example, exoscopic systems lack intelligence and convenience, often lacking multimodal imaging and multifunctional sensing capabilities, and their lenses cannot automatically adjust their focus. During surgery, the obstruction of the shadowless lamp significantly interferes with surgical imaging, impacting surgical quality. Furthermore, information transmission methods are complex and susceptible to interference during transmission, leading to display delays or reduced image quality.

[0004] Therefore, there is an urgent need for a new surgical exoscopic system to address the shortcomings of existing technologies, provide more intelligent services, more efficient signal transmission, and more comfortable user experience to meet the needs of modern surgical operations. Summary of the Invention

[0005] Existing exoscopic devices lack intelligence and convenience, often lacking multimodal imaging and multifunctional sensing capabilities, and their lenses cannot automatically adjust their focus. During surgery, shadowless lamps can significantly interfere with surgical imaging, impacting surgical quality. Information transmission methods are complex and susceptible to interference during transmission, leading to display delays or image quality degradation. To address these issues, the present invention discloses an intelligent surgical exoscopic system that meets the needs of modern surgical procedures, providing more intelligent services, more efficient signal transmission, and a more comfortable user experience.

[0006] The specific plan is as follows:

[0007] An intelligent surgical exoscope system, characterized in that it includes a work trolley, a pneumatic robotic arm, an intelligent multi-optical path imaging system, a 3D display device and an auxiliary control device arranged at the end of the pneumatic robotic arm; the work trolley includes a box body, a display rotating bracket, a four-star foot base and a universal caster, wherein the display rotating bracket and the four-star foot base are respectively installed at the top and bottom of the box body, and the universal caster is cooperatively installed at the four top corners of the four-star foot base; the pneumatic robotic arm includes a pneumatic robotic arm, a pneumatic robotic arm and a pneumatic camera arm, wherein the lower end of the pneumatic robotic arm is installed on the box body, the pneumatic robotic arm has two sections, one of which is connected to the upper end of the pneumatic robotic arm through a big arm spherical joint, the two sections and the other section and the pneumatic camera arm are connected through the small arm spherical joint, and the pneumatic camera arm is connected to the pneumatic camera arm through the small arm spherical joint. The arm is connected to the intelligent multi-path imaging system through a branch-arm spherical joint; the intelligent multi-path imaging system includes multiple integrated zoom imaging optical paths, an intelligent focus controller and a ring-shaped integrated chip; the ring-shaped integrated chip is installed on the periphery of the multiple zoom imaging optical paths, and integrates thermal imaging sensors, gas sensors, Raman spectroscopy sensors and temperature and humidity sensors, and is peripherally connected to a miniature ultrasonic probe and a miniature fiber optic probe; the 3D display device includes a polarized 3D main screen, a polarized 3D secondary screen and a head-mounted display device; the polarized 3D main screen and the polarized 3D secondary screen are connected to the box through a display rotating bracket and can be rotated on a fixed axis; the head-mounted display device includes VR glasses and AR glasses; the auxiliary control device includes a pressure-sensitive foot pedal for controlling the orientation and height of the intelligent multi-path imaging system.

[0008] As a further improvement of the present invention, the front end of the interior of the box is provided with a robotic arm control system, an image processing host and a light source control system, the rear end of the interior of the box is provided with a drawer for storing medical equipment, etc., and the outside of the box is provided with an operation screen and heat dissipation holes. The operation screen is used to start and shut down the exterior mirror system, manually adjust the pneumatic robotic arm when necessary, set the output mode of the image processing host, and control the imaging mode; the heat dissipation holes are used to dissipate heat to the robotic arm control system, the image processing host and the light source control system to prevent the exterior mirror from overheating and causing danger; the robotic arm control system is used to provide motion control, posture adjustment, automatic positioning, balance control, safety functions, braking functions, sensor feedback and integration functions.

[0009] As a further improvement of the present invention, the image processing host is used to process the image information obtained by the camera system and output it in the output mode required by the user. The output mode includes a naked-eye 3D mode and an HMD mode. The naked-eye 3D mode is suitable for a polarized 3D main screen and a polarized 3D secondary screen, and the HMD mode is suitable for a head-mounted display device.

[0010] As a further improvement of the present invention, the image processing host provides intraoperative navigation function and path planning function, wherein the specific implementation method of the intraoperative navigation function is: using pre-acquired medical imaging data to construct a three-dimensional model of the patient's anatomical structure, aligning the real-time position and posture of the exoscope with the three-dimensional model, realizing real-time positioning of the exoscope in the patient's body, displaying the current position and field of view of the exoscope on the three-dimensional model, and providing intraoperative navigation for the doctor; the specific implementation method of the path planning function is: according to the surgical goals and anatomical structures, automatically generating the optimal path for the exoscope to reach the target site, combining with the navigation system, guiding the exoscope to move along the planned path, avoiding key blood vessels and nerve tissues.

[0011] As a further improvement of the present invention, the image processing host also solves the problem of decreased brightness of the surgical field caused by the obstruction of the shadowless lamp during surgery, and comprehensively utilizes a brightness compensation model based on deep learning, multi-scale analysis and edge preservation technology. The specific implementation method is: training a convolutional neural network CNN for surgical field brightness compensation, using occluded and non-occluded surgical field image data for learning, so that it can automatically estimate and compensate for local brightness; applying the CNN model to surgical field images of multiple scales, and using multi-scale fusion technology to obtain clearer and more natural enhancement results; at the same time, introducing an edge-aware loss function in the CNN model, and applying an edge-preserving filter in the post-processing stage to ensure that the enhanced image can retain key structural details and avoid image distortion or artifacts.

[0012] As a further improvement of the present invention, the image processing host also solves the problem of decreased brightness of the surgical field caused by the obstruction of the shadowless lamp during surgery. The specific implementation method is as follows: training a CNN model for surgical field brightness compensation, which is based on the existing image enhancement network structure, such as U-Net, ResNet, etc., and is adjusted according to actual needs. For example, the network depth can be increased, the convolution kernel size can be adjusted, etc., to improve the learning ability and generalization ability of the model; the CNN model is applied to surgical field images of multiple scales, and multi-scale fusion technology is used to obtain clearer and more natural enhancement results; at the same time, an edge-aware loss function is introduced, and an edge-preserving filter is applied in the post-processing stage to ensure that the enhanced image can retain key structural details and avoid image distortion or artifacts;

[0013] The loss function is defined as:

[0014] L = L_mse + λ × L_edge,

[0015] Where L_mse is the mean square error loss function, which is used to measure the pixel-level difference between the enhanced image and the real image; L_edge is the edge loss function, which is used to measure the difference in edge information between the enhanced image and the real image; the Sobel operator or the Canny operator is used to extract the image edge and calculate the difference in edge pixels; λ is the weight coefficient used to balance the importance of the two loss functions;

[0016] Where L_mse is defined as:

[0017] L_mse=1 / N×Σ(I_e-I_gt) 2 ,

[0018] Where I_e is the enhanced image, I_gt is the real image, and N is the total number of pixels;

[0019] L_edge is defined as:

[0020] L_edge=1 / M×Σ(E_e-E_gt) 2 ,

[0021] Among them, E_e is the edge information of the enhanced image, E_gt is the edge information of the real image, and M is the total number of edge pixels.

[0022] As a further improvement of the present invention, the light source control system provides light source brightness adjustment, spectrum adjustment, light source stability control, and uniform illumination detection functions; wherein, the spectrum adjustment function of the light source control system is used for multimodal imaging; multimodal imaging includes white light imaging, ICG fluorescence imaging, and narrow-band imaging; the light source control system is connected to the zoom imaging optical path through an optical fiber, and there are multiple optical fibers for transmitting light of different wavelengths to the front end of the exoscope.

[0023] As a further improvement of the present invention, the multiple zoom imaging optical paths integrated in the intelligent multi-optical path imaging system are used to perform multi-viewpoint 3D imaging and multimodal imaging simultaneously; the two imaging optical paths can respectively simulate the imaging systems of the left and right eyes of the human body, and simultaneously collect two sets of images of the same surgical field. After the two sets of images are three-dimensionally synthesized by the image processing host, they are output to the corresponding 3D display device; the 3D display devices each correspond to two of the imaging optical paths, and the remaining optical paths are used for multimodal imaging; the multiple zoom imaging optical paths are distributed in a circular array around the central axis of the intelligent multi-optical path imaging system, meeting an optical zoom ratio of more than 10 times, and having automatic focus and viewing angle adjustment functions; the intelligent focus controller adopts a rotary button design, and the specific implementation method is as follows: the button can be pressed and rotated, pressing is used to switch between automatic focus and manual focus mode, and rotating is used for manual focus, accompanied by sound feedback to inform the operator of the focusing status.

[0024] As a further improvement of the present invention, the thermal imaging sensor is used to help the surgeon perform tissue perfusion assessment, tumor or inflammation detection, and the results are superimposed in real time on the polarized 3D main screen or presented separately on the polarized 3D secondary screen; the gas sensor is used to detect changes in gas composition in the surgical area, which helps to detect complications early and can perform tissue metabolism monitoring, hypoxia detection, infection or necrosis indication; the Raman spectroscopy sensor provides the surgeon with real-time tissue composition analysis capabilities non-invasively, and the sensor is equipped with a small high-sensitivity spectrometer and a sensor data processing unit, and the results are superimposed in real time on the polarized 3D main screen or presented separately on the polarized 3D secondary screen; the temperature and humidity sensor is used to monitor the temperature and humidity of the surgical environment.

[0025] As a further improvement of the present invention, the micro-ultrasound probe is a miniaturized ultrasonic imaging device that provides the surgeon with real-time deep tissue structure information, assists in surgical navigation, and improves its accuracy.

[0026] As a further improvement of the present invention, the transmission method of the 3D scene information obtained after processing by the image processing host and the 3D display device, and the information transmission method of the pressure-sensitive foot pedal and the robotic arm control system are all 5G communication wireless transmission; the pressure-sensitive foot pedal is composed of a base, a pedal, and a fixed block. It is based on a pressure sensor and a tilt sensor, and there are two on the left and right. The orientation and height of the intelligent multi-path imaging system are controlled by stepping in different ways. The fixed block is used to fix or release the position of the intelligent multi-path imaging system; the operation screen can help the operator customize different functional combinations of the pedals.

[0027] The beneficial effects of the present invention are:

[0028] 1. It realizes the intraoperative navigation and path planning functions, providing surgeons and patients with more intelligent services and comfortable user experience.

[0029] 2. It can effectively solve the problem of decreased brightness of the surgical field caused by shadowless lamp obstruction, provide the surgeon with a clear, natural and detail-rich surgical field view, and improve the quality and efficiency of surgery.

[0030] 3. It provides multimodal imaging function, which collects surgical field information through the integration of multiple imaging optical paths and performs imaging of other modalities, helping surgeons to make better judgments and facilitate multi-surgeon collaborative surgery.

[0031] 4. Combining the advantages of manual and automatic zoom to accommodate special situations or personal preferences, the zoom function has preset magnifications but also allows the surgeon to manually fine-tune it. Combined with a highly integrated ring-shaped integrated chip, this solution enables multifunctional sensing in a very small space, providing comprehensive tissue and environmental information for surgical exoscopes while maintaining the compactness and operational flexibility of the device.

[0032] 5. Both the 3D display device and the pressure-sensitive foot pedal use 5G wireless transmission to transmit information, which simplifies the equipment layout, facilitates equipment expansion and upgrades, improves safety and flexibility, and further meets the needs of modern surgical operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a schematic diagram of the three-dimensional structure of an intelligent surgical exoscopic system of the present invention.

[0034] Figure 2 This is a side structural diagram of an intelligent surgical exoscopic system of the present invention.

[0035] Figure 3 This is a schematic diagram of the rear structure of an intelligent surgical exoscopic system of the present invention.

[0036] Figure 4 Schematic diagram of a multi-viewpoint 3D imaging method of an intelligent multi-light-path imaging system according to an embodiment of the present invention.

[0037] Figure 5 This is a schematic structural diagram of a pressure-sensitive foot pedal of an intelligent surgical exoscopic system of the present invention.

[0038] Figure 6 This is a schematic diagram of a stacking of a ring integrated chip according to an embodiment of the present invention, where arrows represent the direction of data transmission.

[0039] List of reference numerals:

[0040] 1. Universal casters; 2. Four-star foot base; 3. Box; 4. Heat dissipation holes; 5. Operation screen; 6.1. Display rotation bracket; 7. Pneumatic mechanical arm; 8. Pneumatic mechanical arm; 9.1. Arm spherical joint; 9.2. Arm spherical joint; 9.3. Branch arm spherical joint; 10. Pneumatic camera branch arm; 11. Intelligent multi-optical path imaging system; 12. Intelligent focus controller; 13. Polarized 3D main screen; 14. Polarized 3D secondary screen; 15. Head-mounted display device; 16. Drawer; 17.1. Zoom imaging optical path; 18. Ring integrated chip; 19.1. Base; 20.1. Pedal; 21.1. Fixing block; 22. System on chip; 23. Thermal imaging sensor layer; 24. Gas sensor layer; 25. Raman spectroscopy sensor layer; 26. Temperature and humidity sensor layer. DETAILED DESCRIPTION

[0041] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0042] The present invention provides an intelligent surgical exoscopic system, such as Figure 1 、2 As shown in Figure 3, it includes: a workbench, a pneumatic robotic arm, an intelligent multi-optical imaging system 11 located at the end of the pneumatic robotic arm, and a 3D display device. The workbench includes a housing 3, a display rotating bracket 6.1, a four-star base 2, and universal casters 1. The pneumatic robotic arm includes a pneumatic robotic arm 7, a pneumatic robotic arm 8, a pneumatic camera arm 10, and spherical joints. The spherical joints include a main arm spherical joint 9.1; a lower arm spherical joint 9.2; and a sub-arm spherical joint 9.3.

[0043] Specifically, the pneumatic manipulator arm 7 in this embodiment is fixed to the housing 3 and connected to the pneumatic manipulator arm 8 via a main arm spherical joint 9.1. The two pneumatic manipulator arms 8 and the pneumatic camera arm 10 are interconnected via a lower arm spherical joint 9.2. The pneumatic camera arm 10 is connected to the camera system via a sub-arm spherical joint 9.3. The spherical joints in this embodiment can rotate about three mutually perpendicular axes, providing three degrees of freedom. Therefore, the pneumatic manipulator arm in this embodiment is flexible and has a wide range of motion, with a maximum arm span of 1600 mm.

[0044] Specifically, the intelligent multi-light path imaging system 11 includes five integrated zoom imaging light paths 17.1, an intelligent focus controller 12, and a ring integrated chip 18. Figure 4 As shown, the five zoom imaging optical paths 17.1 in this embodiment are distributed in a circular array around the central axis of the intelligent multi-optical path imaging system 11, with a resolution of 3840×2160, an optical zoom ratio of 13 times, and automatic focus and viewing angle adjustment functions.

[0045] Specifically, the ring-shaped integrated chip 18 is mounted on the periphery of the five zoom imaging optical paths 17.1. It has a total height of 6mm, an inner ring radius of 17mm, and an outer ring radius of 22mm. It is made of four MEMS microsensors: a thermal imaging sensor, a gas sensor, a Raman spectrometer sensor, and a temperature and humidity sensor, stacked vertically and compactly. Through-silicon via (TSV) technology is used to achieve inter-layer interconnection. The periphery is connected to a micro-ultrasound probe and a micro-fiber probe via a flexible printed circuit. TSV technology enables high-density, low-latency electrical connections between the layers of the ring-shaped integrated chip 18, while significantly reducing horizontal space usage.

[0046] Specifically, the stacking diagram of the ring integrated chip 18 is as follows: Figure 6 As shown, the specific configuration is as follows: a high-performance system on chip (SoC) 22 is set on the top layer, which integrates a signal processing unit, an analog-to-digital converter and a communication module to realize real-time data processing and transmission; a thermal imaging sensor layer 23, a gas sensor layer 24, a Raman spectrum sensor layer 25 and a temperature and humidity sensor layer 26 are sequentially set on the lower layer; the arrow represents the direction of data transmission, and the data is transmitted to the surgical exoscope body.

[0047] The 3D display device includes a polarized 3D main screen 13, a polarized 3D secondary screen 14, and a head-mounted display device (HMD) 15; the polarized 3D display main screen 13 and the secondary screen 14 are connected to the box 3 through the display rotating bracket 6.1, and can be rotated on a fixed axis to facilitate adjustment according to the operator's orientation.

[0048] The head mounted display device 15 includes VR glasses and AR glasses. In the embodiment, the size of the polarized 3D display main screen 13 is 24 inches, and the size of the polarized 3D display sub-screen 14 is 12 inches.

[0049] The auxiliary control device includes a pressure-sensitive foot pedal.

[0050] Specifically, the pressure-sensitive foot pedal is composed of a base 19.1, a pedal 20.1, and a fixing block 21.1, and there are two left and right pedals. Figure 5 The pressure sensor and tilt sensor are used to control the position and height of the intelligent multi-path imaging system. Specifically, the following methods are used: stepping on the front of pedal 20.1 moves forward; stepping on the rear of pedal 20.1 moves backward; stepping on the left side of pedal 20.1 moves left; stepping on the right side of pedal 20.1 moves right; stepping on the center adjusts the height; stepping on both feet zooms in or out; and controlling the movement speed by tilting the feet. Stepping on fixing block 21.1 fixes the position of intelligent multi-path imaging system 11; lifting fixing block 21.1 releases the fixation. The operation screen 5 allows the operator to customize different function combinations.

[0051] The front end of the box body 3 is provided with a robotic arm control system, an image processing host, and a light source control system, and the rear end of the box body 3 is provided with a drawer 14 for storing medical equipment, etc. The outside of the box body 3 is provided with an operation screen 5 and heat dissipation holes 4.

[0052] The robotic arm control system can provide motion control, posture adjustment, automatic positioning, balance control, safety functions, braking functions, sensor feedback, and integration functions.

[0053] Specifically, the image processing host is mainly used to process the image information obtained by the intelligent multi-optical path imaging system 11 and output it in the output mode required by the user. The output mode includes a naked eye 3D mode and an HMD mode. The naked eye 3D mode is suitable for the polarized 3D main screen 13 and the auxiliary screen 14. The HMD mode is suitable for the head-mounted display device 15, such as Figure 4 As shown, in this embodiment, the image processing host performs simultaneous output of naked eye 3D mode and HMD mode.

[0054] The image processing host can provide intraoperative navigation. Specifically, this involves constructing a three-dimensional model of the patient's anatomical structure using pre-acquired medical imaging data (such as CT and MRI), registering the exoscope's real-time position and posture with the three-dimensional model, and achieving real-time positioning of the exoscope within the patient's body. The exoscope's current position and field of view are then displayed on the three-dimensional model, providing intraoperative navigation for the doctor.

[0055] The image processing host can provide a path planning function. Specifically, based on the surgical objectives and anatomical structures, the intelligent multi-path imaging system 11 automatically generates an optimal path to the target site. Combined with the navigation system, it guides the system along the planned path, avoiding critical blood vessels and nerve tissue. Path optimization can be performed based on factors such as surgical safety and operation time.

[0056] Specifically, the image processing host can also comprehensively utilize a deep learning-based brightness compensation model, multi-scale analysis, and edge-preserving technology to solve the problem of decreased brightness in the surgical field caused by the obstruction of the shadowless lamp during surgery. The specific implementation method is as follows: training a convolutional neural network (CNN) specifically for surgical field brightness compensation, using a large amount of occluded and non-occluded surgical field image data for learning, so that it can automatically estimate and compensate for local brightness; applying the CNN model to surgical field images of multiple scales, and using multi-scale fusion technology to obtain clearer and more natural enhancement results; at the same time, we introduced an edge-aware loss function in the CNN model, and applied an edge-preserving filter in the post-processing stage to ensure that the enhanced image can retain key structural details and avoid image distortion or artifacts.

[0057] The light source control system can provide light source brightness adjustment, spectrum adjustment, light source stability control, and uniform lighting detection functions.

[0058] Specifically, the spectral adjustment function of the light source control system is used for multimodal imaging; such multimodal imaging includes white light imaging, ICG fluorescence imaging, and narrowband imaging. The intelligent multi-optical path imaging system 11 in this embodiment integrates five zoom imaging optical paths 17.1. When the use of the 3D display device is reduced, the zoom imaging optical path 17.1 can be reserved for multimodal imaging.

[0059] Specifically, the light source control system is connected to the zoom imaging optical path 17.1 via an optical fiber. In this embodiment, each of the five imaging optical paths 17.1 is provided with an optical fiber connected to the light source control system for light source transmission to meet the needs of multimodal imaging.

[0060] The operation screen 5 is used to start and shut down the exterior mirror system, manually adjust the pneumatic robotic arm when necessary, set the output mode of the image processing host, and control the imaging mode.

[0061] Specifically, the five zoom imaging optical paths 17.1 integrated in the intelligent multi-optical path imaging system 11 are used to simultaneously perform multi-viewpoint 3D imaging and multi-modal imaging. Figure 4 This diagram illustrates a multi-viewpoint 3D imaging method for an intelligent multi-optical imaging system 11 in an embodiment. Five 3D display devices capture 3D scene information from different perspectives. Any two zoom imaging optical paths 15.1 within the intelligent multi-optical imaging system 11 are selected. These two optical paths can respectively simulate the imaging systems of the left and right eyes of the human body, simultaneously capturing two sets of images of the same surgical field. These two sets of images are then synthesized in three dimensions by the image processing host and output in glasses-free 3D mode to a polarized 3D main screen 13 and a secondary screen 14, or in HMD mode to a head-mounted display device 15. The choice of perspective depends on the position of the surgeon. Preferably, adjacent zoom imaging optical paths 17.1 are output to a single 3D display device, as shown. In this case, four surgeons can perform surgery using the head-mounted display device 15 while one surgeon performs surgery using the polarized 3D main screen 13. When performing multimodal imaging, the use of 2 to 4 head-mounted display devices 15 can be reduced accordingly, and the 1 to 3 vacant zoom imaging optical paths 17 . 1 can be used for imaging in other modalities respectively.

[0062] Specifically, the 3D scene information obtained after processing by the image processing host and the information transmission method between the 3D display device, the pressure-sensitive foot pedal and the robotic arm control system is 5G wireless transmission. Because the transmitted information involves the privacy data of multiple parties, a specific medical channel will be selected for encrypted transmission during transmission, while ensuring high frame rate and high-quality transmission.

[0063] Specifically, the intelligent focusing controller 12 adopts a rotary button design, and the specific implementation method is: the button can be pressed and rotated, pressing is used to switch between automatic focus and manual focus mode, and rotating is used for manual focus, accompanied by sound feedback to inform the operator of the focus status.

[0064] Specifically, the thermal imaging sensor is used to help the surgeon perform tissue perfusion assessment, tumor or inflammation detection, etc. Its specific implementation method is: it works by detecting infrared radiation emitted from the surface of an object, converting these radiations into electrical signals and processing them into temperature distribution images. The results are superimposed in real time on the polarized 3D main screen 13 or presented separately on the polarized 3D sub-screen 14.

[0065] Specifically, the gas sensor is an electrochemical sensor used to detect changes in gas composition in the surgical area, which may help to detect complications early and can perform tissue metabolism monitoring, hypoxia detection, infection or necrosis indication.

[0066] Specifically, the Raman spectroscopy sensor can provide the operator with real-time tissue composition analysis capabilities in a non-invasive manner. Its specific implementation method is as follows: a miniature fiber optic probe is connected to an optical fiber to emit near-infrared light to the tissue and collect scattered light from the tissue at the same time. The small high-sensitivity spectrometer on the sensor analyzes the scattered light collected from the tissue, and the sensor data processing unit analyzes the spectral data in real time. The results are superimposed in real time on the polarization 3D main screen 13 or presented separately on the polarization 3D secondary screen 14.

[0067] Specifically, the temperature and humidity sensor is used to monitor the temperature and humidity of the operating environment.

[0068] Specifically, the micro-ultrasound probe is a miniaturized ultrasonic imaging device that can provide the surgeon with real-time deep tissue structure information, assist in surgical navigation, and improve its accuracy.

[0069] The technical means disclosed in the solutions of the present invention are not limited to those disclosed in the above-mentioned embodiments, but also include technical solutions composed of any combination of the above-mentioned technical features. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. An intelligent surgical exoscopic system, characterized in that: It includes a work trolley, a pneumatic robotic arm, an intelligent multi-path imaging system arranged at the end of the pneumatic robotic arm, a 3D display device and an auxiliary control device; the work trolley includes a box body, a display rotating bracket, a four-star foot base and a universal caster, wherein the display rotating bracket and the four-star foot base are respectively installed on the top and bottom of the box body, and the universal caster is cooperatively installed at the four top corners of the four-star foot base; the pneumatic robotic arm includes a pneumatic robotic arm, a pneumatic robotic arm and a pneumatic camera arm, wherein the lower end of the pneumatic robotic arm is installed on the box body, the pneumatic robotic arm has two sections, one of which is connected to the upper end of the pneumatic robotic arm through a big arm ball joint, the two sections and the other section are connected to the pneumatic camera arm through a small arm ball joint, and the pneumatic camera arm is connected to the intelligent multi-path imaging system through the arm ball joint; the intelligent multi-path imaging The system includes multiple integrated zoom imaging optical paths, an intelligent focusing controller and a ring-shaped integrated chip; the ring-shaped integrated chip is installed on the periphery of the multiple zoom imaging optical paths and integrates thermal imaging sensors, gas sensors, Raman spectroscopy sensors and temperature and humidity sensors, and is peripherally connected to a miniature ultrasonic probe and a miniature fiber optic probe; the 3D display device includes a polarized 3D main screen, a polarized 3D secondary screen and a head-mounted display device; the polarized 3D main screen and the polarized 3D secondary screen are connected to the box through a display rotating bracket and can be rotated around a fixed axis; the head-mounted display device includes VR glasses and AR glasses; the auxiliary control device includes a pressure-sensitive foot pedal for controlling the orientation and height of the intelligent multi-optical path imaging system; the multiple zoom imaging optical paths integrated in the intelligent multi-optical path imaging system are used to simultaneously perform multi-viewpoint 3D imaging and multimodal imaging.

2. The intelligent surgical exoscopic system according to claim 1, characterized in that: The front end of the box is provided with a robotic arm control system, an image processing host and a light source control system, the rear end of the box is provided with a drawer for storing medical equipment, etc., and the outside of the box is provided with an operation screen and heat dissipation holes. The operation screen is used to start and shut down the exterior mirror system, manually adjust the pneumatic robotic arm, set the image processing host output mode, and control the imaging mode; the heat dissipation holes are used to dissipate heat to the robotic arm control system, the image processing host and the light source control system; the robotic arm control system is used to provide motion control, posture adjustment, automatic positioning, balance control, safety functions, braking functions, sensor feedback and integration functions.

3. The intelligent surgical exoscopic system according to claim 2, characterized in that: The image processing host is used to process the image information obtained by the camera system and output it in the output mode required by the user. The output modes include naked-eye 3D mode and HMD mode. The naked-eye 3D mode is suitable for polarized 3D main screen and polarized 3D secondary screen, and the HMD mode is suitable for head-mounted display devices.

4. The intelligent surgical exoscopic system according to claim 2, characterized in that: The image processing host provides intraoperative navigation and path planning functions, wherein the intraoperative navigation function is specifically implemented as follows: using pre-acquired medical imaging data to construct a three-dimensional model of the patient's anatomical structure, aligning the real-time position and posture of the exoscope with the three-dimensional model to achieve real-time positioning of the exoscope within the patient's body, and displaying the current position and field of view of the exoscope on the three-dimensional model to provide intraoperative navigation for the doctor; the path planning function is specifically implemented as follows: based on the surgical objectives and anatomical structure, automatically generating the optimal path for the exoscope to reach the target site, and combining with the navigation system to guide the exoscope to move along the planned path, avoiding key blood vessels and nerve tissues.

5. The intelligent surgical exoscopic system according to claim 2, characterized in that: The image processing host also solves the problem of decreased surgical field brightness caused by shadowless lamps being blocked during surgery. The specific implementation method is as follows: training a CNN model for surgical field brightness compensation; applying the CNN model to surgical field images at multiple scales and using multi-scale fusion technology to obtain clearer and more natural enhancement results; at the same time, introducing an edge-aware loss function and applying an edge-preserving filter in the post-processing stage to ensure that the enhanced image can retain key structural details and avoid image distortion or artifacts; The loss function is defined as: L = L_mse +λ×L_edge, Where L_mse is the mean square error loss function, which is used to measure the pixel-level difference between the enhanced image and the real image; L_edge is the edge loss function, which is used to measure the difference in edge information between the enhanced image and the real image; the Sobel operator or the Canny operator is used to extract the image edge and calculate the difference in edge pixels; λ is the weight coefficient used to balance the importance of the two loss functions; Where L_mse is defined as: L_mse = 1 / N×Σ(I_e - I_gt) 2 , Where I_e is the enhanced image, I_gt is the real image, and N is the total number of pixels; L_edge is defined as: L_edge = 1 / M×Σ(E_e - E_gt) 2 , Among them, E_e is the edge information of the enhanced image, E_gt is the edge information of the real image, and M is the total number of edge pixels.

6. The intelligent surgical exoscopic system according to claim 2, characterized in that: The light source control system provides light source brightness adjustment, spectrum adjustment, light source stability control, and uniform illumination detection functions; wherein, the spectrum adjustment function of the light source control system is used for multimodal imaging; multimodal imaging includes white light imaging, ICG fluorescence imaging, and narrow-band imaging; the light source control system is connected to the zoom imaging optical path via optical fibers, and there are multiple optical fibers for transmitting light of different wavelengths to the front end of the exterior mirror.

7. The intelligent surgical exoscopic system according to claim 2, characterized in that: The two imaging optical paths can respectively simulate the imaging systems of the left and right eyes of the human body, and simultaneously collect two sets of images of the same surgical field. After the two sets of images are three-dimensionally synthesized by the image processing host, they are output to the corresponding 3D display device; the 3D display devices each correspond to two of the imaging optical paths, and the remaining optical paths are used for multimodal imaging; multiple zoom imaging optical paths are distributed in a circular array around the central axis of the intelligent multi-optical path imaging system, meeting an optical zoom ratio of more than 10 times, and have automatic focus and viewing angle adjustment functions; the intelligent focus controller adopts a rotary button design, and the specific implementation method is: the button can be pressed and rotated, pressing it is used to switch between automatic focus and manual focus mode, and rotating it is used for manual focus, accompanied by sound feedback to inform the operator of the focusing status.

8. The intelligent surgical exoscopic system according to claim 2, characterized in that: The thermal imaging sensor is used to assist the surgeon in assessing tissue perfusion and detecting tumors or inflammation, with the results superimposed in real time on the polarized 3D main screen or presented separately on the polarized 3D secondary screen. The gas sensor is used to detect changes in gas composition in the surgical area, facilitating early detection of complications and enabling tissue metabolism monitoring, hypoxia detection, and infection or necrosis indication. The Raman spectroscopy sensor non-invasively provides the surgeon with real-time tissue composition analysis capabilities. The sensor is equipped with a small, highly sensitive spectrometer and a sensor data processing unit, with the results superimposed in real time on the polarized 3D main screen or presented separately on the polarized 3D secondary screen. The temperature and humidity sensor is used to monitor the temperature and humidity of the surgical environment.

9. The intelligent surgical exoscopic system according to claim 1, characterized in that: The micro-ultrasound probe is a miniaturized ultrasonic imaging device that provides the surgeon with real-time deep tissue structure information, assists in surgical navigation, and improves its accuracy.

10. The intelligent surgical exoscopic system according to claim 2, characterized in that: The transmission method of the 3D scene information obtained after processing by the image processing host and the 3D display device, and the information transmission method of the pressure-sensitive foot pedal and the robotic arm control system are all 5G communication wireless transmission; the pressure-sensitive foot pedal is composed of a base, a pedal, and a fixed block. It is based on a pressure sensor and a tilt sensor. There are two on the left and right. The orientation and height of the intelligent multi-path imaging system are controlled by stepping on it in different ways. The fixed block is used to fix or release the position of the intelligent multi-path imaging system; the operation screen can help the operator customize different functional combinations of the pedals.

Images