A head-mounted surgical field closed-loop feedback system

Abstract

This invention discloses a head-mounted surgical visual field feedback system, comprising: a head-mounted visual field feedback device, which includes: a head fixation component, a camera component, a display component, a wireless transmission module, and a power supply module; the head fixation component includes a horizontal annular band and a top head band, with the two ends of the top head band symmetrically mounted on the left and right sides of the horizontal annular band; the front and rear ends of the horizontal annular band are respectively provided with a first mounting base and a second mounting base; the camera component is mounted on the first mounting base; the wireless transmission module is mounted on the second mounting base and is connected to the display component and the camera component via signal lines; the display component is mounted on the first mounting base; the power supply module is integrated with the wireless transmission module and is used to power the wireless transmission module, the display component, and the camera component. This invention can alleviate discomfort for users wearing the device for extended periods and improve operational stability during surgery.

Description

Technical Field

[0001] This invention relates to the field of surgical equipment technology, and in particular to a head-mounted surgical field closed-loop feedback system. Background Technology

[0002] Currently, head-mounted surgical camera systems are widely used in clinical practice for surgical teaching, remote consultations, and intraoperative collaboration. Traditional head-mounted surgical camera systems typically consist of a headband / full-coverage helmet and a camera mounted on it. The camera captures video footage during surgery and outputs it to a local display via a wireless transceiver. Existing head-mounted surgical camera systems have several drawbacks: First, the battery, processor, and other components are usually integrated into the headpiece. Wearing this system for extended surgeries can cause discomfort for the surgeon, indirectly affecting the stability of the procedure. Second, existing head-mounted surgical camera systems typically only display the raw image captured by the camera in real-time, and cannot guarantee an unobstructed field of view, increasing the operational difficulty for medical staff. Third, existing head-mounted surgical camera systems cannot meet the needs of surgeons who require simultaneous access to both their own naked-eye view and the display screen's field of view. Specifically, existing head-mounted surgical camera systems primarily output images in two ways: one is to output the image to an external display screen (e.g., the technical solution with application number 202080087395.7); the other is a display screen that completely obscures the eyes, allowing medical personnel to see only the processed image (e.g., the technical solution with application number 201510053398.X). The former requires the user to turn their head to observe the external display screen, while the latter only allows the user to see the processed image, lacking a sense of immersion. Therefore, developing a new wire harness oil coating system to overcome these problems in existing technologies is a direction that requires further research by those skilled in the art. Summary of the Invention

[0003] The purpose of this invention is to provide a head-mounted surgical field closed-loop feedback system that can alleviate the discomfort of long-term wearers and improve the operational stability during surgery.

[0004] This invention provides a head-mounted surgical visual field closed-loop feedback system, comprising:

[0005] A head-mounted visual feedback device, comprising: a head fixation component, a camera assembly, a display assembly, a wireless transmission module, and a power supply module;

[0006] The head fixing component includes a horizontal annular band and a top head band, with the two ends of the top head band symmetrically installed on the left and right sides of the horizontal annular band; the front end of the horizontal annular band is provided with a first mounting seat, and the rear end of the horizontal annular band is provided with a second mounting seat;

[0007] The camera component is mounted on the first mounting base, and the camera component is used to acquire the original image;

[0008] The wireless transmission module is mounted on the second mounting base. The wireless transmission module is connected to the display component and the camera component via signal lines. The wireless transmission module is configured to output the original image obtained by the camera component to an external local server and to transmit the feedback image from the external local server to the display component.

[0009] The display component is mounted on the first mounting base and is used to display the feedback screen;

[0010] The power supply module is integrated with the wireless transmission module and is configured to provide power to the wireless transmission module, the display component, and the camera component. That is, the power supply module is electrically connected to the wireless transmission module, the display component, and the camera component, respectively.

[0011] The head fastener is made of breathable fabric and an elastic adjustment strap.

[0012] This technical solution involves the doctor wearing a head-mounted visual feedback device before surgery, secured by a head fixation device, with the camera facing forward and activated. During surgery, the camera continuously captures raw images, which are then wirelessly transmitted to an external local server. The external server processes the raw images using a pre-set image processing program and generates feedback images. The wireless transmission module receives these feedback images and displays them on the display component. In this process, the horizontal ring band has a first component consisting of the camera and display components, and a second component consisting of the wireless transmission module and power supply module installed at its front and rear ends, respectively. This achieves a substantially symmetrical load-bearing structure in the horizontal direction compared to existing technologies. Furthermore, the headband provides a vertical force at the top of the head, further optimizing the horizontal load balance. This alleviates discomfort for prolonged wearers and improves operational stability during surgery.

[0013] Preferably, in the aforementioned head-mounted surgical field closed-loop feedback system,

[0014] The camera assembly includes a camera body and an illumination assembly; the camera body includes an integrated miniature camera and a 4K / 60fps CMOS sensor; the illumination assembly includes miniature LED beads, and the orientation of the miniature LED beads is configured to be coaxial with the orientation of the lens of the miniature camera.

[0015] By adopting this technical solution, the camera body is used to acquire raw images during the operation, while the lighting component provides illumination, which helps to reduce the shadows cast by instruments and hands during the operation, thus optimizing the feedback image finally presented on the display component.

[0016] Preferably, in the above-described head-mounted surgical field closed-loop feedback system, the display component includes a display panel and a flipping mechanism; the display panel is an OLED display or a Micro-LED display; the fixed end of the flipping mechanism is rotatably connected to the first mounting base, and the rotating end of the flipping mechanism is fixedly mounted on the display panel. The screen resolution of the display panel is configured to be no less than 1280×720, and the screen brightness of the display panel is ≥1000 nits.

[0017] This technical solution involves displaying feedback images on a display panel. During surgery, the tilt angle of the display panel relative to the patient can be adjusted via a flipping mechanism, allowing the display panel to alternate between active and idle states.

[0018] Preferably, in the above-described head-mounted surgical field closed-loop feedback system, the flipping mechanism includes a damping shaft configured to rotate between 0° and -90° horizontally.

[0019] By employing this technical solution, the damping shaft can maintain its current angle after rotating to a certain angle without sagging or rebounding. This enables multi-angle tilt adjustment of the display panel.

[0020] Preferably, in the above-mentioned head-mounted surgical visual field closed-loop feedback system, the display panel is configured such that the exit pupil position of its built-in optical module is located above the eyeball of the wearer wearing the head-mounted visual field feedback device.

[0021] By employing this technical solution, the optical imaging module utilizes a freeform prism or off-axis reflection design to focus the display panel image onto the retina of the user's eye (usually the non-dominant eye), with a field of view of approximately 30-40 degrees. This allows the user to flexibly switch between the naked-eye observation field of view and the feedback image field of view obtained from the display component without turning their head during surgery. The user then integrates these two fields of view through mental association, optimizing their understanding of the wound condition during clinical surgery.

[0022] Preferably, in the above-mentioned head-mounted surgical visual field closed-loop feedback system, the number of head-mounted visual field feedback devices is configured to be several, and the several head-mounted visual field feedback devices respectively exchange signals with the external local server.

[0023] The external local server integrates a 4DGS processing unit, which is configured to synthesize the original images output by several head-mounted visual feedback devices into an unobstructed visual image using a pre-stored 4D Gaussian splash algorithm, and to output the unobstructed visual image to the wireless transmission module of several head-mounted visual feedback devices respectively.

[0024] By adopting this technical solution, the 4D Gaussian splashing algorithm is used to generate a surgical field of view from multiple video streams that can hide interference such as the doctor's hands and instruments and directly display the lesion area, thereby improving surgical efficiency.

[0025] Compared with the prior art, the present invention has achieved the following technical advancements:

[0026] First, this invention improves the horizontal weight-bearing balance of the head-mounted device at both the front and rear ends. Therefore, it can alleviate discomfort for users who wear it for extended periods and improve operational stability during surgery.

[0027] Secondly, the present invention can achieve multi-angle tilt angle adjustment of the display panel, and during the adjustment process, the display panel can maintain the current angle after being rotated to a certain angle.

[0028] Thirdly, the present invention can combine original images from multiple perspectives and obtain a feedback image with interference such as the doctor's hands and instruments through 4D Gaussian splashing algorithm, thereby improving the surgical efficiency of medical staff.

[0029] Finally, this invention enables users to flexibly switch between visual observation (obtained by the naked eye) and feedback image field of view (obtained based on the display component) without turning their heads, thus optimizing the control over wound conditions during clinical surgery. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of a single head-mounted visual feedback device.

[0031] The component names corresponding to the various reference numerals in the figure are as follows:

[0032] 110. Horizontal ring band; 120. Headband; 130. Adjustable knob; 140. First mounting base; 150. Second mounting base; 200. Camera body; 300. Lighting assembly; 400. Wireless transmission module; 410. Connecting cable; 510. Display panel; 520. Flip mechanism. Detailed Implementation

[0033] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention.

[0034] Example 1, please refer to Figure 1 :

[0035] A head-mounted surgical visual field closed-loop feedback system includes: a plurality of head-mounted visual field feedback devices, wherein the plurality of head-mounted visual field feedback devices have the same structure.

[0036] The head-mounted visual feedback device includes: a head fixation component, a camera assembly, a display assembly, a wireless transmission module 400, and a power supply module. The head fixation component is constructed of breathable fabric and an elastic adjustment strap. Specifically, the head fixation component includes a horizontal annular band 110 and a top head band 120, with the two ends of the top head band 120 symmetrically mounted on the left and right sides of the horizontal annular band 110. The horizontal annular band 110 and the top head band 120 may also be equipped with an adjustable knob 130 or similar length adjustment structure. The adjustable knob may be implemented using a ratchet knob or similar structure. The front end of the horizontal annular band 110 is provided with a first mounting base 140, and the rear end of the horizontal annular band 110 is provided with a second mounting base 150.

[0037] The camera assembly is mounted on the first mounting base 140 and is used to acquire the original image. Specifically, the camera assembly includes a camera body 200 and an illumination assembly 300. The camera body 200 includes an integrated miniature camera and a 4K / 60fps CMOS sensor. The illumination assembly 300 includes miniature LED beads, and the orientation of the miniature LED beads is configured to be coaxial with the lens orientation of the miniature camera.

[0038] The wireless transmission module 400 is mounted on the second mounting base 150. The wireless transmission module 400 is connected to the display component and the camera component via signal lines. The wireless transmission module 400 is configured to output the original image obtained by the camera component to an external local server and to transmit the feedback image of the external local server to the display component.

[0039] The display component is mounted on the first mounting base 140 and extends beyond the front end of the horizontal annular band 110. Specifically, the display component includes a display panel 510 and a flip mechanism 520; the display panel 510 is an OLED display or a Micro-LED display; the fixed end of the flip mechanism 520 is rotatably connected to the first mounting base 140, and the rotating end of the flip mechanism 520 is fixedly mounted on the display panel 510. In this example, the flip mechanism 520 uses a damped pivot, which is configured to rotate between 0° and -90° horizontally. The display panel 510 is configured such that the exit pupil of its built-in optical module is located above the eyeball of the wearer of the head-mounted visual feedback device.

[0040] The power supply module is integrated with the wireless transmission module 400 and is configured to supply power to the wireless transmission module 400, the display component, and the camera component. Specifically, the power supply module is electrically connected to the display component and the camera component via a power supply line. The power supply line and the signal line together form a connection line 410. The power supply module can be based on a 3.7V lithium polymer pouch battery.

[0041] The external local server integrates a 4DGS processing unit, which is configured to synthesize the original images output by several head-mounted visual feedback devices into an unobstructed visual image using a pre-stored 4D Gaussian splash algorithm, and to output the unobstructed visual image to the wireless transmission modules 400 of the several head-mounted visual feedback devices respectively. The external local server can be implemented based on a desktop PC / desktop GPU connected to a wireless signal transceiver module.

[0042] In this example, the process of generating an unobstructed view using the 4D Gaussian splash algorithm can be set as follows:

[0043] The original images are received synchronously. Using the initial pose and sparse point cloud of each surgical video as initial conditions, the properties of each Gaussian element are optimized based on the differentiable Gaussian splash rendering pipeline, and a 4DGS model containing all dynamic objects is constructed.

[0044] In 3D space, the Gaussian elements corresponding to the occlusions are directly identified, and the occlusion Gaussian element mask in 3D space is generated. The Gaussian elements of the occlusions that need to be removed are removed from the 4DGS model to generate an image without occlusions.

[0045] Obtain discrete tracking points, convert the discrete tracking points into virtual objects in the 4DGS coordinate system, and superimpose the virtual objects onto the 4DGS model after real-time removal of Gaussian points marked by occluded Gaussian element masks.

[0046] An unobstructed view is rendered from a preset virtual camera position.

[0047] In practice, its working process is as follows:

[0048] Before the surgery begins, each surgeon wears a head-mounted visual feedback device (VFP) on their head using a head-mounted fixation device, with the camera facing forward and activated. During the surgery, the camera continuously captures raw images, which are then continuously output to an external local server via a wireless transmission module 400. The external local server processes multiple raw images synchronously using a preset 4D Gaussian splashing algorithm and generates a feedback image. This feedback image provides a surgical field of view that effectively hides distractions such as the surgeon's hands and instruments, directly displaying the lesion area. The wireless transmission module 400 receives the feedback image and outputs it to the display component for display. The exit pupil of the built-in optical module of the display panel 510 is located above the wearer's eyeball. This optical imaging module uses a freeform prism or off-axis reflection design to focus the image from the display panel 510 onto the retina of the user's eye (usually the non-dominant eye), with a field of view of approximately 30-40 degrees. Therefore, during surgery, the user can flexibly switch between the naked-eye observation field of view and the feedback image field of view obtained from the display component without turning their head. The user integrates these two fields of view through brain association, optimizing the grasp of the wound condition during clinical surgery. In the above process, since the front and rear ends of the horizontal ring band 110 are respectively equipped with a first component consisting of a camera component and a display component, and a second component consisting of a wireless transmission module 400 and a power supply module, a basically symmetrical load-bearing structure in the horizontal direction is achieved compared with the prior art. At the same time, by setting the headband 120 to provide a force in the vertical direction at the top of the head, the load balance in the horizontal direction is further optimized. Therefore, it can alleviate the discomfort of long-term wearers and improve the operational stability during surgery.

[0049] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they shall still fall within the protection scope of the present invention.

Claims

1. A head-mounted surgical field closed-loop feedback system, characterized in that, include: A head-mounted visual feedback device, comprising: a head fixation component, a camera assembly, a display assembly, a wireless transmission module, and a power supply module; The head fixing component includes a horizontal annular band and a top head band. The two ends of the top head band are symmetrically installed on the left and right sides of the horizontal annular band. The front end of the horizontal annular band is provided with a first mounting seat, and the rear end of the horizontal annular band is provided with a second mounting seat. The camera component is mounted on the first mounting base, and the camera component is used to acquire the original image; The wireless transmission module is mounted on the second mounting base. The wireless transmission module is connected to the display component and the camera component via signal lines. The wireless transmission module is configured to output the original image obtained by the camera component to an external local server and to transmit the feedback image from the external local server to the display component. The display component is mounted on the first mounting base and is used to display the feedback screen; The power supply module is integrated with the wireless transmission module and is configured to provide power to the wireless transmission module, the display component and the camera component.

2. The head-mounted surgical field closed-loop feedback system according to claim 1, characterized in that, The camera assembly includes a camera body and an illumination assembly; the camera body includes an integrated miniature camera and a 4K / 60fps CMOS sensor; the illumination assembly includes miniature LED beads, and the orientation of the miniature LED beads is configured to be coaxial with the orientation of the lens of the miniature camera.

3. The head-mounted surgical field closed-loop feedback system according to claim 2, characterized in that, The display component includes a display panel and a flipping mechanism; the display panel is an OLED display or a Micro-LED display; the fixed end of the flipping mechanism is rotatably connected to the first mounting base, and the rotating end of the flipping mechanism is fixedly mounted on the display panel.

4. The head-mounted surgical field closed-loop feedback system according to claim 3, characterized in that, The flipping mechanism includes a damping shaft configured to rotate between 0° and -90° horizontally.

5. The head-mounted surgical field closed-loop feedback system according to claim 3, characterized in that, The display panel is configured such that the exit pupil of its built-in optical module is located above the eyeball of the wearer of the head-mounted visual feedback device.

6. The head-mounted surgical field closed-loop feedback system according to claim 5, characterized in that, The number of head-mounted visual feedback devices is configured to be several, and each of the several head-mounted visual feedback devices exchanges signals with the external local server; The external local server integrates a 4DGS processing unit, which is configured to synthesize the original images output by several head-mounted visual feedback devices into an unobstructed visual image using a pre-stored 4D Gaussian splash algorithm, and to output the unobstructed visual image to the wireless transmission module of several head-mounted visual feedback devices respectively.

Images