Three-dimensional exterior mirror operation system

Abstract

The invention discloses a three-dimensional exterior mirror operation system, which belongs to the technical field of medical instrument control, and comprises a microscopic exterior mirror module, an image display module, an image processing module, an image processing module and a display module, the human body shooting module shoots the face, eyeballs and body postures of a doctor, the control and adjustment module controls and adjusts the image display module according to the content shot by the human body shooting module, automatic adjustment of image display and automatic movement of a display screen in the operation process can be achieved, and the operation quality and efficiency are improved.

Description

Technical Field

[0001] The present application belongs to the field of medical device control technology, and specifically relates to a three-dimensional exoscopic surgery system. Background Art

[0002] With the rapid development of medical imaging technology, surgical procedures are gradually moving towards minimally invasive and precision surgery. As an important auxiliary tool, exoscopic systems play a vital role in minimally invasive surgery, open surgery, and remote collaboration. Exoscopic systems use cameras to transmit real-time images of the surgical area to a monitor for the doctor to observe and operate. This allows doctors to perform precise surgical procedures with minimal exposure to the patient's internal tissues, thereby reducing surgical trauma, shortening postoperative recovery time, and improving surgical outcomes.

[0003] There are many types of surgical exoscopic systems available on the market, but there are still some problems that cannot be ignored. The exoscopic system is not very intelligent and convenient. The lens cannot zoom automatically and requires additional zooming operation, which affects the efficiency of the operation. The display screen is fixed and difficult to adjust in time. When the operation time is long, the doctor will maintain a posture for a long time to adapt to the display screen, causing physical discomfort, which may affect the quality of the operation. Summary of the Invention

[0004] To solve the above problems and technical defects, the present application adopts the following technical solution: a three-dimensional exoscopic surgery system, comprising: A microscopic external viewing module is used to take microscopic photos and illuminate the surgical area using a microscopic external viewing scope; An image display module is used to output and display the content of the microscopic photography; Human body shooting module, used to shoot the doctor's face, eyes and body posture; The control and adjustment module is used to control and adjust the image display module according to the content captured by the human body shooting module.

[0005] Preferably, the human body shooting module includes: a face positioning module, an eye tracking module and a posture recognition module; A facial positioning module is used to determine the doctor's facial orientation and position; Eye tracking module, used to track and locate the doctor's eyes; The posture recognition module is used to locate the patient's posture and the patient's surgical area, and determine the doctor's body posture and body orientation based on the positioning results.

[0006] Furthermore, the control and adjustment module includes: a three-dimensional adjustment module, a focal length adjustment module and a rotation adjustment module; A three-dimensional adjustment module, configured to output a three-dimensional adjustment signal according to the doctor's facial orientation and position, and to control the position of the image display module so that the position and orientation of the image display module match the doctor's facial orientation and position; The focus adjustment module is used to track and locate the doctor's eyeballs to determine the doctor's gaze point on the image displayed by the image display module, and output a focus adjustment signal to control the microscopic external viewing module to focus the gaze point; The rotation adjustment module is used to control the output of the rotation adjustment signal according to the doctor's body posture and body orientation, and rotate the image displayed by the image display module to make the displayed image match the doctor's body posture and body orientation.

[0007] Furthermore, the image display module includes a three-dimensional adjustment bracket and a display screen. The display screen is mounted on the three-dimensional adjustment bracket, and the three-dimensional adjustment bracket is controlled and adjusted by a three-dimensional adjustment signal.

[0008] Preferably, the microscopic external viewing module includes an optical recognition module for integrating visible light and near-infrared band lighting to output band lighting that matches the usage environment, and at the same time, a built-in annular LED array eliminates reflections from surgical metal instruments.

[0009] Preferably, it also includes a voice control module for recognizing the doctor's language and converting the recognition result into a control signal to adjust and control the image display module and the microscopic external viewing module.

[0010] Furthermore, the control signal output by the voice control module has a higher priority than the adjustment signal output by the gesture recognition module.

[0011] A three-dimensional endoscopic surgical device includes a service processor and a distributed memory. The service processor is connected to the memory. The distributed memory stores a service self-management program configured to store machine-readable instructions. The service processor executes the service self-management program. When the instructions are executed by the processor, a three-dimensional endoscopic surgical system as described above is realized.

[0012] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the contents of the three-dimensional exoscopic surgical system as described above are realized.

[0013] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the contents of the three-dimensional exoscopic surgery system described above.

[0014] Compared with the prior art, the present invention has the following advantages: This application uses the human body shooting module to monitor and collect information on the doctor's face, eyeballs, body posture and orientation in real time, and transmits the collected information to the control and adjustment module respectively. The three-dimensional adjustment module, the focal length adjustment module and the rotation adjustment module respectively process the received information to generate corresponding adjustment signals. The three-dimensional adjustment signal controls the three-dimensional adjustment bracket of the image display module to drive the display screen to adjust the position and orientation; the focal length adjustment signal controls the microscopic external viewing module to focus on the gaze point of the surgical area; the rotation adjustment signal controls the image display module to rotate the displayed image. Through the continuous collaborative work of each module, automatic adjustment of the image display during the operation and automatic movement of the display screen are realized, thereby improving the quality and efficiency of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In the attached figure: Figure 1 A schematic diagram of the system structure of an embodiment of the present application; Figure 2 This is a schematic diagram of the device structure of an embodiment of the present application. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0017] Example 1 like Figure 1 As shown, a three-dimensional exoscopic surgery system comprises: A microscopic external viewing module is used to take microscopic photos and illuminate the surgical area using a microscopic external viewing scope; The microscopic external viewing module includes an optical recognition module, which is used to integrate visible light and near-infrared band lighting to output band lighting that matches the usage environment. At the same time, the built-in ring LED array eliminates reflections from surgical metal instruments.

[0018] The microscopic external viewing module integrates a high-resolution microscope lens and a shadowless surgical light to perform real-time three-dimensional microscopic photography and lighting of the surgical area. It has a built-in electric zoom lens and supports remote control of focus adjustment.

[0019] An image display module is used to output and display the content of the microscopic photography; The image display module includes a three-dimensional adjustment bracket and a display screen. The display screen is installed on the three-dimensional adjustment bracket, and the three-dimensional adjustment bracket is controlled and adjusted through a three-dimensional adjustment signal. The three-dimensional adjustment bracket adopts a six-degree-of-freedom robotic arm structure, and the driving end is connected to the display screen. The three-dimensional adjustment bracket has multi-degree-of-freedom adjustment capabilities. It is equipped with a driving component that is compatible with the control and adjustment module and can receive the three-dimensional adjustment signal sent by the control and adjustment module.

[0020] Under the action of the three-dimensional adjustment signal, the three-dimensional adjustment bracket can realize translation adjustment along the X-axis, Y-axis and Z-axis and rotation adjustment around the corresponding axes, thereby driving the change of the position and orientation of the display screen, so that the position and orientation of the display screen match the doctor's facial orientation and position, making it easier for the doctor to observe the image comfortably.

[0021] The display screen uses a high-refresh-rate medical OLED display, supports 4K / 3D image output, and its display resolution and color reproduction meet the high-precision requirements of surgical observation.

[0022] The human body shooting module is used to shoot the doctor's face, eyes and body posture, capture the doctor's face, eyes and body posture data in real time, and locate the patient's surgical area.

[0023] The human body shooting module includes: facial positioning module, eye tracking module and posture recognition module; The facial positioning module is used to determine the doctor's facial orientation and position. It uses a binocular depth camera to determine the spatial coordinates and orientation angle of the doctor's face through feature point recognition. It uses image recognition technology to continuously photograph the doctor's face, extract features from the captured facial images, and obtain key facial feature points, such as the corners of the eyes, the tip of the nose, and the corners of the mouth. Based on the position changes of the key feature points, the doctor's facial orientation and specific position in space are determined, and the determination results are transmitted in real time to the three-dimensional adjustment module in the control and adjustment module.

[0024] The eye tracking module is used to track and locate the doctor's eyes. It is equipped with an infrared active light source and a high-speed camera to capture the doctor's eye movement trajectory in real time and calculate the coordinates of the gaze point. It processes the collected eye images and extracts characteristic information such as the pupil center and corneal reflection point of the eye. By analyzing the changing trajectory of the characteristic information, it can realize real-time tracking and positioning of the doctor's eyes, determine the doctor's gaze point position on the image displayed by the image display module, and send the gaze point information to the focus adjustment module of the control and adjustment module.

[0025] The posture recognition module is used to locate the patient's posture and the patient's surgical area, determine the doctor's body posture and body orientation based on the positioning results, identify the doctor's body posture and the patient's surgical area position through a wide-angle camera combined with a skeletal key point detection algorithm, perform feature analysis on the patient's posture image, determine the patient's body position, identify and locate the surgical area image, clarify the position range of the surgical area in space, combine the positioning results of the patient's posture and the surgical area, and determine the doctor's body posture and body orientation presented to adapt to the surgical operation through spatial position relationship calculation, and transmit it to the rotation adjustment module of the control and adjustment module.

[0026] The control and adjustment module is used to control and adjust the image display module according to the content captured by the human body shooting module.

[0027] The control and adjustment module includes: a three-dimensional adjustment module, a focal length adjustment module and a rotation adjustment module; A three-dimensional adjustment module, configured to output a three-dimensional adjustment signal according to the doctor's facial orientation and position, and to control the position of the image display module so that the position and orientation of the image display module match the doctor's facial orientation and position; Receive facial positioning data, calculate the coordinates and rotation angles of the display screen, and drive the three-dimensional adjustment bracket to make the screen face the doctor's line of sight; According to the preset position matching algorithm, the doctor's facial position and orientation parameters are converted into corresponding three-dimensional adjustment signals. The three-dimensional adjustment signals are transmitted to the three-dimensional adjustment bracket of the image display module, which controls the three-dimensional adjustment bracket to drive the display screen to perform translation and rotation movements until the position and orientation of the display screen are optimally matched with the doctor's facial orientation and position, ensuring that the doctor can clearly observe the screen without deliberately adjusting his head posture.

[0028] The focus adjustment module is used to track and locate the doctor's eyeballs to determine the doctor's gaze point on the image displayed by the image display module, and output a focus adjustment signal to control the microscopic external viewing module to focus the gaze point; Based on eye tracking data, the position of the doctor's gaze point on the display screen is mapped, and the corresponding target point in the surgical area is reversely calculated, and the electric zoom lens of the microscopic external viewing module is controlled to automatically focus on this point; Based on the coordinate position of the gaze point in the image and the imaging parameters of the microscopic external viewing module, a focus adjustment signal is generated and transmitted to the microscopic external viewing module to control the position of the optical components inside the microscope lens to adjust the position, so that the microscope lens can accurately focus on the position corresponding to the gaze point in the surgical area, enhance the clarity of the image in this area, and realize automatic zoom of the lens.

[0029] A rotation adjustment module is used to control the output of a rotation adjustment signal according to the doctor's body posture and body orientation, and to rotate the image displayed by the image display module so that the displayed image matches the doctor's body posture and body orientation; Based on the posture recognition data, the display screen is rotated in real time using affine transformation to ensure that the screen direction is consistent with the doctor's viewing direction; The rotation adjustment signal is transmitted to the image display module, which controls the display screen to adjust the rotation angle of the displayed surgical image so that the orientation of the displayed image matches the doctor's body posture and orientation, avoiding visual discomfort caused by inconsistency between the doctor's body posture and the image orientation, and improving the coordination of surgical operations.

[0030] It also includes a voice control module for recognizing the doctor's language and converting the recognition results into control signals to adjust and control the image display module and the microscopic external viewing module.

[0031] The control signal output by the voice control module has a higher priority than the adjustment signal output by the gesture recognition module.

[0032] Example 2 like Figure 2 As shown, a three-dimensional endoscopic surgical device includes a service processor and a distributed memory, wherein the service processor is connected to the memory, and the distributed memory stores a service self-management program configured to store machine-readable instructions. The service processor executes the service self-management program, and when the instructions are executed by the processor, the three-dimensional endoscopic surgical system as described in Example 1 is realized.

[0033] Example 3 From a hardware perspective, the present application provides an embodiment of an electronic device that contains all or part of the contents of a three-dimensional endoscopic surgical system, wherein the electronic device includes a service processor and a distributed memory, wherein the service processor is connected to the memory, wherein the distributed memory stores a service self-management program configured to store machine-readable instructions, and wherein the service processor executes the service self-management program, and when the instructions are executed by the processor, the three-dimensional endoscopic surgical system as described above is realized.

[0034] From a hardware perspective, in order to effectively improve the flexibility, versatility, and efficiency of data acquisition, the present application provides embodiments of electronic devices that include all or part of a 3D endoscopic surgery system. The electronic devices specifically include the following: A processor, a memory, a communications interface, and a bus; wherein the processor, the memory, and the communications interface communicate with each other via the bus; the communications interface is used to realize information transmission between the core business system of the three-dimensional endoscopic surgery system, user terminals, and related databases and other related devices; the logic controller can be a desktop computer, a tablet computer, a mobile terminal, etc., but this embodiment is not limited to this.

[0035] In this embodiment, the logic controller can be implemented with reference to the embodiment of the three-dimensional exoscopic surgery system in the embodiment, the contents of which are incorporated herein and the repeated parts are not repeated.

[0036] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc., wherein the smart wearable device may include smart glasses, smart watches, smart bracelets, etc.

[0037] In actual applications, portions of the 3D exoscopic surgery system may be executed on the electronic device as described above, or all operations may be performed on the client device. The specific selection may depend on the processing capabilities of the client device and the limitations of the user's usage scenario, and this application does not impose any restrictions on this. If all operations are performed on the client device, the client device may also include a processor.

[0038] The above-mentioned client device may have a communication module (i.e., a communication unit), which can communicate with a remote server to realize data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0039] Example 4 The embodiments of the present application also provide a computer-readable storage medium capable of implementing the three-dimensional exoscopic surgical system in which the execution subject is a server or a client in the above-mentioned embodiments. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements all the contents of the three-dimensional exoscopic surgical system in which the execution subject is a server or a client in the above-mentioned embodiments.

[0040] The embodiments of the present application may be provided as methods, apparatuses, or computer program products. Thus, the present application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0041] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (apparatus), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0042] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0043] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0044] The above-described embodiments merely represent preferred implementations of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person skilled in the art would be able to make numerous variations, improvements, and substitutions without departing from the spirit of the present application, all of which fall within the scope of protection of the present application.

Claims

1. A three-dimensional exoscopic surgical system, characterized in that: include: A microscopic external viewing module is used to take microscopic photos and illuminate the surgical area using a microscopic external viewing scope; An image display module is used to output and display the content of the microscopic photography; Human body shooting module, used to shoot the doctor's face, eyes and body posture; The control and adjustment module is used to control and adjust the image display module according to the content captured by the human body shooting module.

2. A three-dimensional exoscopic surgery system according to claim 1, characterized in that: The human body shooting module includes: a face positioning module, an eye tracking module and a posture recognition module; A facial positioning module is used to determine the doctor's facial orientation and position; Eye tracking module, used to track and locate the doctor's eyes; The posture recognition module is used to locate the patient's posture and the patient's surgical area, and determine the doctor's body posture and body orientation based on the positioning results.

3. A three-dimensional exoscopic surgery system according to claim 2, characterized in that: The control and adjustment module includes: a three-dimensional adjustment module, a focal length adjustment module and a rotation adjustment module; A three-dimensional adjustment module, configured to output a three-dimensional adjustment signal according to the doctor's facial orientation and position, and to control the position of the image display module so that the position and orientation of the image display module match the doctor's facial orientation and position; The focus adjustment module is used to track and locate the doctor's eyeballs to determine the doctor's gaze point on the image displayed by the image display module, and output a focus adjustment signal to control the microscopic external viewing module to focus the gaze point; The rotation adjustment module is used to control the output of the rotation adjustment signal according to the doctor's body posture and body orientation, and rotate the image displayed by the image display module to make the displayed image match the doctor's body posture and body orientation.

4. A three-dimensional exoscopic surgery system according to claim 3, characterized in that: The image display module includes a three-dimensional adjustment bracket and a display screen. The display screen is installed on the three-dimensional adjustment bracket, and the three-dimensional adjustment bracket is controlled and adjusted through a three-dimensional adjustment signal.

5. The three-dimensional exoscopic surgery system according to claim 1, characterized in that: The microscopic external viewing module includes an optical recognition module for integrating visible light and near-infrared band lighting to output band lighting that matches the usage environment. At the same time, a built-in annular LED array eliminates reflections from surgical metal instruments.

6. The three-dimensional exoscopic surgery system according to claim 1, characterized in that: It also includes a voice control module for recognizing the doctor's language and converting the recognition results into control signals to adjust and control the image display module and the microscopic external viewing module.

7. The three-dimensional exoscopic surgery system according to claim 6, characterized in that: The control signal output by the voice control module has a higher priority than the adjustment signal output by the gesture recognition module.

8. A three-dimensional exoscopic surgical device, characterized in that: The device includes a service processor and a distributed memory, the service processor is connected to the memory, the distributed memory stores a service self-management program, and is configured to store machine-readable instructions. The service processor executes the service self-management program, and when the instructions are executed by the processor, a three-dimensional endoscopic surgical system as described in claim 1 is implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the content of the three-dimensional exoscopic surgery system described in claim 1 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the content of the three-dimensional exoscopic surgery system described in claim 1 is realized.

Images