Multi-component interaction system and method for endoscopic surgery

Abstract

The embodiment of the invention discloses a multi-component interaction system and method for endoscopic surgery. The system comprises an industrial personal computer, an image trolley, a pedal and an instrument, the industrial personal computer comprises a first industrial personal computer and a second industrial personal computer; wherein the first industrial personal computer is used for collecting interaction information triggered by a user through pedaling in real time and sending the interaction information to the image trolley and the second industrial personal computer; the second industrial personal computer is used for controlling the instrument to perform surgical operation and image acquisition based on the received interaction information to obtain a surgical image corresponding to the surgical operation, and sending the surgical image to the image trolley; and the image trolley is used for performing image processing on the operation image based on the received interaction information to obtain a fused image and displaying the fused image. Through the technical scheme of the embodiment of the invention, multi-component interaction of the endoscopic surgery can be quickly and conveniently realized, and the interaction efficiency among the components in the system is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of communication technology, and in particular to a multi-component interaction system and method for endoscopic surgery. Background Art

[0002] With the development of technology, the components used in surgery are becoming more and more diverse, resulting in increasingly complex interactions between the various components.

[0003] At present, point-to-point individual interaction is usually used to achieve independent interaction between components. However, this individual interaction method reduces the interaction efficiency between components and requires the establishment of interaction channels between each component that needs to interact to transmit interaction information, resulting in multiple different types of interaction channels between components, resulting in wiring confusion and entanglement. Summary of the invention

[0004] The embodiments of the present invention provide a multi-component interaction system and method for endoscopic surgery, so as to quickly and conveniently realize the multi-component interaction of endoscopic surgery and improve the interaction efficiency between components in the system.

[0005] According to one aspect of the present invention, a multi-component interactive system for endoscopic surgery is provided, the system comprising:

[0006] Industrial computers, imaging trolleys, pedals and instruments; the industrial computers include: a first industrial computer and a second industrial computer;

[0007] Wherein, the first industrial computer is used to collect the interaction information triggered by the user through the foot pedal in real time, and send the interaction information to the imaging trolley and the second industrial computer;

[0008] The second industrial computer is used to control the instrument to perform surgical operation and image acquisition based on the received interactive information, obtain surgical images corresponding to the surgical operation, and send the surgical images to the imaging trolley;

[0009] The imaging trolley is used to perform image processing on the surgical image based on the received interactive information to obtain a fused image, and then display the fused image.

[0010] Furthermore, the first industrial computer is connected to the foot pedal; and the second industrial computer is connected to the device.

[0011] Furthermore, the imaging trolley comprises: a display and an image processing module;

[0012] The image processing module is used to analyze the received interactive information to obtain information to be fused, perform image processing based on the information to be fused and the surgical image to obtain a fused image, and send the fused image to the display;

[0013] The display is used to display the fused image.

[0014] Furthermore, the display is also used to display current status data; the current status data includes: at least one of indicator light status, joint position, joint current and current instrument information.

[0015] Furthermore, the second industrial computer includes: an electric knife control module and a camera control module;

[0016] The electrosurgical unit control module is used to analyze the received interactive information to obtain an electrosurgical unit control instruction, and perform a surgical operation based on the electrosurgical unit control instruction;

[0017] The camera control module is used to analyze the received interactive information to obtain an image acquisition instruction, and perform image acquisition based on the image acquisition instruction to obtain a surgical image corresponding to the surgical operation, and send the surgical image to the imaging trolley.

[0018] Furthermore, the electrosurgical knife control instructions include: electrosurgical knife energy control instructions and electrosurgical knife position control instructions.

[0019] Furthermore, specific connection lines are used for communication between the first industrial computer and the imaging trolley, between the first industrial computer and the second industrial computer, and between the first industrial computer and the second industrial computer.

[0020] Furthermore, the specific connecting line is composed of a network cable and an electronic cable; the specific connecting line is used for Ethernet communication, serial port communication, IO communication and video communication.

[0021] Furthermore, the specific connecting wire includes a conductor, an insulator, a twisted pair, an aluminum thermal tape, a bottom tape, a shielding tape and a sheath.

[0022] According to another aspect of the present invention, a multi-component interaction method for endoscopic surgery is provided, the method comprising:

[0023] The first industrial computer collects interactive information triggered by the user's foot in real time, and sends the interactive information to the imaging trolley and the second industrial computer;

[0024] Controlling the instrument to perform surgical operation and image acquisition based on the received interactive information by the second industrial computer, obtaining surgical images corresponding to the surgical operation, and sending the surgical images to the imaging trolley;

[0025] The imaging trolley performs image processing on the surgical image based on the received interactive information to obtain a fused image, and displays the fused image.

[0026] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the multi-component interaction method for endoscopic surgery as described in any one of the embodiments of the present invention.

[0027] The technical solution of the embodiment of the present invention uses the first industrial computer to collect the interactive information triggered by the user through foot pedal in real time, and sends the interactive information to the imaging trolley and the second industrial computer; uses the second industrial computer to control the instrument to perform surgical operations and image acquisition based on the received interactive information, obtains the surgical image corresponding to the surgical operation, and sends the surgical image to the imaging trolley; uses the imaging trolley to process the surgical image based on the received interactive information, obtains the fused image, and displays the fused image. In this way, the multi-component interaction of endoscopic surgery can be realized quickly and conveniently, avoiding multiple point-to-point interactive transmissions, and improving the interaction efficiency between components within the system.

[0028] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 is a schematic diagram of the structure of a multi-component interactive system for endoscopic surgery provided according to Embodiment 1 of the present invention;

[0031] Figure 2 is an exemplary diagram of the internal structure of a specific connecting line involved in Embodiment 1 of the present invention;

[0032] Figure 3 This is a flowchart of a multi-component interaction method for endoscopic surgery provided according to Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] Embodiment 1

[0036] Figure 1 A schematic diagram of a multi-component interactive system for endoscopic surgery is provided for Embodiment 1 of the present invention. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0037] like Figure 1 As shown, the multi-component interactive system for endoscopic surgery may include: an industrial computer 110 , an imaging trolley 120 , a pedal 130 and an instrument 140 ; ​​the industrial computer 110 includes: a first industrial computer 111 and a second industrial computer 112 .

[0038] Among them, the first industrial computer 111 is used to collect the interactive information triggered by the user through the foot pedal 130 in real time, and send the interactive information to the imaging trolley 120 and the second industrial computer 112; the second industrial computer 112 is used to control the instrument 140 to perform surgical operations and image acquisition based on the received interactive information, obtain surgical images corresponding to the surgical operations, and send the surgical images to the imaging trolley 120; the imaging trolley 120 is used to process the surgical images based on the received interactive information, obtain fused images, and display the fused images.

[0039] Among them, the foot pedal 130 can be used to generate different interactive information according to the user's triggering method. The interactive information may refer to information for controlling various components in the system to perform corresponding actions. The interactive information may include but is not limited to information to be fused, electrosurgical knife control instructions, and image acquisition instructions. The surgical image may refer to a real-time image during the surgical operation. Continuous surgical images can form a surgical video. The fused image may refer to an image obtained by fusing the information to be fused with the surgical image.

[0040] Specifically, the user generates interactive information by controlling the foot pedal 130. The first industrial computer 111 collects the generated interactive information in real time, and sends the interactive information that meets the communication conditions of the imaging trolley to the imaging trolley 120, and sends the interactive information that meets the communication conditions of the second industrial computer to the second industrial computer 112. The second industrial computer 112 analyzes the received interactive information to obtain an analysis result, and controls the instrument 140 to perform a surgical operation and collect images of the surgical operation based on the analysis result, obtains a surgical image corresponding to the surgical operation, and sends the surgical image to the imaging trolley 120. The imaging trolley 120 processes the surgical image based on the received interactive information to obtain a fused image, and displays the fused image.

[0041] Among them, the communication condition may refer to the communication protocol between two communication components. In the embodiment of the present invention, the pre-established communication protocol may be used to filter the transmittable information, so that when the same connecting line is used between the various components, the non-transmittable information can be effectively filtered, and then a unified hardware connection deployment can be performed, thereby improving the wiring efficiency of the connecting lines between the various components in the system. The connecting line can ensure that various key signals, such as image signals (surgical images), control signals (electric knife control instructions) and data signals (information to be fused), are accurately and stably transmitted between different system components, so that doctors can obtain a clear surgical field of view and accurately control the surgical instrument 140.

[0042] It should be noted that if the received interactive information needs to be standardized, the first industrial computer 111 can standardize the interactive information when receiving it, obtain standard interactive information, and send the standard interactive information to the imaging trolley 120 and the second industrial computer 112. In the single industrial computer mode, the standard wiring harness (connecting wire) is 3 in series. In the dual industrial computer mode, the first industrial computer 111 and the second industrial computer 112 are connected in series to form a ring network, and the standard wiring harness is increased to 4.

[0043] The technical solution of the embodiment of the present invention uses the first industrial computer 111 to collect the interactive information triggered by the user through the foot pedal 130 in real time, and sends the interactive information to the imaging trolley 120 and the second industrial computer 112; uses the second industrial computer 112 to control the instrument 140 to perform surgical operations and image acquisition based on the received interactive information, obtains the surgical image corresponding to the surgical operation, and sends the surgical image to the imaging trolley 120; uses the imaging trolley 120 to process the surgical image based on the received interactive information, obtains the fused image, and displays the fused image. In this way, the multi-component interaction of endoscopic surgery can be realized quickly and conveniently, avoiding multiple point-to-point interactive transmissions, and improving the interaction efficiency between components in the system.

[0044] On the basis of the above technical solution, the first industrial computer 111 is connected to the foot pedal 130; the second industrial computer 112 is connected to the instrument 140. The advantage of this arrangement is that multiple industrial computers can collaborate to complete the surgical operation, avoiding a single industrial computer from performing multi-dimensional complex processing, ensuring the collaborative work of the system, and thus improving the efficiency of the surgical operation. For example, the system can connect various components such as endoscopes, light sources, image processing equipment, and operating consoles to promote their collaborative operation, thereby achieving efficient operation of the entire surgical system.

[0045] On the basis of the above technical solution, the imaging trolley 120 includes: a display and an image processing module; the image processing module is used to analyze the received interactive information to obtain the information to be fused, and perform image processing based on the information to be fused and the surgical image to obtain a fused image, and send the fused image to the display; the display is used to display the fused image.

[0046] The information to be fused may refer to the annotated information to be displayed. For example, the information to be fused may be, but is not limited to, system information to be displayed, such as a system number, etc. The fused image may refer to an image with annotated information obtained by fusing the information to be fused with the surgical image.

[0047] Specifically, the image stream is mainly divided into two parts, one part is the acquired 3D / 2D surgical image (original video data), and the other part is the information to be fused (video overlay control). The information to be fused to be displayed is superimposed and fused with the acquired surgical image and the fused image is output to the 3D display.

[0048] On the basis of the above technical solution, the display is also used to display current status data; the current status data includes: at least one of indicator light status, joint position, joint current and current instrument information.

[0049] The control software sends the current status data to the information interaction software through the TCP / IP protocol, so that the current status data is transmitted as interactive information and transmitted to the display through the existing connection lines between the components for display.

[0050] On the basis of the above technical solution, the second industrial computer 112 includes: an electrosurgical unit control module and a camera control module; the electrosurgical unit control module is used to analyze the received interactive information to obtain electrosurgical unit control instructions, and perform surgical operations based on the electrosurgical unit control instructions; the camera control module is used to analyze the received interactive information to obtain image acquisition instructions, and perform image acquisition based on the image acquisition instructions to obtain surgical images corresponding to the surgical operations, and send the surgical images to the imaging trolley 120.

[0051] The electrosurgical knife control instructions include: electrosurgical knife energy control instructions and electrosurgical knife position control instructions. The image acquisition instructions can be used to control the image acquisition frequency.

[0052] Specifically, the surgical robot may have multiple robotic arms. The end of each robotic arm may be equipped with instruments with different functions, such as an image collector and an electric knife. The electric knife control module analyzes the received interactive information to obtain an electric knife control instruction, and performs the surgical operation based on the electric knife control instruction. The camera control module analyzes the received interactive information to obtain an image acquisition instruction, and performs image acquisition based on the image acquisition instruction to obtain the surgical image corresponding to the surgical operation, and sends the surgical image to the imaging trolley 120. The second industrial computer 112 may also uniformly analyze the interactive information to obtain the electric knife control instruction and the image acquisition instruction, and send the electric knife control instruction to the electric knife control module, and send the image acquisition instruction to the camera control module.

[0053] On the basis of the above technical solution, specific connecting lines are used for communication between the first industrial computer 111 and the imaging trolley 120 , between the first industrial computer 111 and the second industrial computer 112 , and between the first industrial computer 111 and the second industrial computer 112 .

[0054] in, Figure 2 A diagram showing an example of the internal structure of a particular connection line. Figure 2, specific connecting wires include conductors, insulators, twisted pairs, aluminum heat strips, bottom lines, shielding tapes and sheaths. Specific connecting wires are composed of network cables and electronic wires; specific connecting wires are used for Ethernet communication, serial communication, IO communication and video communication. Ethernet communication can utilize Ethernet and high-performance real-time Ethernet communication protocols (Ethernet for Control Automation Technology, EtherCAT). Serial communication can utilize RS485 serial ports. High-quality connecting wires can reduce signal interference and attenuation, reduce the risk of system failures caused by connection problems, ensure the smooth progress of surgery, and further improve the reliability of the system. And through specific connecting wires, reasonable wiring can be carried out to reduce the confusion and entanglement of cables, making the operating room more tidy and orderly, reducing the inconvenience of operation and safety hazards caused by the clutter of cables, and further optimizing the layout of the surgical environment.

[0055] Specifically, the user triggers the interactive information through the foot pedal 130. The first industrial computer 111 transmits the interactive information to the imaging trolley 120 and the second industrial computer 112 through EtherCAT. The electrosurgical knife control module analyzes the received interactive information to obtain the electrosurgical knife control instruction, and performs the surgical operation based on the electrosurgical knife control instruction. For example, the excitation of monopolar or bipolar energy is controlled by the electrosurgical knife control line. The camera control module analyzes the received interactive information to obtain the image acquisition instruction, and performs image acquisition based on the image acquisition instruction to obtain the surgical image corresponding to the surgical operation, and sends the surgical image to the imaging trolley 120 through EtherCAT. The image processing module analyzes the received interactive information to obtain the information to be fused, and performs image processing based on the information to be fused and the surgical image to obtain the fused image, and sends the fused image to the display. The display displays the fused image. The system in the embodiment of the present invention adopts a ring network routing, so that the image to be displayed can be transmitted to the main display through one line routing, and the system control, image display and interaction between various components are realized through the ring network.

[0056] It should be noted that if there is an HDMI video signal, adding an optical fiber HDMI cable to the combination can also transmit the video signal. Therefore, through a reasonably designed connecting cable, related components can be easily added or replaced to adapt to different surgical needs and technological developments, further enhancing the flexibility and scalability of the system.

[0057] Embodiment 2

[0058] Figure 3A flowchart of a multi-component interaction method for endoscopic surgery is provided for the second embodiment of the present invention. This embodiment is applicable to the multi-component interaction of endoscopic surgery. The method can be executed by a multi-component interaction system for endoscopic surgery, and the multi-component interaction system for endoscopic surgery can be implemented in the form of hardware and / or software. The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here. Figure 3 As shown, the method may include the following steps:

[0059] S310, using the first industrial computer to collect interactive information triggered by the user's foot in real time, and sending the interactive information to the imaging trolley and the second industrial computer.

[0060] S320, controlling the instrument to perform surgical operation and image acquisition based on the received interactive information through the second industrial computer, obtaining surgical images corresponding to the surgical operation, and sending the surgical images to the imaging trolley.

[0061] S330 , performing image processing on the surgical image based on the received interactive information by the imaging trolley to obtain a fused image, and displaying the fused image.

[0062] The technical solution of the embodiment of the present invention uses the first industrial computer to collect the interactive information triggered by the user through foot pedal in real time, and sends the interactive information to the imaging trolley and the second industrial computer; uses the second industrial computer to control the instrument to perform surgical operations and image acquisition based on the received interactive information, obtains the surgical image corresponding to the surgical operation, and sends the surgical image to the imaging trolley; uses the imaging trolley to process the surgical image based on the received interactive information, obtains the fused image, and displays the fused image. In this way, the multi-component interaction of endoscopic surgery can be realized quickly and conveniently, avoiding multiple point-to-point interactive transmissions, and improving the interaction efficiency between components in the system.

[0063] On the basis of the above technical solution, the first industrial computer is connected to the foot pedal; the second industrial computer is connected to the instrument.

[0064] On the basis of the above technical solution, "performing image processing on the surgical image based on the received interactive information through the imaging trolley to obtain a fused image, and displaying the fused image" may include: analyzing the received interactive information through the image processing module to obtain the information to be fused, and performing image processing based on the information to be fused and the surgical image to obtain a fused image, and sending the fused image to the display; displaying the fused image through the display.

[0065] Among them, the imaging trolley includes: a display and an image processing module.

[0066] Based on the above technical solution, the method also includes: displaying current status data through a display; the current status data includes: at least one of indicator light status, joint position, joint current and current instrument information.

[0067] On the basis of the above technical solution, "controlling the instrument through the second industrial computer based on the received interactive information to perform surgical operations and image acquisition, obtaining surgical images corresponding to the surgical operations, and sending the surgical images to the imaging trolley" may include: analyzing the received interactive information through the electrosurgical knife control module to obtain electrosurgical knife control instructions, and performing surgical operations based on the electrosurgical knife control instructions; analyzing the received interactive information through the camera control module to obtain image acquisition instructions, and performing image acquisition based on the image acquisition instructions to obtain surgical images corresponding to the surgical operations, and sending the surgical images to the imaging trolley.

[0068] Wherein, the second industrial computer includes: an electric knife control module and a camera control module.

[0069] On the basis of the above technical solution, the electric knife control instructions include: electric knife energy control instructions and electric knife position control instructions.

[0070] On the basis of the above technical solution, specific connecting lines are used for communication between the first industrial computer and the imaging trolley, between the first industrial computer and the second industrial computer, and between the first industrial computer and the second industrial computer.

[0071] On the basis of the above technical solution, the method also includes: performing Ethernet communication, serial communication, IO communication and video communication through specific connection lines.

[0072] Among them, the specific connecting wires are composed of network cables and electronic wires.

[0073] On the basis of the above technical solution, the specific connecting wire includes a conductor, an insulator, a twisted pair, an aluminum thermal tape, a bottom tape, a shielding tape and a sheath.

[0074] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the multi-component interaction method for endoscopic surgery provided in any embodiment of the present application.

[0075] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0076] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0077] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-component interactive system for endoscopic surgery, characterized in that: The system includes: an industrial computer, an imaging trolley, a pedal and an instrument; the industrial computer includes: a first industrial computer and a second industrial computer; wherein, The first industrial computer is used to collect interactive information triggered by the user through the foot pedal in real time, and send the interactive information to the imaging trolley and the second industrial computer; The second industrial computer is used to control the instrument to perform surgical operation and image acquisition based on the received interactive information, obtain surgical images corresponding to the surgical operation, and send the surgical images to the imaging trolley; The imaging trolley is used to perform image processing on the surgical image based on the received interactive information to obtain a fused image, and then display the fused image.

2. The system according to claim 1, characterized in that The first industrial computer is connected to the foot pedal; the second industrial computer is connected to the device.

3. The system according to claim 1, characterized in that The imaging trolley comprises: a display and an image processing module; The image processing module is used to analyze the received interactive information to obtain information to be fused, perform image processing based on the information to be fused and the surgical image to obtain a fused image, and send the fused image to the display; The display is used to display the fused image.

4. The system according to claim 3, characterized in that The display is also used to display current status data; the current status data includes: at least one of indicator light status, joint position, joint current and current instrument information.

5. The system according to claim 1, characterized in that The second industrial computer includes: an electric knife control module and a camera control module; The electrosurgical unit control module is used to analyze the received interactive information to obtain an electrosurgical unit control instruction, and perform a surgical operation based on the electrosurgical unit control instruction; The camera control module is used to analyze the received interactive information to obtain an image acquisition instruction, and perform image acquisition based on the image acquisition instruction to obtain a surgical image corresponding to the surgical operation, and send the surgical image to the imaging trolley.

6. The system according to claim 5, characterized in that The electric knife control instructions include: electric knife energy control instructions and electric knife position control instructions.

7. The system according to claim 1, characterized in that Specific connection lines are used for communication between the first industrial computer and the imaging trolley, between the first industrial computer and the second industrial computer, and between the first industrial computer and the second industrial computer.

8. The system according to claim 7, characterized in that The specific connecting line is composed of a network cable and an electronic line; the specific connecting line is used for Ethernet communication, serial port communication, IO communication and video communication.

9. The system according to claim 7, characterized in that The specific connecting wire comprises a conductor, an insulator, a twisted pair, an aluminum thermal tape, a bottom wire, a shielding tape and a sheath.

10. A multi-component interaction method for endoscopic surgery, characterized in that: The method comprises: The first industrial computer collects interactive information triggered by the user's foot in real time, and sends the interactive information to the imaging trolley and the second industrial computer; Controlling the instrument to perform surgical operation and image acquisition based on the received interactive information by the second industrial computer, obtaining surgical images corresponding to the surgical operation, and sending the surgical images to the imaging trolley; The imaging trolley performs image processing on the surgical image based on the received interactive information to obtain a fused image, and displays the fused image.

Images