A multi-component interaction system and method for endoscopic surgery

Abstract

The embodiment of the application discloses a kind of multi-component interaction system and method of endoscopic surgery.The system includes industrial computer, image trolley, foot pedal and instrument;Industrial computer includes: first industrial computer and second industrial computer;Wherein, the first industrial computer is used to collect the interactive information triggered by user through foot pedal in real time, and sends interactive information to image trolley and second industrial computer;Second industrial computer is used to control instrument based on received interactive information to carry out surgical operation and image acquisition, obtain the surgical image corresponding to surgical operation, and send surgical image to image trolley;Image trolley is used to carry out image processing on surgical image based on received interactive information, obtain fusion image, and display fusion image.By the technical scheme of the embodiment of the application, multi-component interaction of endoscopic surgery can be quickly and conveniently realized, and the interaction efficiency between internal components of the system is improved.

Description

TECHNICAL FIELD

[0001] The embodiment of the application relates to the technical field of communication, in particular to a multi-component interaction system and method for endoscopic surgery. BACKGROUND

[0002] With the development of technology, more and more components are used in surgery, resulting in more and more complex interactions between components.

[0003] At present, a point-to-point separate interaction mode is usually used to realize independent interaction between components. However, the separate interaction mode reduces the interaction efficiency between components, and an interaction channel needs to be built between each component for transmitting interaction information, resulting in multiple different types of interaction channels between components, and causing wiring confusion and entanglement. SUMMARY

[0004] The embodiment of the application provides a multi-component interaction system and method for endoscopic surgery, so as to quickly and conveniently realize multi-component interaction of endoscopic surgery and improve the interaction efficiency between internal components of the system.

[0005] According to an aspect of the application, a multi-component interaction system for endoscopic surgery is provided, which comprises:

[0006] an industrial computer, an image trolley, a foot pedal and an instrument; the industrial computer comprises a first industrial computer and a second industrial computer;

[0007] The first industrial computer is configured to collect interaction information triggered by the foot pedal in real time, and send the interaction information to the image trolley and the second industrial computer.

[0008] The second industrial computer is configured to control the instrument to perform a surgical operation and image acquisition based on the received interaction information, obtain a surgical image corresponding to the surgical operation, and send the surgical image to the image trolley.

[0009] The image trolley is configured to perform image processing on the surgical image based on the received interaction information, obtain a fusion image, and display the fusion image.

[0010] Further, the first industrial computer is connected with the foot pedal, and the second industrial computer is connected with the instrument.

[0011] Further, the image trolley comprises a display and an image processing module.

[0012] The image processing module is configured to analyze the received interaction information to obtain to-be-fused information, perform image processing based on the to-be-fused information and the surgical image to obtain a fused image, and send the fused image to the display.

[0013] The display is configured to display the fused image.

[0014] Further, the display is further configured to display current state data; the current state data includes at least one of an indicator light state, a joint position, a joint current, and current instrument information.

[0015] Further, the second industrial computer includes an electrotome control module and a camera control module.

[0016] The electrotome control module is configured to analyze the received interaction information to obtain electrotome control instructions, and perform surgical operation based on the electrotome control instructions.

[0017] The camera control module is configured to analyze the received interaction information to obtain image acquisition instructions, perform image acquisition based on the image acquisition instructions, obtain surgical images corresponding to the surgical operation, and send the surgical images to the image cart.

[0018] Further, the electrotome control instructions include electrotome energy control instructions and electrotome position control instructions.

[0019] Further, the first industrial computer and the image cart, the first industrial computer and the second industrial computer, and the first industrial computer and the second industrial computer are connected by a specific connection line for communication.

[0020] Further, the specific connection line is composed of a network cable and an electronic cable; the specific connection line is used for Ethernet communication, serial communication, IO communication, and video communication.

[0021] Further, the specific connection line includes a conductor, an insulator, a twisted pair, an aluminum hot strip, a bottom line, a shielding tape, and a sheath.

[0022] According to another aspect of the present application, a multi-component interaction method for endoscopic surgery is provided, the method comprising:

[0023] The first industrial computer is configured to collect interaction information triggered by a user through a foot pedal in real time, and send the interaction information to the image cart and the second industrial computer.

[0024] The second industrial computer is configured to control instruments to perform surgical operation and image acquisition based on the received interaction information, obtain surgical images corresponding to the surgical operation, and send the surgical images to the image cart.

[0025] The image cart performs image processing on the surgical image based on the received interaction information, obtains a fusion image, and displays the fusion image.

[0026] According to another aspect of the present application, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the multi-component interaction method of endoscopic surgery according to any of the embodiments of the present application.

[0027] The technical solution of the embodiment of the present application uses the first industrial computer to collect the interaction information triggered by the user through the foot pedal in real time, and sends the interaction information to the image cart and the second industrial computer; uses the second industrial computer to control the surgical operation and image acquisition based on the received interaction information, obtains the surgical image corresponding to the surgical operation, and sends the surgical image to the image cart; uses the image cart to perform image processing on the surgical image based on the received interaction information, obtains a fusion image, and displays the fusion image. Thus, the multi-component interaction of endoscopic surgery can be quickly and conveniently realized, avoiding multiple point-to-point interaction transmissions, and improving the interaction efficiency between internal components of the system.

[0028] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0030] Figure 1 is a structural schematic diagram of a multi-component interaction system of endoscopic surgery according to embodiment one of the present application;

[0031] Figure 2 is an example diagram of the internal structure of a specific connection line according to embodiment one of the present application;

[0032] Figure 3 is a flowchart of a multi-component interaction method of endoscopic surgery according to embodiment two of the present application. DETAILED DESCRIPTION

[0033] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should be within the scope of protection of the present application.

[0034] It should be noted that the terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in other than the order illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to include all the steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0035] Embodiment one

[0036] Figure 1 A structural schematic diagram of an endoscopic surgery multi-component interaction system is provided for the first embodiment of the present application. The components shown herein, their connections and relationships, and their functions are only as examples, and are not intended to limit the implementation of the present application described and / or claimed herein.

[0037] As Figure 1 shown, the endoscopic surgery multi-component interaction system can include an industrial computer 110, an image trolley 120, a foot pedal 130 and an instrument 140; the industrial computer 110 includes a first industrial computer 111 and a second industrial computer 112.

[0038] The first industrial computer 111 is configured to collect the interaction information triggered by the user through the foot pedal 130 in real time, and send the interaction information to the image trolley 120 and the second industrial computer 112; the second industrial computer 112 is configured to control the instrument 140 to perform a surgical operation and image acquisition based on the received interaction information, obtain a surgical image corresponding to the surgical operation, and send the surgical image to the image trolley 120; the image trolley 120 is configured to perform image processing on the surgical image based on the received interaction information, obtain a fusion image, and display the fusion image.

[0039] The foot pedal 130 can be used to generate different interaction information according to the triggering mode of the user. The interaction information can refer to information for controlling the corresponding actions of various components in the system. The interaction information can include, but is not limited to, to-be-fused information, electrotome control instructions, and image acquisition instructions. The surgical image can refer to a real-time image during the surgical operation. The continuous surgical images can constitute a surgical video. The fused image can refer to an image obtained by fusing the to-be-fused information and the surgical image.

[0040] Specifically, the user generates interaction information by controlling the foot pedal 130. The first industrial computer 111 acquires the generated interaction information in real time, and sends the interaction information meeting the communication condition of the image cart to the image cart 120, and sends the interaction information meeting the communication condition of the second industrial computer to the second industrial computer 112. The second industrial computer 112 analyzes the received interaction information to obtain an analysis result, and controls the surgical instrument 140 to perform a surgical operation and acquires an image of the surgical operation based on the analysis result, obtains a surgical image corresponding to the surgical operation, and sends the surgical image to the image cart 120. The image cart 120 processes the surgical image based on the received interaction information to obtain a fused image, and displays the fused image.

[0041] The communication condition can refer to the communication protocol between two communication components. In the embodiment of the present application, the pre-established communication protocol can be used to filter transmittable information, so that when the same connection line is used between various components, the non-transmittable information can be effectively filtered, and uniform hardware connection deployment can be performed, thereby improving the wiring efficiency of the connection line between various components in the system. The connection line can ensure that various key signals such as image signals (surgical images), control signals (electrotome control instructions), and data signals (to-be-fused information) are accurately and stably transmitted between different system components, so that the doctor can obtain a clear surgical field of view and accurately control the surgical instrument 140.

[0042] It should be noted that if the received interaction information needs to be standardized, the first industrial computer 111 can standardize the interaction information when receiving the interaction information to obtain standard interaction information, and send the standard interaction information to the image cart 120 and the second industrial computer 112. In the single-industrial-computer mode, the standard wiring harness (connection line) is 3 in series. In the dual-industrial-computer mode, the first industrial computer 111 and the second industrial computer 112 are connected in series to form a ring network, and the standard wiring harness is increased to 4.

[0043] The technical scheme of the embodiment of the present application acquires the interaction information triggered by the user through the foot pedal 130 in real time by using the first industrial computer 111, and sends the interaction information to the image trolley 120 and the second industrial computer 112; controls the surgical operation and image acquisition of the instrument 140 based on the received interaction information by using the second industrial computer 112, obtains the surgical image corresponding to the surgical operation, and sends the surgical image to the image trolley 120; and performs image processing on the surgical image based on the received interaction information by using the image trolley 120, obtains the fusion image, and displays the fusion image. Therefore, the multi-component interaction of the endoscopic surgery can be quickly and conveniently realized, the point-to-point multiple interaction transmission is avoided, and the interaction efficiency between the internal components of the system is improved.

[0044] On the basis of the above technical scheme, the first industrial computer 111 is connected with the foot pedal 130, and the second industrial computer 112 is connected with the instrument 140. The advantage of this arrangement is that multiple industrial computers can be used to cooperatively complete the surgical operation, the single industrial computer is prevented from performing complex processing in multiple dimensions, the cooperative work of the system is ensured, and therefore the surgical operation efficiency is improved. For example, the system can connect the endoscope, the light source, the image processing device, the operation console and other components, promote the cooperative operation between them, and therefore realize the efficient operation of the entire surgical system.

[0045] On the basis of the above technical scheme, the image trolley 120 includes a display and an image processing module; the image processing module is configured to analyze the received interaction information to obtain the to-be-fused information, perform image processing based on the to-be-fused information and the surgical image to obtain the fusion image, and send the fusion image to the display; and the display is configured to display the fusion image.

[0046] The to-be-fused information can be the to-be-displayed annotation information. For example, the to-be-fused information can be but is not limited to the system information to be displayed, such as a system number. The fusion image can be an image with annotation information obtained by fusing the to-be-fused information and the surgical image.

[0047] Specifically, the image stream mainly includes two parts, one part is the 3D / 2D surgical image (original video data) collected, and the other part is the to-be-fused information (video superposition control). The to-be-fused information to be displayed is superimposed and fused with the collected surgical image, and the fusion image is output to the 3D display.

[0048] On the basis of the above technical scheme, the display is further configured to display current state data; the current state data includes at least one of a indicator light state, a joint position, a joint current and current instrument information.

[0049] The control software sends current state data to the information interaction software through a tcp / ip protocol, so that the current state data is transmitted as interaction information and is transmitted to a display through an existing connection line between components for display.

[0050] On the basis of the above technical solution, the second industrial computer 112 comprises: an electrotome control module and a camera control module; the electrotome control module is configured to analyze the received interaction information to obtain an electrotome control instruction and perform a surgical operation based on the electrotome control instruction; and the camera control module is configured to analyze the received interaction information to obtain an image acquisition instruction and perform image acquisition based on the image acquisition instruction, to obtain a surgical image corresponding to the surgical operation, and send the surgical image to the video cart 120.

[0051] The electrotome control instruction comprises an electrotome energy control instruction and an electrotome position control instruction. The image acquisition instruction can be used to control an image acquisition frequency.

[0052] Specifically, the surgical robot can have multiple mechanical arms. Each mechanical arm can be equipped with an instrument with different functions, such as an image acquisition device and an electrotome. The electrotome control module analyzes the received interaction information to obtain an electrotome control instruction and performs a surgical operation based on the electrotome control instruction. The camera control module analyzes the received interaction information to obtain an image acquisition instruction and performs image acquisition based on the image acquisition instruction, to obtain a surgical image corresponding to the surgical operation, and sends the surgical image to the video cart 120. The second industrial computer 112 can also analyze the interaction information to obtain the electrotome control instruction and the image acquisition instruction, and send the electrotome control instruction to the electrotome control module and the image acquisition instruction to the camera control module.

[0053] On the basis of the above technical solution, the first industrial computer 111 and the video cart 120, the first industrial computer 111 and the second industrial computer 112, and the first industrial computer 111 and the second industrial computer 112 all communicate through specific connection lines.

[0054] The specific connection line comprises a first connection line, a second connection line, and a third connection line. Figure 2 An example of the internal structure of the specific connection line is given. Referring to Figure 2The specific connection line includes a conductor, an insulator, a twisted pair, an aluminum hot band, a bottom line, a shielding band, and a sheath. The specific connection line is composed of a network cable and an electronic cable; and the specific connection line is used for Ethernet communication, serial communication, IO communication, and video communication. The Ethernet communication can utilize an Ethernet and an Ethernet for Control Automation Technology (EtherCAT) high-performance real-time Ethernet communication protocol. The serial communication can utilize an RS485 serial port. The high-quality connection line can reduce signal interference and attenuation, reduce the risk of system failure caused by connection problems, ensure the smooth progress of the operation, and further improve the reliability of the system. Moreover, through the specific connection line, the wiring can be reasonably arranged to reduce the disorder and entanglement of the cables, make the operating room more tidy and orderly, reduce the inconvenience and safety hazards caused by the disorder of the cables, and further optimize the layout of the operating environment.

[0055] Specifically, the user triggers the interaction information through the foot pedal 130. The first industrial computer 111 transmits the interaction information to the image trolley 120 and the second industrial computer 112 through EtherCAT. The electrotome control module analyzes the received interaction information to obtain an electrotome control instruction, and performs surgical operation based on the electrotome control instruction. For example, the excitation of monopolar or bipolar energy is controlled through the electrotome control line. The camera control module analyzes the received interaction information to obtain an image acquisition instruction, and performs image acquisition based on the image acquisition instruction to obtain a surgical image corresponding to the surgical operation, and transmits the surgical image to the image trolley 120 through EtherCAT. The image processing module analyzes the received interaction information to obtain to-be-fused information, and performs image processing based on the to-be-fused information and the surgical image to obtain a fused image, and transmits the fused image to the display. The display displays the fused image. The system in the embodiment of the present application adopts ring network wiring, so that the image to be displayed can be transmitted to the main display through one wiring, and system control, image display, and interaction between components are realized through the ring network.

[0056] It should be noted that if there is an HDMI video signal, an optical fiber HDMI line is added for combination, and the transmission of the video signal can also be performed, so that through the reasonably designed connection line, the related components can be conveniently added or replaced to adapt to different surgical needs and technical development, and the flexibility and expandability of the system are further enhanced.

[0057] Embodiment two

[0058] Figure 3A flowchart of a multi-component interaction method of an endoscopic surgery is provided for the second embodiment of the present application, which can be applicable to the multi-component interaction of an endoscopic surgery. The method can be executed by a multi-component interaction system of an endoscopic surgery, which can be realized in the form of hardware and / or software. The explanations of the same or corresponding terms as those in the above embodiments are not repeated here. As shown in FIG. 12, Figure 3 The method can include the following steps:

[0059] S310, collecting, by the first industrial computer, the interaction information triggered by the user through the foot pedal in real time, and sending the interaction information to the image cart and the second industrial computer.

[0060] S320, controlling, by the second industrial computer, the instrument to perform the surgical operation and image acquisition based on the received interaction information, obtaining the surgical image corresponding to the surgical operation, and sending the surgical image to the image cart.

[0061] S330, performing, by the image cart, image processing on the surgical image based on the received interaction information, obtaining the fusion image, and displaying the fusion image.

[0062] The technical scheme of the embodiment of the present application can quickly and conveniently realize the multi-component interaction of the endoscopic surgery, avoid multiple interaction transmissions between points, and improve the interaction efficiency between internal components of the system.

[0063] On the basis of the above technical scheme, the first industrial computer is connected with the foot pedal, and the second industrial computer is connected with the instrument.

[0064] On the basis of the above technical scheme, the image processing on the surgical image based on the received interaction information by the image cart to obtain the fusion image and display the fusion image can include: analyzing, by the image processing module, the received interaction information to obtain the to-be-fused information, performing image processing based on the to-be-fused information and the surgical image to obtain the fusion image, and sending the fusion image to the display; and displaying, by the display, the fusion image.

[0065] The image cart includes a display and an image processing module.

[0066] On the basis of the above technical scheme, the method further comprises: displaying the current state data through the display; and the current state data comprises at least one of the indicator light state, the joint position, the joint current and the current instrument information.

[0067] On the basis of the above technical scheme, the method further comprises: displaying the current state data through the display; and the current state data comprises at least one of the indicator light state, the joint position, the joint current and the current instrument information.

[0068] On the basis of the above technical scheme, the method further comprises: displaying the current state data through the display; and the current state data comprises at least one of the indicator light state, the joint position, the joint current and the current instrument information.

[0069] On the basis of the above technical scheme, the method further comprises: displaying the current state data through the display; and the current state data comprises at least one of the indicator light state, the joint position, the joint current and the current instrument information.

[0070] On the basis of the above technical scheme, the method further comprises: displaying the current state data through the display; and the current state data comprises at least one of the indicator light state, the joint position, the joint current and the current instrument information.

[0071] On the basis of the above technical scheme, the method further comprises: displaying the current state data through the display; and the current state data comprises at least one of the indicator light state, the joint position, the joint current and the current instrument information.

[0072] On the basis of the above technical scheme, the method further comprises: displaying the current state data through the display; and the current state data comprises at least one of the indicator light state, the joint position, the joint current and the current instrument information.

[0073] On the basis of the above technical scheme, the method further comprises: displaying the current state data through the display; and the current state data comprises at least one of the indicator light state, the joint position, the joint current and the current instrument information.

[0074] The embodiment of the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the multi-component interaction method for endoscopic surgery provided in any embodiment of the present application.

[0075] The computer program code can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce the computer implemented process such that the

[0076] It should be understood that the various forms of flow shown in the figures are illustrative examples of implementing the steps of the application. Several steps have been described as being performed by a single device. It will be understood that these steps can be performed by a single device or multiple devices, and that the steps can be performed in an order different from that shown in the figures. For example, the steps described in the figures can be performed in parallel or in a different order, as long as the desired results of the application are achieved. The application is not limited in this regard.

[0077] The specific embodiments have been shown and described for the purposes of illustrating the physiological principles of the application and its practical application. It is therefore to be understood that various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the application. The scope of the application is not to be limited by specific illustrative embodiments. The application is to cover any and all modifications and the same is therefore intended to be within the scope of the application.

Claims

1. A multi-component interactive system for endoscopic surgery, characterized in that, The system includes: an industrial control computer, an image trolley, a foot pedal, and instruments; the industrial control computer includes: a first industrial control computer and a second industrial control computer; wherein... The first industrial control computer is used to collect interactive information triggered by the user through the foot pedal in real time, and send the interactive information to the image trolley and the second industrial control computer; The second industrial control computer is used to control the instrument to perform surgical operations and image acquisition based on the received interactive information, obtain surgical images corresponding to the surgical operations, and send the surgical images to the imaging trolley; The imaging trolley is used to process the surgical images based on the received interactive information to obtain a fused image, and then display the fused image. Specifically, the first industrial control computer communicates with the image trolley, the first industrial control computer communicates with the second industrial control computer, and the image trolley communicates with the second industrial control computer using specific connecting lines. The specific connecting lines consist of network cables and electronic wires. The specific connecting lines are used for Ethernet communication, serial communication, I / O communication, and video communication. The specific connecting lines include conductors, insulators, twisted pairs, aluminum heating tape, ground wire, shielding tape, and sheath.

2. The system according to claim 1, characterized in that, The first industrial control computer is connected to the foot pedal; the second industrial control computer is connected to the machine.

3. The system according to claim 1, characterized in that, The image trolley includes: a display and an image processing module; The image processing module is used to analyze the received interactive information to obtain information to be fused, and to perform image processing based on the information to be fused and the surgical image to obtain a fused image, and to send the fused image to the display. The display is used to show the fused image.

4. The system according to claim 3, characterized in that, The display is also used to display current status data; the current status data includes at least one of: indicator light status, joint position, joint current, and current device information.

5. The system according to claim 1, characterized in that, The second industrial control computer includes: an electric knife control module and a camera control module; The electrosurgical control module is used to analyze the received interactive information to obtain electrosurgical control commands, and to perform surgical operations based on the electrosurgical control commands; The camera control module is used to analyze the received interactive information to obtain image acquisition instructions, and to acquire images based on the image acquisition instructions to obtain surgical images corresponding to the surgical operation, and then send the surgical images to the imaging trolley.

6. The system according to claim 5, characterized in that, The electrosurgical control commands include: electrosurgical energy control commands and electrosurgical position control commands.

7. A computer program product comprising a computer program that, when executed by a processor, implements a multi-component interaction method for endoscopic surgery based on the multi-component interaction system for endoscopic surgery according to claim 1, characterized in that... The method includes: The first industrial control computer collects the interactive information triggered by the user's foot pedal in real time, and sends the interactive information to the image trolley and the second industrial control computer. The second industrial control computer controls the instruments to perform surgical operations and image acquisition based on the received interactive information, obtains the surgical images corresponding to the surgical operations, and sends the surgical images to the imaging trolley; The imaging trolley processes the surgical images based on the received interactive information to obtain a fused image, which is then displayed.

Images