Bandwidth adjustment method, system and storage medium of remote surgical robot

Abstract

The present application discloses a bandwidth adjustment method, system and storage medium of a remote surgical robot, including: a first remote host and a second remote host with a network channel; a first terminal and a second terminal, one end of which is a remote doctor console and the other end is a patient surgery platform; a central server; the bandwidth adjustment method includes: in response to a first interactive operation performed by the first terminal on the second terminal, generating first interactive data, and transmitting it to the second remote host through the network channel by the first remote host, so that the second terminal can perform motion control based on the first interactive data; identifying a first data type of the first interactive data transmitted in the network channel through at least one of the first remote host and the second remote host; and adjusting the bandwidth of the network channel by the central server according to the first data type. The present invention realizes the adaptive dynamic adjustment of the required network bandwidth according to the data type transmitted in the network channel during the remote surgery execution process.

Description

Technical Field

[0001] The present invention belongs to the field of medical devices, and in particular relates to a bandwidth adjustment method, system and storage medium for a remote surgical robot. Background Art

[0002] With the continuous improvement of remote surgical robots and network infrastructure, remote surgery using surgical robots has become a mature surgical technique in hospitals. Patients' acceptance of remote surgery is also increasing, and the market demand for remote surgical robots is also increasing. Remote surgery is highly dependent on network communication, and network latency and instability can directly affect the quality and effectiveness of the surgery.

[0003] On the other hand, the network bandwidth of remote surgical robots typically requires a certain size, such as at least 60M. This is especially true for applications such as remote diagnosis and surgery, where real-time performance requirements are very strict and higher bandwidth may be required. Different remote surgical scenarios have different network bandwidth requirements, and existing remote surgical systems cannot dynamically adjust the network bandwidth required for real-time remote surgery based on the operations performed during the operation. Summary of the Invention

[0004] To address the problem that existing remote surgery cannot adaptively and dynamically adjust the required network bandwidth based on the operations during the surgery, the present invention proposes the following technical solutions:

[0005] A first aspect of the present invention provides a bandwidth adjustment method for a remote surgical robot, comprising: a first remote host and a second remote host, a network channel being established between the first remote host and the second remote host; a remote surgical robot, comprising a first terminal connected to the first remote host, and a second terminal connected to the second remote host, one end of the first terminal and the second terminal being a remote doctor console and the other end being a patient surgical platform; a central server, connected to at least one of the first remote host and the second remote host; the bandwidth adjustment method comprises: generating first interaction data in response to a first interaction operation performed by the first terminal on the second terminal, and transmitting the first interaction data to the second remote host via the network channel through the first remote host, so that the second terminal can perform motion control based on the first interaction data; identifying a first data type of the first interaction data transmitted in the network channel through at least one of the first remote host and the second remote host; and adjusting the bandwidth of the network channel through the central server according to the first data type.

[0006] Preferably, the central server is communicatively connected to a preset network operator server; according to the first data type, the bandwidth of the network channel is adjusted through the central server, including: matching the bandwidth parameters corresponding to the first data type through at least one end of the first remote host and the second remote host, and sending a first bandwidth adjustment request to the central server based on the bandwidth parameters; through the central server, based on the first bandwidth adjustment request, initiating a second bandwidth adjustment request to the network operator server, so as to adjust the bandwidth of the network channel through the network operator server.

[0007] Preferably, the first interaction data includes multiple first data types, and the multiple first data types are preset with data type priorities; matching the bandwidth parameters corresponding to the first data types through at least one end of the first remote host and the second remote host includes: selecting at least one first data type from the multiple first data types according to the data type priority through at least one end of the first remote host and the second remote host, and matching the corresponding bandwidth parameters according to the selected first data type.

[0008] Preferably, at least one of the first terminal and the second terminal stores mapping information, and the bandwidth parameter includes multiple bandwidth levels in order of bandwidth, and the mapping information includes: a mapping relationship between multiple first data types and multiple bandwidth levels; based on the bandwidth parameter, sending a first bandwidth adjustment request to the central server, including: through at least one of the first remote host and the second remote host, matching the target bandwidth level corresponding to the first data type in the mapping information stored in the corresponding first terminal and the second terminal, and initiating a first bandwidth adjustment request to the central server based on the target bandwidth level.

[0009] Preferably, the bandwidth of the network channel is adjusted through the central server according to the first data type, including: obtaining historical interaction data types transmitted in the network channel through at least one end of the first remote host and the second remote host, and comparing the first data type with the historical interaction data type to determine the switching status of the first data type transmitted in the network channel; according to the switching status, the bandwidth of the network channel is adjusted through the central server.

[0010] Preferably, the switching state includes a first switching state and a second switching state, wherein the first switching state includes a state of switching data types under different surgical stages or surgical tasks, and the second switching state includes a state of switching data types under the same surgical stage or surgical task; according to the switching state, the bandwidth of the network channel is adjusted by the central server, including: monitoring the duration of the first switching state, and when the duration exceeds a preset first duration, adjusting the bandwidth of the network channel according to a preset first amplitude; or, monitoring the duration of the second switching state, and when the duration exceeds a preset second duration, adjusting the bandwidth of the network channel according to a preset second amplitude; wherein the first amplitude is greater than the second amplitude, and the preset first duration is less than the preset second duration.

[0011] Preferably, the surgical stage includes at least any one of the following: power-on self-test, remote pairing, remote connection, remote interactive communication, remote surgical operation, remote pairing release, and power-off self-test; the surgical task includes any one of the following: surgical operation task, teaching task, and debugging task.

[0012] Preferably, it also includes at least one auxiliary surgical device, and at least one end of the first terminal and the second terminal is detachably connected to at least one auxiliary surgical device; according to the first data type, the bandwidth of the network channel is adjusted by the central server, including: in response to the second interactive operation performed by the first terminal or the second terminal with the auxiliary surgical device, second interactive data is generated, and transmitted between the first remote host and the second remote host via the network channel; through at least one of the first remote host and the second remote host, the second data type of the second interactive data transmitted in the network channel is identified, and according to the preset relative type priority between the first data type and the second data type, one of the data types is selected from the first data type and the second data type; according to the selected first data type or second data type, the bandwidth of the network channel is adjusted by the central server.

[0013] Preferably, the bandwidth of the network channel is adjusted through the central server according to the first data type, including: determining whether the first data type contains a preset target data type; detecting the configuration information of the first interactive data of the preset target data type through at least one of the first remote host and the second remote host; and adjusting the bandwidth of the network channel through the central server according to the configuration information.

[0014] Preferably, the first interaction data includes: third interaction data transmitted from the remote doctor console to the patient surgery platform, and fourth interaction data sent from the patient surgery platform to the doctor console, and a master-slave data type priority is preset between the third data type of the third interaction data and the fourth data type of the fourth interaction data; the central server is respectively communicated with at least one end of the remote doctor console and the patient surgery platform; according to the first data type, the bandwidth of the network channel is adjusted by the central server, including: according to the master-slave data type priority, selecting one of the data types from the third data type and the fourth data type; according to the selected third data type or fourth data type, the bandwidth of the network channel is adjusted by the central server.

[0015] A second aspect of the present invention provides a bandwidth adjustment system for a remote surgical robot, the bandwidth adjustment system comprising: a first remote host and a second remote host, a network channel being established between the first remote host and the second remote host; a remote surgical robot comprising a first terminal connected to the first remote host, and a second terminal connected to the second remote host, one end of the first terminal and the second terminal being a remote doctor console and the other end being a patient surgical platform; a central server connected to at least one of the first remote host and the second remote host; a processor configured to: generate first interaction data in response to a first interaction operation performed by the first terminal on the second terminal, and transmit the first interaction data to the second remote host via the network channel through the first remote host, so that the second terminal can perform motion control based on the first interaction data; identify a first data type of the first interaction data transmitted in the network channel through at least one of the first remote host and the second remote host; and adjust the bandwidth of the network channel through the central server according to the first data type.

[0016] A third aspect of the present invention provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned bandwidth adjustment method for a remote surgical robot are implemented.

[0017] The beneficial effects of the present invention are: the patient surgical platform and the remote doctor console in the remote surgical robot will involve the transmission of multiple data types during the interactive operation of remote surgery to complete remote master-slave control; depending on the type of data transmitted, it can represent the actual surgical scene of the remote surgical robot, such as before, during, or after the operation; based on the data type, the required network bandwidth can be adaptively adjusted. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1is a system schematic diagram of an embodiment of a remote surgical robot according to the present invention;

[0019] Figure 2 It is a structural schematic diagram of an embodiment of a surgical instrument of the present invention;

[0020] Figure 3 is a system schematic diagram of another embodiment of the remote surgical robot of the present invention;

[0021] Figure 4 This is the first timing diagram of the network communication bandwidth adjustment of the present invention;

[0022] Figure 5 This is a second timing diagram of the network communication bandwidth adjustment of the present invention;

[0023] Figure 6 Schematic diagram of bandwidth allocation for multiple groups of remote surgical robots according to the present invention;

[0024] Figure 7 This is a third timing diagram of the network communication bandwidth adjustment of the present invention;

[0025] Figure 8 This is a first flow chart of the bandwidth adjustment method of the present invention;

[0026] Figure 9 This is a schematic diagram of the interactive operation of the remote surgical robot of the present invention;

[0027] Figure 10 It is a simple schematic diagram of the connection between the remote surgical robot and the auxiliary surgical device of the present invention;

[0028] Figure 11 Schematic diagram of a preferred embodiment of the bandwidth adjustment method of the present invention;

[0029] Figure 12 This is the fourth timing diagram of the network communication bandwidth adjustment of the present invention;

[0030] Figure 13 is a second flow chart of the bandwidth adjustment method of the present invention;

[0031] Figure 14 This is the fifth timing diagram of the network communication bandwidth adjustment of the present invention;

[0032] Figure 15 3 is a third flow chart of the bandwidth adjustment method of the present invention;

[0033] Figure 16 This is the sixth timing diagram of network communication bandwidth adjustment according to the present invention;

[0034] Figure 17 It is a flow chart of the bandwidth allocation method of the present invention. DETAILED DESCRIPTION

[0035] To facilitate understanding of the present invention, the present invention will be described in more detail below with reference to the accompanying drawings and specific embodiments. Preferred embodiments of the present invention are shown in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in this specification. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the disclosure of the present invention.

[0036] It should be noted that, unless otherwise defined, all technical and scientific terms used in this specification have the same meanings as those understood by those skilled in the art to which this invention belongs. The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the invention. For example, the term "plurality" includes two or more.

[0037] 1. Remote surgical robots

[0038] Robotic telesurgery technology uses information networks as a carrier for the transmission of medical information. It employs a master-slave approach, combining robotics, virtual reality, artificial intelligence, and computer technologies for remote diagnosis, guidance, and operation of medical surgeries. This allows remote doctors to control surgical robots via information networks and perform surgeries on remote patients. A telesurgery robot consists of a master and slave device: the master operating device is the remote doctor's console, and the slave operating device is the patient's surgical platform. The doctor performs operations on the remote doctor's console and sends control commands to the slave operating device, controlling the patient's surgical platform to perform the surgery.

[0039] The doctor and the remote doctor console are located in a first geographical location, and the patient and the patient surgery platform are located in a second geographical location; because the first geographical location and the second geographical location are usually far apart, such as in different hospitals, buildings, regions, countries, etc., the remote doctor console and the patient surgery platform transmit surgical process data through an information network channel, such as master-slave control signaling, endoscopic video data, audio data, tactile feedback data, pairing data and other master-slave device interaction data.

[0040] Specifically, the remote doctor console is provided with a first remote host, and the patient surgery platform is provided with a second remote host. The remote doctor console and the patient surgery platform are connected to the information network channel through the first remote host and the second remote host respectively to realize the transmission of the surgical process data.

[0041] Remote surgical robots can specifically include: laparoscopic surgical robots (single-hole, multi-hole), natural cavity surgical robots, percutaneous puncture surgical robots, orthopedic surgical robots, pan-vascular surgical robots, dental surgical robots, neurosurgery surgical robots, etc., including the aforementioned remote doctor console and patient surgical platform, so that remote surgical technology can be realized.

[0042] For example, Figure 1 and Figure 2 As shown, the laparoscopic surgical robot (single-port) includes a remote physician console 100 and a patient surgical platform 200. The remote physician console 100 is used to send control commands to the patient surgical platform 200 based on the physician's operations to control the patient surgical platform 200. It is also used to display images captured by the patient surgical platform 200. The patient surgical platform 200 responds to the control commands sent by the remote physician console 100 and performs corresponding operations. The patient surgical platform 200 is also used to capture images within the body.

[0043] The patient surgical platform 200 includes a robotic arm 210, a power mechanism 220 mounted on the robotic arm 210, a surgical instrument 230 mounted on the power mechanism 220, and a sleeve 240 that encases the long shaft 231 of the surgical instrument 230. When the patient surgical platform 200 responds to control commands from the remote physician console 100, the robotic arm 210 is used to adjust the position of the surgical instrument 230, the power mechanism 220 is used to drive the surgical instrument 230 to perform the corresponding operation, and the end effector 232 of the surgical instrument 230 is used to extend into the body and, through its distal end instrument, perform surgical operations and / or acquire in-vivo images.

[0044] For example, Figure 3 A natural orifice surgical robot 300 is shown. The natural orifice surgical robot 300 includes a remote doctor console and a patient surgical platform 320. The remote doctor console includes a handle 310 and an imaging cart 330 that are interconnected, and / or the patient surgical platform 320 also includes an imaging cart 330. The patient surgical platform 320 is connected to a catheter instrument 340, and is connected to a sensor system 350, as well as a control system 360 for achieving control between the catheter instrument 340, the sensor system 350, and the imaging cart 330. When a doctor performs various procedures on a patient next to the patient surgical platform 320, he or she can trigger control instructions by operating the handle 310 and sending them to the patient surgical platform 320 for driving, thereby controlling the catheter instrument 340 to advance, retract, bend, turn, etc.

[0045] The patient surgical platform 320 can typically be moved to the side of the operating table to engage the catheter instrument 340 and, under control commands, control the catheter instrument 340 to move vertically, horizontally, or in non-vertical and non-horizontal directions, thereby providing a better preoperative preparation angle for the operation of the catheter instrument 340. The control commands can be triggered by the doctor operating the patient surgical platform 320, or by the doctor directly clicking or pressing a button on the patient surgical platform 320. Of course, in other embodiments, the control commands can also be voice control or commands triggered by a force feedback mechanism.

[0046] like Figure 3 As shown, the patient surgical platform 320 may further include a base 321, a sliding base 322 that can be raised and lowered along the base 321, and two robotic arms 323 fixedly connected to the sliding base 322. The robotic arms 323 may include multiple arm segments connected at joints, each of which provides the robotic arms 323 with multiple degrees of freedom, for example, seven degrees of freedom corresponding to the seven arm segments. A power unit (not shown) is mounted at the distal end of the robotic arm 323. The power unit is used to engage the catheter instrument 340 and, under the power of the power unit, controls the distal end of the catheter instrument 340 to bend and turn accordingly. The two robotic arms 323 may have identical or partially identical structures, with one robotic arm 323 used to engage the inner catheter instrument 341 and the other robotic arm 323 used to engage the outer catheter instrument 342. During installation, the outer catheter device 342 may be installed first, and after the outer catheter device 342 is installed, the catheter of the inner catheter device 341 may be inserted into the catheter of the outer catheter device 342 .

[0047] The sensor system 350 has one or more subsystems for receiving information about the catheter device 340. The subsystems may include: a position sensor system; a shape sensor system for determining the position, orientation, speed, velocity, pose, and / or shape of the tip of the catheter device 340 and / or one or more segments along a catheter that may comprise the catheter device 340; and / or a visualization system for capturing images from the tip of the catheter device 340.

[0048] The imaging cart 330 may be equipped with a display system 331 and an irrigation system (not shown). The display system 331 is used to display images or representations of the surgical site and catheter instruments 340 generated by the subsystems of the sensor system 350. Real-time images of the surgical site and catheter instruments 340 captured by the visualization system may also be displayed. Pre-operative or intra-operative images of the surgical site may also be presented using image data from imaging technologies such as computed tomography (CT), magnetic resonance imaging (MRI), optical coherence tomography (OCT), and ultrasound.

[0049] The preoperative or intraoperative image data can be presented as a two-dimensional, three-dimensional, or four-dimensional (e.g., time-based or rate-based) image and / or as an image from a model created based on the preoperative or intraoperative image data set. A virtual navigation image can also be displayed in which the actual position of the catheter device 340 is registered with the preoperative image to present a virtual image of the catheter device 340 within the surgical site to the operator from the outside.

[0050] The control system 360 includes at least one memory and at least one processor. It is understood that the control system 360 can be integrated into the patient surgical platform 320 or the imaging cart 330, or can be independently provided. The control system 360 can support wireless communication protocols such as IEEE 802.11, IrDA, Bluetooth, HomeRF, DECT, and wireless telemetry. The control system 360 can transmit one or more signals instructing the catheter instrument 340 to move, which are used by the power unit to move the catheter instrument 340. The catheter instrument 340 can extend to the surgical location in the body through the opening of the patient's natural cavity or surgical incision.

[0051] Furthermore, the control system 360 may include a mechanical control system (not shown) and an image processing system (not shown). The mechanical control system is used to control the movement of the catheter instrument 340 and, therefore, may be integrated into the patient surgical platform 320. The image processing system is used for virtual navigation path planning and, therefore, may be integrated into the imaging vehicle 330. Of course, the various subsystems of the control system 360 are not limited to the specific ones listed above and can be reasonably configured according to actual circumstances.

[0052] The image processing system can use the aforementioned imaging techniques to image the surgical site based on preoperative or intraoperative images of the surgical site. Software used in conjunction with manual input can also be used to convert the recorded images into two-dimensional or three-dimensional composite images of a portion or entire anatomical organ or region. During a virtual navigation procedure, the sensor system 350 can be used to calculate the position of the catheter instrument 340 relative to the patient's anatomy. This position can be used to generate external tracking images and internal virtual images of the patient's anatomy, achieving registration of the actual position of the catheter instrument 340 with the preoperative images, thereby allowing a virtual image of the catheter instrument 340 within the surgical site to be presented to the operator from the outside.

[0053] The inner catheter device 341 and the outer catheter device 342 have substantially the same structural composition, and each comprises a slender and flexible inner catheter 41 and an outer catheter 42, wherein the diameter of the outer catheter 42 is slightly larger than that of the inner catheter 41, so that the inner catheter 41 can pass through the outer catheter 42 and provide a certain degree of support for the inner catheter 41, so that the inner catheter 41 can reach the target location in the patient's body, so as to facilitate operations such as tissue or cell sampling from the target location.

[0054] Certain movements of the handle 310 can cause corresponding movements of the catheter instrument 340. For example, when a doctor operates the directional lever of the handle 310 to move upward or downward, the movement of the directional lever of the handle 310 can be mapped to a corresponding pitch movement of the distal end of the catheter instrument 340. When the doctor operates the directional lever of the handle 310 to move left or right, the movement of the directional lever of the handle 310 can be mapped to a corresponding yaw movement of the distal end of the catheter instrument 340. In this embodiment, the handle 310 can control the movement of the distal end of the catheter instrument 340 within a 360-degree spatial range.

[0055] In addition to their own equipment, the remote physician console and patient surgical platform are also equipped with various auxiliary surgical devices, such as the aforementioned surgical instruments, catheter instruments, irrigation systems, and operating tables, as well as puncture cards, energy platforms, monitors, fluid suction systems, sterile drapes, and pneumoperitoneum machines. The patient surgical platform is equipped with at least most of these auxiliary surgical devices to directly operate on the patient; the remote physician console is equipped with a smaller number of auxiliary surgical devices to assist the physician in performing the surgical procedure.

[0056] 2. Network communication bandwidth of remote surgical robots

[0057] The network channel between the remote doctor console and the patient's surgical platform is mainly through the nodes representing the remote doctor console and the patient's surgical platform in the cloud network, and communicates through a single virtual channel / internet channel. You can use the application to reserve and adjust the bandwidth size, please refer to Figure 4The specific process is as follows:

[0058] 1. Application Login

[0059] On smart devices such as mobile phones, computers, tablets, remote doctor consoles, and patient surgery platforms, users log in to the central server through an application. After successful login, the central server requests a resource list to query the remote doctor console and patient surgery platform nodes and determine the appointment status of each node. This list may include the remote doctor console and patient surgery platform nodes for which bandwidth can be controlled within the login account's permissions. Upon receiving the request, the central server requests authentication from the operator server and queries resource and appointment status. Upon successful authentication, the operator server returns the resource list and appointment status, which the central server forwards to the application.

[0060] 2. Query node bandwidth

[0061] The application displays the resource list and appointment status received on the smart terminal device's display. The application selects at least one node from the resource list and requests bandwidth from the central server. Upon receiving the request, the central server requests authentication from the carrier server and queries the bandwidth of the selected node. Upon successful authentication, the carrier server returns the node's bandwidth, which is forwarded to the application and displayed on the smart terminal device's display. Based on this node's bandwidth, the bandwidth between the remote physician's console and the patient's surgical platform is determined.

[0062] 3. Schedule bandwidth adjustment

[0063] The application displays the resource list and reservation status received on the smart terminal device's display. The user selects at least one node from the resource list, sets adjustment conditions such as the reservation time, reservation bandwidth value, and reservation query period, and initiates a "reservation bandwidth node" request to the central server. When the set reservation time arrives, the central server requests the operator's server to adjust the selected node to the reserved bandwidth value, ensuring that the network channel between the remote physician console and the patient's surgical platform maintains the reserved bandwidth value.

[0064] During the bandwidth adjustment process of the operator server, the central server periodically queries the bandwidth size of the reserved node according to the preset query cycle or fixed query cycle (for example, every 30 seconds). If the reserved bandwidth value is reached within the preset time, the current reservation of the selected node is cleared; if it is not reached within the preset time, a reservation failure prompt is sent to the application.

[0065] 3. Operation and Communication of Remote Surgery Robot

[0066] The doctor operates on the touch screen, foot pedals, handles, buttons and other control structures of the remote doctor console to trigger interactive operations on the patient's surgical platform; the doctor operates on the touch screen and buttons and other control structures of the patient's surgical platform to trigger interactive operations on the remote doctor console; different interactive operations will trigger different control states between the remote doctor console and the patient's surgical platform, resulting in different data transmission volumes in the network channel between the remote doctor console and the patient's surgical platform, so the network channel requires different bandwidths to ensure low network latency and stable data transmission.

[0067] Specific operations include power-on self-test, remote pairing, remote connection, remote interactive communication, remote surgery, remote pairing removal, and power-off self-test, which implement different surgical phases. Alternatively, they implement different surgical tasks, such as surgical operation tasks, teaching tasks, and debugging tasks. Under the control states corresponding to different operation triggers, the data transmission volume of the network channel varies, specifically due to the different types of data transmitted in the network channel.

[0068] For example, during remote pairing, remote connection, and remote pairing removal, the network channel transmits at least device information data, authentication data, and / or protocol data; during remote interactive communication and teaching tasks, the network channel transmits at least video data and / or audio data; during remote surgical operations and surgical tasks, the network channel transmits at least video data, audio data, and / or control signaling data. Specifically, a flag bit can be specified in the interactive data packet to identify its data type.

[0069] As a further example, the corresponding bandwidth size is adjusted according to the type of data transmitted in the network channel. For example, when only control signaling data is transmitted, 1Mbps of bandwidth is allocated; when teaching task-related data such as video data and / or audio data is transmitted, 10Mbps of bandwidth is allocated; when surgical operation task-related data such as video data, audio data and / or control signaling data is transmitted, 100Mbps of bandwidth is allocated.

[0070] For the operation and communication of remote surgical robots, bandwidth can be preset for different surgical stages, surgical tasks, and / or surgical types, and bandwidth adjustment can be automatically requested when triggered, eliminating the need to manually reserve and adjust bandwidth using the application. Figure 5 As shown, an implementation method for automatically adjusting bandwidth is provided. For example, the bandwidth for remote pairing between the remote doctor console and the patient surgery platform is preset to 10Mbps, and the process of going online, remote pairing, and adjusting bandwidth is explained. The specific instructions are as follows:

[0071] Going online: The remote doctor console and patient surgery platform connect to the central server and establish a communication connection with the central server. Based on the communication connection, they initiate registration with the central server. Registration information includes: IP information, port information, network type, etc. of each device board. After registration is complete, at least one end logs in to the central server, completing the online process and maintaining the online status.

[0072] Remote pairing: The remote doctor console and the patient surgery platform each initiate a pairing request to the central server. After the central server successfully pairs the two ends, it connects the communication connection between the remote doctor console and the patient surgery platform and returns the corresponding network information.

[0073] Bandwidth Adjustment: Before switching to other surgical phases, surgical tasks, and / or surgical types, the network channel's preset bandwidth for the remote pairing phase is set to 10 Mbps. The remote physician console requests the central server to adjust the bandwidth to 10 Mbps. Upon receiving the request, the central server sends a bandwidth query to the carrier server. After receiving confirmation, it authenticates the carrier server and requests an adjustment of the bandwidth value to 10 Mbps. After the adjustment is complete, the bandwidth value returned by the carrier is forwarded to the remote physician console. The request to adjust the bandwidth to 10 Mbps from the central server can also be made by the patient's surgical platform.

[0074] Specifically, when determining the surgical stage, surgical task, and / or surgical type to enter / switch, it can be determined by at least one of the following: 1) the control status between the remote doctor console and the patient surgical platform, 2) the connection status of the auxiliary surgical device detachably connected to the remote doctor console and the patient surgical platform, and 3) the type of data transmitted in the network channel.

[0075] Furthermore, when multiple pairs of master and slave devices (remote physician console and patient surgical platform) exist on the same network channel, each pair shares the same channel bandwidth and transmits data based on that bandwidth. This requires ensuring low latency and stable data transmission between each pair, while also ensuring that the total bandwidth required by each pair falls within the bandwidth threshold of the network channel. Therefore, bandwidth must be dynamically allocated to each pair based on the surgical stage M, surgical task N, and / or surgical type K being performed by each pair, to meet the bandwidth constraints of the network channel and the surgical requirements of each pair.

[0076] For example, Figure 6As shown in the figure, the remote doctor console A and the patient surgery platform A are performing the surgery stage m1, surgery task n1, and surgery type k1, and the corresponding allocated bandwidth is high (such as 100 Mbps); the remote doctor console C and the patient surgery platform C are performing the surgery stage m3, surgery task n3, and surgery type k3, and the corresponding allocated bandwidth is relatively high (such as 60 Mbps); the remote doctor console B and the patient surgery platform B are performing the surgery stage m2, surgery task n2, and surgery type k2, and the corresponding allocated bandwidth is low (such as 10 Mbps).

[0077] 4. Dynamic bandwidth adjustment based on the interaction process between master and slave devices

[0078] The bandwidth adjustment system of a remote surgical robot includes: a remote surgical robot, including a first terminal and a second terminal connected in communication, one of the first terminal and the second terminal being a remote doctor's console (master hand device) and the other end being a patient surgical platform (slave hand device); a central server, communicatively connected to at least one of the first terminal and the second terminal; a processor, configured to: trigger a first control state of the remote surgical robot in response to an interactive operation of the first terminal with respect to the second terminal; and adjust the bandwidth of the communication connection between the first terminal and the second terminal through the central server according to the first control state.

[0079] In this embodiment, an interactive operation between a first terminal and a second terminal refers to an operation performed on the first terminal, whereby the second terminal responds to the operation performed on the first terminal. This interactive operation is considered an interactive operation. For example, operating the operating components (claws / rings, handles, etc.) on the remote doctor's console controls the movement of the robotic arms and surgical instruments on the patient's surgical platform. Setting endoscope parameters on the touchscreen on the patient's surgical platform controls the display mode on the remote doctor's console.

[0080] In response to an interactive operation on the first terminal, control information is generated for the second terminal, and a first control state is represented based on the control information, or a first control state is mapped to a target from among a plurality of preset control states based on the control information. The control information may be information automatically generated when the operation is performed on the first terminal, and the first control state is a state variable based on the bandwidth adjustment configuration.

[0081] In addition to configuring the state variables of the first control state, bandwidth adjustment based on the configuration also configures the bandwidth size corresponding to each first control state, which is the dependent variable of the aforementioned state variables; matches the corresponding bandwidth size based on the first control state; and requests the central server to adjust the bandwidth based on the matched bandwidth size.

[0082] The central server is used to communicate with the network operator server. For each group of remote surgical robots logged into the central server, if bandwidth adjustment is needed, it will communicate with the network operator server in a unified manner based on the bandwidth adjustment request to adjust the bandwidth of the network channel to the requested bandwidth size.

[0083] Furthermore, when adjusting the bandwidth of the communication connection between the first and second terminals, either one of the terminals is connected to the central server, or both terminals are connected to the central server separately. Specifically, the former involves the remote physician console or patient surgical platform being connected to the central server, while the latter involves the remote physician console and patient surgical platform being connected to the central server separately. To reduce the load on the central server, it is preferred that the first and second terminals be connected to the central server separately.

[0084] Specifically, Figure 7 This example illustrates the bandwidth adjustment process for the first and second terminals when only the first terminal is in communication with the central server. The second terminal sends the first control status it triggers to the first terminal. The first terminal then checks the first control status triggered by the first and second terminals in real time or at a preset interval based on the surgical process. When the first control status remains unchanged, no bandwidth adjustment is required. When the first control status changes, a bandwidth adjustment request is sent to the central server based on the changed first control status.

[0085] After receiving the request, the central server triggers a bandwidth adjustment request to the network operator server: first, the network operator server is queried for the node bandwidth of the requested first terminal, and the node bandwidth is returned based on the confirmation query information; the central server sends its own identity information or the identity information of the first terminal for authentication by the operator server to ensure the security of bandwidth adjustment; after the authentication is passed, the node bandwidth of the first terminal can be adjusted to the requested bandwidth; after receiving the adjusted bandwidth size, the central server forwards it to the first terminal, which can be prompted on the first terminal.

[0086] Based on the bandwidth adjustment system of the telesurgery robot, a bandwidth adjustment method of the telesurgery robot is described. Please refer to Figure 8 A flow chart of a bandwidth adjustment method is provided, and an embodiment of the bandwidth adjustment method is provided, which is specifically as follows:

[0087] 801. In response to an interactive operation of the first terminal on the second terminal, triggering a first control state of the remote surgical robot;

[0088] In this embodiment, the remote doctor console and the patient surgical platform include operating components, and the operating components include at least one of the following: a graphical user interface and physical hardware; in response to a touch operation on the graphical user interface of the remote doctor console, or a physical operation on the physical hardware, an interactive operation on the patient surgical platform is triggered; in response to a touch operation on the graphical user interface of the patient surgical platform, or a physical operation on the physical hardware, an interactive operation on the remote doctor console is triggered.

[0089] Specifically, the graphical user interface of the first terminal is displayed on a display, such as a touch screen, and includes at least one control for controlling the second terminal. Touching the control triggers interactive operations on the second terminal. For example, the graphical user interface of the remote doctor's console includes at least one of the following controls for the patient's surgical platform: a connection control, a pairing control, a robot configuration control (such as configuration controls for the imaging system, energy platform, and robotic arm permissions), a surgical operation control, a patient surgical platform switching control, a pairing release control, a connection release control, etc. For example, the graphical user interface of the patient's operating table includes at least one of the following controls for the remote doctor's console: a connection control, a pairing control, a robot configuration control (such as configuration controls for the imaging system, energy platform, and robotic arm permissions), a remote doctor's console switching control, a pairing release control, a connection release control, etc. Any touch operation that can be used to represent different surgical scenarios can be configured to trigger the first control state, which will not be repeated here.

[0090] For example, if the first control state is set to include: preoperative control state, intraoperative control state, and postoperative control state; in response to touch operations on the connection control, pairing control, and robot configuration control, the preoperative control state is triggered; in response to touch operations on the surgical task selection control and the patient surgical platform switching control, the intraoperative control state is triggered; in response to touch operations on the pairing release control and the connection release control, the postoperative control state is triggered.

[0091] Specifically, the physical hardware of the first terminal, such as the remote surgical robot, includes a controller (such as a gripper / ring, handle, etc.), foot pedals, actuators, ergonomic components (to adjust the posture of the doctor's console to facilitate doctor operation), control buttons, etc.; such as the patient surgical platform includes: a tiller, a center column, a boom pivot, a boom, a telescopic joint, an arm height joint, a patient distance joint, a robotic arm, a base, control buttons, etc.

[0092] Exemplarily, if the first control state is set to include: a preoperative control state, an intraoperative control state, and a postoperative control state; the preoperative control state is triggered in response to physical operation of the actuator, ergonomic components, power button, pairing button, robot configuration button of the remote doctor console, or the deployment operation of the tiller, center column, boom pivot, boom, telescopic joint, arm height joint, patient distance joint and robotic arm of the patient surgical platform, as well as physical operation of the base and the power button; the intraoperative control state is triggered in response to physical operation of the controller and foot pedal of the remote doctor console; the postoperative control state is triggered in response to physical operation of the power off button of the remote doctor console, the retraction operation of the tiller, center column, boom pivot, boom, telescopic joint, arm height joint, patient distance joint and robotic arm of the patient surgical platform, as well as physical operation of the power off button.

[0093] In one embodiment, Figure 9 In the illustrated telesurgery robot 900, the physical hardware of the remote doctor's console may include a handle 910, and a graphical user interface may be provided on a display system 921 on an imaging cart 920; in response to physical manipulation of the handle 910 or touch manipulation of the display system 921, a corresponding first control state is triggered. The physical hardware of the patient surgical platform may include a sliding base 930 and two robotic arms 940 fixedly connected to the sliding base 930; the graphical user interface may be provided on a display system 951 on an imaging cart 950; in response to physical manipulation of the sliding base 930 or the robotic arms 940, or touch manipulation of the display system 951, a corresponding first control state is triggered. Any interactive operation that can be used to represent different surgical scenarios can be configured to trigger the first control state, which will not be further described here.

[0094] 802. According to the first control state, adjust, by the central server, a bandwidth of the communication connection between the first terminal and the second terminal.

[0095] In this embodiment, the central server is communicatively connected to a preset network operator server; through at least one of the first terminal and the second terminal, the bandwidth parameters corresponding to the first control state are matched, and based on the bandwidth parameters, a first bandwidth adjustment request is sent to the central server; through the central server, based on the first bandwidth adjustment request, a second bandwidth adjustment request is initiated to the network operator server to adjust the bandwidth of the communication connection between the first terminal and the second terminal through the network operator server.

[0096] In this embodiment, Figure 7The example shows that the bandwidth parameters corresponding to the first control state are matched by the first terminal, and the first bandwidth adjustment request is sent to the central server, that is, it is executed through the remote doctor console or the patient surgery platform; the first terminal and the second terminal can also respectively perform the bandwidth parameter matching of their own first control state and send the first bandwidth adjustment request.

[0097] Specifically, the bandwidth parameter may include a specific bandwidth value, such as 10 Mbps, 50 Mbps, 100 Mbps, etc. It may also be a bandwidth value range, such as [10 Mbps, 30 Mbps], [30 Mbps, 50 Mbps], [50 Mbps, 70 Mbps], [70 Mbps, 100 Mbps], etc. The first bandwidth adjustment request initiated includes the bandwidth parameter. Based on the bandwidth parameter included in the first bandwidth adjustment request, the central server generates a second bandwidth adjustment request, further requesting the network operator server to adjust the bandwidth.

[0098] Furthermore, when the central server finds that the bandwidth returned by the network operator server is different from the requested bandwidth value, or the returned bandwidth is not within the requested bandwidth value, the network operator server adjusts the bandwidth to the requested bandwidth value. When the bandwidth parameter is a bandwidth value range, a preset value within the bandwidth value range can be selected, preferably the maximum bandwidth value, such as 30Mbps in [10Mbps, 30Mbps].

[0099] In one embodiment, the interactive operation includes: an operation for triggering any one of multiple surgical stages, and an operation for triggering any one of multiple surgical tasks; the first control state represents: the state of the interactive operation permission of the first terminal to the second terminal in any surgical stage and / or any surgical task.

[0100] Among them, at least one of the first terminal and the second terminal stores a historical control state, indicating the current surgical stage and surgical task of the remote surgical robot; before matching the bandwidth parameters corresponding to the first control state, the at least one terminal also includes: comparing the first control state and the historical control state; if the comparison result is that the control states of the two are the same, that is, they are in the same surgical stage and surgical task, then there is no need to send a first bandwidth adjustment request to the central server, indicating that there is no need to adjust the bandwidth of the network channel; if the comparison result is that the control states of the two are different, that is, they indicate a switch in the surgical stage and surgical task, then a first bandwidth adjustment request is sent based on the bandwidth parameters of the first control state to adjust the bandwidth of the network channel to adapt to the switched surgical stage and surgical task.

[0101] Specifically, any one of the surgical stages includes at least one of the following: power-on self-test, remote pairing, remote connection, remote interactive communication, remote surgical operation, remote pairing release, and power-off self-test; triggering any one of the surgical tasks includes at least one of the following: surgical operation task, teaching task, and debugging task.

[0102] In one embodiment, the surgical stage may also include remote pairing switching, and the sorting of some surgical stages includes: remote pairing, remote connection, remote interactive communication, remote surgical operation / remote pairing release / remote pairing switching; the sorting between the surgical stage triggered by the interactive operation and the surgical stage in the historical control state is compared; if the former is sorted before the latter, the network channel is maintained at the bandwidth of the latter; if the former is sorted after the latter, the network channel is adjusted to the network of the former.

[0103] In one embodiment, the interactive operation includes: an operation for triggering any one of multiple surgical stages, and an operation for triggering any one of multiple surgical tasks; the first control state represents: the state of the interactive operation permission of the first terminal to the second terminal in any surgical stage and / or any surgical task.

[0104] Each surgical stage and the corresponding interactive operations and interactive operation permissions are as follows:

[0105] A. In response to physical manipulation of a power-on component (such as a button or joystick), or by presetting a power-on time in the graphical user interface, triggering a power-on self-test phase after the preset time has elapsed, the first terminal reads the device information of the second terminal and verifies the device configuration information of the first and second terminals to facilitate subsequent execution of other surgical phases on the second terminal. Its interactive operation permissions include at least the power-on self-test permission for the aforementioned operations.

[0106] B. In response to physical manipulation of a pairing component (such as a button / joystick) on the display screen, or touch manipulation of a pairing control in the graphical user interface, the remote pairing phase is triggered; the first terminal may request and display a pairing list of the second terminal, select the target second terminal to initiate a pairing request, and complete the pairing; its interactive operation permissions include at least pairing permissions for the aforementioned operations.

[0107] C. In response to physical operation of configuration components (such as buttons / joysticks) on the display screen, or touch operation of robot configuration controls in the graphical user interface, the remote connection stage is triggered; the first terminal and the second terminal can configure control parameters of the imaging system, energy platform, robotic arm permissions, etc.; their interactive operation permissions include at least the robot configuration permissions for the aforementioned operations.

[0108] D. In response to physical manipulation of a communication component (such as a button / joystick / scroll ball) on a display screen, or touch manipulation of a communication control on a graphical user interface, remote interactive communication is triggered; the first and second terminals can select or operate on image data, voice data, robot status, image annotation, etc. The interactive operation permissions include at least communication permissions for the aforementioned operations.

[0109] E. Trigger remote surgical operations in response to physical operations on operating components (grips / rings, handles, etc.) or foot pedals on the remote surgical robot, or touch operations on operating controls on the graphical user interface; the remote surgical robot performs master-slave control, robotic arm switching, and other operations on the patient's surgical platform; its interactive operation permissions include at least the aforementioned surgical operation permissions.

[0110] F. In response to a physical operation of a pairing release component (such as a button / joystick) on the display screen, or a touch operation of a pairing release control in the graphical user interface, remote pairing release is triggered. The first terminal and the second terminal verify whether the pairing release conditions are met (for example, whether the remote surgery operation is still being performed) and complete the pairing release process. Their interactive operation permissions include at least pairing release permissions for the aforementioned operations. The pairing release component and the pairing component can be the same component, and different triggering methods are set for pairing release and pairing.

[0111] G. In response to a physical operation of a shutdown component (such as a button / joystick) or a touch operation of a shutdown control in the graphical user interface, the power-on self-test phase is triggered. The first terminal and the second terminal verify whether the shutdown conditions are met and complete the shutdown process. The interactive operation permissions include at least shutdown permissions. The power-on component and the shutdown component can be the same component, and different triggering methods for power-on and power-off can be configured.

[0112] H. In response to a physical selection operation on a surgical operation task component (such as a button / joystick), or a touch operation on a surgical operation task control in a graphical user interface, a surgical operation task is triggered; the first terminal and the second terminal can perform the surgical operation (such as master-slave control); their interactive operation permissions include at least: routine surgical operation permissions.

[0113] I. In response to a physical selection operation on a teaching task component (such as a button / joystick), or a touch operation on a teaching task control in a graphical user interface, a teaching task is triggered; the first terminal and the second terminal can perform substitute teaching and anatomical scene display; their interactive operation permissions include at least: voice / video and other information transmission permissions.

[0114] J. In response to a physical selection of a debugging task component (such as a button / joystick) or a touch operation on a debugging task control in a graphical user interface, a debugging task is triggered; the first terminal and the second terminal can perform debugging of the remote surgical robot; their interactive operation permissions include at least remote surgical robot debugging permissions. The surgical operation task component, the teaching task component, and the debugging task component can be integrated into a task selection component and configured with different triggering methods.

[0115] In addition, a display screen can be provided to display each surgical stage area and a selection control. The physical hardware can be configured as a directional control component and a confirmation component. The directional control component can be: 1) two buttons or a joystick that control the movement of the selection control in the up / down or left / right directions along the display screen, or 2) four buttons or a joystick that control the movement of the selection control in the up / down / left / right directions along the display screen. When the selection control is positioned within a surgical stage area, the confirmation component is operated to trigger that surgical stage. The joystick integrates the directional control component and the confirmation component, and can be pressed perpendicularly to the up / down / left / right directions of the display screen to confirm the triggering of the surgical stage.

[0116] For any surgical stage and any surgical task, the preset bandwidth size can be based on the following principles: power-on self-test / power-off self-test, remote pairing / remote connection / remote pairing removal, remote interactive communication / remote surgical operation, as well as surgical operation tasks, debugging tasks, and teaching tasks. The preset bandwidth size can be a bandwidth level, a bandwidth value range, or a bandwidth value.

[0117] Exemplarily, at least one of the first terminal and the second terminal stores mapping information, and the bandwidth parameter includes multiple bandwidth levels in order of bandwidth, and the mapping information includes: a mapping relationship between multiple different first control states and multiple bandwidth levels; through at least one of the first terminal and the second terminal, the target bandwidth level corresponding to the first control state is matched in the stored mapping information, and a first bandwidth adjustment request is initiated to the central server based on the target bandwidth level.

[0118] In this embodiment, for the aforementioned surgical stages and surgical tasks, different bandwidth levels corresponding to the first control state are set in the preset bandwidth size. The larger the bandwidth preset, the higher the level, and the smaller the bandwidth preset, the lower the level. For example, 5 bandwidth levels are set from low to high: 10 Mbps, 20Mbps, 30Mbps, 40Mbps, and 60Mbps, then: {power-on self-test / power-off self-test: 10Mbps; remote pairing / remote connection / remote pairing release: 20Mbps; remote interactive communication / remote surgical operation: 60Mbps; surgical operation task: 60Mbps; debugging task: 40Mbps; teaching task: 30Mbps}.

[0119] Each bandwidth level corresponds to a bandwidth value range or a bandwidth value. The first bandwidth adjustment request includes the bandwidth value range or the bandwidth value. After receiving the first bandwidth request, the central server queries the network operator server for the bandwidth size of the current network channel: if it is greater than the bandwidth value range, the bandwidth size is adjusted to the upper limit of the bandwidth value range; if it is less than the bandwidth value range, the bandwidth size is adjusted to the lower limit of the bandwidth value range; the bandwidth size of the current network channel is adjusted to the requested bandwidth value.

[0120] In addition, when both the remote doctor console and the patient surgery platform store mapping information, they respectively match their own target bandwidth levels of the first control state, and initiate first bandwidth adjustment requests to the central server based on their respective target bandwidth levels.

[0121] When only the remote doctor console stores the mapping information, the patient surgery platform sends its first control state to the remote doctor console. The remote doctor console uniformly matches the target bandwidth level of its own and the patient surgery platform's first control state. At this time, the remote doctor console initiates a first bandwidth adjustment request to the central server based on the target bandwidth level.

[0122] When only the patient surgery platform stores the mapping information, the remote doctor console sends its first control state to the patient surgery platform. The patient surgery platform uniformly matches the target bandwidth level of its own and the remote doctor console's first control state. At this time, the patient surgery platform initiates a first bandwidth adjustment request to the central server based on the target bandwidth level.

[0123] In addition, the mapping information can also be stored in the central server, which will uniformly match the target bandwidth level corresponding to the first control state of the remote doctor console and the patient surgical platform based on the stored mapping information, and directly initiate a second bandwidth adjustment request to the network operator server based on the target bandwidth level.

[0124] In one embodiment, the bandwidth adjustment system also includes at least one auxiliary surgical device, and at least one end of the first terminal and the second terminal is detachably connected to the at least one auxiliary surgical device; in response to the interactive operation performed by the first terminal or the second terminal and the auxiliary surgical device, the second control state of the remote surgical robot is triggered, and the interactive operation at least includes the disassembly and assembly operation corresponding to the detachable connection; when the first control state and the second control state are triggered at the same time, one of the first control state and the second control state is selected from the first control state and the second control state according to the preset control state priority through at least one end of the first terminal and the second terminal; according to the selected control state, the bandwidth of the communication connection between the first terminal and the second terminal is adjusted through the central server.

[0125] For example, Figure 10 As shown, if the remote doctor console is connected to an auxiliary surgical device, it may include at least one of the following: an endoscope control headset, an EEG monitoring headset, an eye tracking device, a large display screen, etc.; if the patient operating platform is connected to an auxiliary surgical device, it may include at least one of the following: surgical instruments, catheter instruments, flushing systems, operating tables, sterile covers, puncture cards, energy platforms, monitors, liquid suction systems, pneumoperitoneum machines, etc.

[0126] Different interactive operations between the first terminal or the second terminal and different auxiliary surgical devices represent different surgical stages or surgical tasks, and the corresponding second control state is triggered; for example, the second control state is set to include: preoperative control state, intraoperative control state, and postoperative control state, and the interactive operations include unconnected operation, connected operation, and dismantled operation; then the unconnected operation corresponds to the preoperative control state, the connected operation corresponds to the intraoperative control state, and the dismantled operation corresponds to the postoperative control state.

[0127] Furthermore, when the remote skilled robot performs different surgical projects, the auxiliary surgical devices required for each surgical project are preset: when all the preset auxiliary surgical devices have not completed all connection operations, the pre-operative control state is triggered; when all the connection operations are completed, the intra-operative control state is triggered; when any auxiliary surgical device is removed, the post-operative control state is triggered; among them, the key auxiliary surgical device for each surgical project can be further set, and the post-operative control state will be triggered only when the key auxiliary surgical device is removed.

[0128] For example, if the second control state triggered is the pre-operative control state, the bandwidth of the network channel is adjusted to 10 Mbps; if the second control state triggered is the intra-operative control state, the bandwidth of the network channel is adjusted to 60 Mbps; if the second control state triggered is the post-operative control state, the bandwidth of the network channel is adjusted to 10 Mbps;

[0129] Exemplarily, the key auxiliary surgical devices of the remote doctor's console include: an endoscope control head-mounted device; the key auxiliary surgical devices of the patient's surgical platform include: surgical instruments, catheter instruments, puncture cards, sterile covers, monitors, and operating tables.

[0130] In one embodiment, the preset control state priority may indicate that, between the first and second control states, the control state with the lower matching bandwidth value has a higher priority. For example, if the first control state is the preoperative state and the matching bandwidth is 20 Mbps, and the second control state is the intraoperative state and the matching bandwidth is 60 Mbps, then the remote surgical robot has not yet begun performing the surgical operation, and the first control state has a higher priority. The control state with the higher priority is selected from the first and second control states.

[0131] In one embodiment, the preset control state priority can also represent: the importance of the interactive operations performed on the remote physician console and the patient surgical platform relative to the interactive operations performed on the auxiliary surgical device, which is used to determine whether the bandwidth needs to be adjusted or the amount of bandwidth to be adjusted to meet the required bandwidth of the surgical project.

[0132] For example, if the first control state includes: a first pre-operative control state (such as power-on self-test, remote pairing, and remote connection), a first intra-operative control state (such as remote interactive communication and remote surgical operation), and a first post-operative control state (such as remote pairing release and power-off self-test); and the second control state includes: a second pre-operative control state, a second intra-operative control state, and a second post-operative control state; with the first control state as a reference, the preset control state priorities include:

[0133] 1. The first preoperative control state > the second intraoperative control state / second postoperative control state; this means that when the remote doctor console and the patient surgical platform are performing interactive operations such as power-on self-test, remote pairing, and remote connection, the network channel is adjusted to, for example, 10Mbps or 20Mbps. When all auxiliary surgical devices are connected or removed, the surgical robot self-test and remote pairing procedures must be completed before subsequent surgical operations can be performed. Therefore, the network channel bandwidth does not need to be adjusted to 60Mbps or 10Mbps at this time;

[0134] 2. The first intraoperative control state is less than the second preoperative control state / second postoperative control state. This means that when the remote doctor console and the patient surgical platform are performing remote interactive communication and remote surgical operations, the network channel is adjusted to 60Mbps, for example. If all auxiliary surgical devices are not connected or removed, the surgical conditions are not met and the network channel bandwidth only needs to be 10Mbps.

[0135] 3. The first postoperative control state is greater than the second preoperative control state / second intraoperative control state. This means that when the remote doctor's console and the patient's surgical platform are performing interactive operations such as remote pairing removal and shutdown self-test, the network channel is adjusted to, for example, 10Mbps or 20Mbps, indicating that the remote surgical robot has stopped the surgical operation. Therefore, even if all auxiliary surgical devices are connected, there is no need to perform the surgical operation, and the network channel does not need to be adjusted to 60Mbps.

[0136] In addition, the remote doctor console and the patient surgical platform store a historical first control state or a second control state, indicating the surgical stage and surgical task of the current remote surgical robot; when the historical first control state is stored and the current second control state is triggered, or when the historical second control state is stored and the current first control state is triggered, it is determined that the first control state and the second control state are triggered at the same time.

[0137] In addition, it also includes an external image acquisition device, which is communicatively connected to the first terminal or the second terminal detachably connected to the auxiliary surgical device; before triggering the second control state of the remote surgical robot in response to the interactive operation performed by the first terminal or the second terminal with the auxiliary surgical device, it also includes: acquiring a surgical scene image through the external image acquisition device, wherein the surgical scene image includes the auxiliary surgical device and the image of the first terminal or the second terminal detachably connected to the auxiliary surgical device; according to the surgical scene image, the disassembly and assembly operation of the first terminal or the second terminal and the auxiliary surgical device is identified through the first terminal or the second terminal detachably connected to the auxiliary surgical device.

[0138] In this embodiment, if the remote physician console is connected to an auxiliary surgical device, an external image acquisition device equipped with an image recognition algorithm is provided at one end of the remote physician console; if the patient surgical platform is connected to the auxiliary surgical device, an external image acquisition device equipped with an image recognition algorithm is provided at one end of the patient surgical platform. The external image acquisition device is spatially positioned so that the captured image of the surgical scene includes the junction between the auxiliary surgical device and the remote physician console, or the junction between the auxiliary surgical device and the patient surgical platform.

[0139] Based on surgical scene images, the deployed image recognition algorithm is applied to identify disassembly and installation operations between the remote surgeon's console and auxiliary surgical devices, or between the patient's surgical platform and auxiliary surgical devices. Image recognition algorithms include machine learning algorithms and deep learning algorithms, such as convolutional neural networks, recurrent neural networks, and support vector machines.

[0140] In addition to being able to identify the assembly and disassembly operations between the remote doctor's console and the auxiliary surgical device, and between the patient's surgical platform and the auxiliary surgical device through an in vitro image acquisition device and the application of an image recognition algorithm, additional assembly and disassembly trigger components, such as ranging sensors and pressure sensors, can be set at the joint between the patient's surgical platform and the auxiliary surgical device to identify the assembly and disassembly operations. The assembly and disassembly operations can also be confirmed based on the detection results of the connection status of the auxiliary surgical device by the remote surgical robot.

[0141] In addition, the aforementioned disassembly and assembly operations refer to physical disassembly and assembly operations; the interactive operations between the remote doctor's console and the auxiliary surgical device, and between the patient's surgical platform and the auxiliary surgical device, can also include start-stop operations, which refer to start and stop operations under the control state. The start-stop operation can be achieved through the start and stop button / joystick operation, or the start and stop display touch operation. For example, the start and stop operation under the control state can be: the operation of starting and stopping energy output such as current, voltage, light, and electromagnetic.

[0142] In one embodiment, the first control state includes: a third control state of the remote doctor console over the patient surgical platform, and a fourth control state of the patient surgical platform over the remote doctor console, and a master-slave state priority is preset between the third control state and the fourth control state; the central server is communicatively connected to one end of the remote doctor console and the patient surgical platform; the third control state is sent to the patient surgical platform through the remote doctor console, and the patient surgical platform selects one of the control states from its current fourth control state and the received third control state according to the master-slave state priority; or, the fourth control state is sent to the remote doctor console through the patient surgical platform, and the remote doctor console selects one of the control states from its current third control state and the received fourth control state according to the master-slave state priority; according to the control state selected by the remote doctor console or the patient surgical platform, the bandwidth of the communication connection between the remote doctor console and the patient surgical platform is adjusted through the central server.

[0143] In this embodiment, the central server is only connected to the remote doctor console or the patient surgery platform for communication, and the third control state and the fourth control state are aggregated to one end of the two. Based on the selected control state, one of the aggregated ends uniformly sends a first bandwidth adjustment request to the central server to adjust the bandwidth of the network channel.

[0144] Specifically, the remote doctor console also stores a third control state triggered historically. When the patient surgical platform sends a fourth control state to the remote doctor console, one of the control states is selected to determine whether the bandwidth needs to be adjusted; or the patient surgical platform also stores a fourth control state triggered historically, indicating the current surgical stage and surgical task of the remote surgical robot; when the remote doctor console sends the third control state to the patient surgical platform, one of the control states is selected to determine whether the bandwidth needs to be adjusted.

[0145] In one embodiment, the master-slave state priority can also be expressed as the control state priority: in the third and fourth control states, the connection state with the lower matched bandwidth value has a higher priority. For example, the third control state represents the postoperative control state of the remote surgical robot, and the matched bandwidth is 20Mbps. The fourth control state represents the intraoperative control state of the remote surgical robot, and the matched bandwidth is 60Mbps; this indicates that the remote surgical robot should end the surgical operation at this time, and the third control state has a higher priority. The above examples of relative control state priorities also apply to the examples of master-slave state priorities here, so they will not be repeated here.

[0146] Wherein, one of the control states is selected from the third control state and the fourth control state according to the master-slave state priority from high to low.

[0147] In one embodiment, the master-slave state priority can also be expressed as the control state priority: between the third and fourth control states, the control state with the lower matching bandwidth value has a higher priority. For example, the fourth control state is the postoperative control state, and the matching bandwidth is 20Mbps, while the third control state is the intraoperative control state, and the matching bandwidth is 60Mbps; this indicates that the remote surgical robot should end the surgical operation at this time, and the fourth control state has a higher priority. The above examples of relative control state priorities also apply to the examples of master-slave state priorities here, so they will not be repeated here.

[0148] In addition, the central server is respectively connected to the remote doctor console and the patient surgery platform; the third control state is sent to the central server through the remote doctor console, and the fourth control state is sent to the central server through the patient surgery platform; through the central server, one of the control states is selected from the third control state and the fourth control state according to the master-slave state priority, and the bandwidth of the communication connection between the remote doctor console and the patient surgery platform is adjusted according to the selected control state.

[0149] In this embodiment, the central server can also communicate with the remote doctor console and the patient surgery platform at the same time. The third control state and the fourth control state are aggregated in the central server, and the bandwidth of the network channel is adjusted based on the selected control state.

[0150] Please refer to the following Figure 11 A flow chart of a bandwidth adjustment method is provided, which provides a preferred embodiment of the bandwidth adjustment method, as shown below:

[0151] During the interactive operation with the patient surgical platform, the remote doctor console can first determine surgical task 1 based on the interactive operation, and then further determine surgical stage 1 based on the interactive operation, thereby placing the remote surgical robot in the third control state. Similarly, during the interactive operation with the remote doctor console, the patient surgical platform can first determine surgical task 2 and then surgical stage 2, and further place the remote surgical robot in the fourth control state. One of the control states is selected based on the master-slave state priority.

[0152] The remote doctor's console is detachably connected to an auxiliary surgical device A, and during the interactive operation between the two, the second control state of the remote surgical robot can be determined; the patient's surgical platform is detachably connected to an auxiliary surgical state B, and during the interactive operation between the two, the second control state of the remote surgical robot can also be determined.

[0153] Based on the control state previously selected based on the master-slave state priority and the second control state, a control state is further selected according to the control state priority to initiate a first bandwidth adjustment request to the central server, so that the central server can initiate a second bandwidth adjustment request to the network operator server, so that the network operator server can adjust the bandwidth between the remote doctor console and the patient surgery platform based on the second bandwidth adjustment request.

[0154] 5. Dynamic Bandwidth Adjustment Based on Assisted Surgical Devices

[0155] The bandwidth adjustment system includes: at least one auxiliary surgical device; a remote surgical robot, including a first terminal and a second terminal in communication connection, wherein one end of the first terminal and the second terminal is a remote doctor console and the other end is a patient surgical platform, and at least one end of the first terminal and the second terminal is detachably connected to at least one of the auxiliary surgical devices; a central server, which is in communication connection with at least one end of the first terminal and the second terminal; a processor, which is configured to: trigger a first connection state between the two in response to an interactive operation performed by the first terminal or the second terminal and the auxiliary surgical device; and adjust the bandwidth of the communication connection between the first terminal and the second terminal through the central server according to the first connection state.

[0156] Specifically, Figure 12 This example illustrates the bandwidth adjustment process for the first and second terminals when only the first terminal is communicating with the central server. The second terminal sends the first control status triggered by itself to the first terminal. The first terminal then checks the first control status triggered by the first and second terminals in real time or at a preset interval based on the surgical process. When the first connection status remains unchanged, no bandwidth adjustment is required. When the first connection status changes, a bandwidth adjustment request is sent to the central server based on the changed first connection status.

[0157] After receiving the request, the central server triggers a bandwidth adjustment request to the network operator server: first, the network operator server is queried for the node bandwidth of the requested first terminal, and the node bandwidth is returned based on the confirmation query information; the central server sends its own identity information or the identity information of the first terminal for authentication by the operator server to ensure the security of bandwidth adjustment; after the authentication is passed, the node bandwidth of the first terminal can be adjusted to the requested bandwidth; after receiving the adjusted bandwidth size, the central server forwards it to the first terminal, which can be prompted on the first terminal.

[0158] In this embodiment, the interactive operation includes a start / stop operation and a disassembly operation; in response to the start / stop operation or the disassembly operation performed by the first terminal or the second terminal and the auxiliary surgical device, the first connection state between the two is triggered.

[0159] Specifically, the surgical assistance device triggers the start / stop operation in response to a touch operation on the graphical user interface or a physical operation on the physical hardware, wherein the operating components may be buttons / joysticks, and the graphical user interface may be displayed on a touch screen.

[0160] Specifically, it also includes an in vitro image acquisition device, which is communicatively connected to a first terminal or a second terminal detachably connected to the auxiliary surgical device; the surgical scene image is acquired by the in vitro image acquisition device, wherein the surgical scene image includes the auxiliary surgical device and an image of the first terminal or the second terminal detachably connected to the auxiliary surgical device; according to the surgical scene image, the disassembly and assembly operation of the first terminal or the second terminal and the auxiliary surgical device is identified through the first terminal or the second terminal detachably connected to the auxiliary surgical device.

[0161] Please refer to Figure 13 A flow chart of a bandwidth adjustment method is provided, and an embodiment of the bandwidth adjustment method is provided, which is specifically as follows:

[0162] 1301. In response to an interactive operation performed between the first terminal or the second terminal and the surgical assistance device, triggering a first connection state between the two.

[0163] In this embodiment, at least one of the first terminal and the second terminal stores a historical first connection state, and each first connection state is preset with a connection state priority; through at least one of the first terminal and the second terminal, one of the first connection states is selected from the currently triggered first connection state and the historical first connection state according to the connection state priority; according to the selected first connection state, the bandwidth of the communication connection between the first terminal and the second terminal is adjusted through the central server.

[0164] Specifically, the central server is only connected to the remote doctor console or the patient surgery platform for communication. The first connection states generated by the remote doctor console or the patient surgery platform are aggregated to one end of the two. One of the aggregated ends selects one of the first connection states based on the connection state priority, and uniformly sends a first bandwidth adjustment request to the central server to adjust the bandwidth of the network channel.

[0165] Specifically, the remote doctor console also stores a historically triggered first connection state, indicating the current surgical stage and surgical task of the remote surgical robot. When the patient surgical platform sends the first connection state generated by itself to the remote doctor console, one of the first connection states is selected to determine whether the bandwidth needs to be adjusted; or the patient surgical platform also stores a historically triggered first connection state, indicating the current surgical stage and surgical task of the remote surgical robot; when the remote doctor console sends the first connection state to the remote doctor console, one of the first connection states is selected to determine whether the bandwidth needs to be adjusted.

[0166] In one embodiment, the connection state priority can also be expressed as the control state priority: in the first connection state, the data type with a lower matched bandwidth value has a higher priority. For example, the first connection state triggered historically represents the postoperative stage, and the matched bandwidth is 20Mbps. The first connection state triggered currently represents the intraoperative stage, and the matched bandwidth is 60Mbps. This indicates that the remote surgical robot should end the surgical operation at this time. The priority of the first connection state triggered historically is higher, and the bandwidth is adjusted to 20Mbps. The aforementioned example of relative control state priority also applies to the example of connection state priority here, so it will not be repeated here.

[0167] Wherein, according to the master-slave data type priority from high to low, one of the first connection states is selected from the historically triggered first connection state and the currently triggered first connection state.

[0168] In this embodiment, the central server can also communicate with the remote doctor console and the patient surgery platform at the same time. The first connection states generated by the two are aggregated in the central server. The central server selects the first connection state according to the connection state priority and adjusts the bandwidth of the network channel.

[0169] Furthermore, the central server may also store historical first connection states, and after receiving the first connection states sent by either the remote doctor console or the patient surgery platform, select one of the first connection states based on the connection state priority and adjust the bandwidth of the network channel.

[0170] 1302. According to the first connection state, adjust, by the central server, the bandwidth of the communication connection between the first terminal and the second terminal.

[0171] In one embodiment, the first control state of the remote surgical robot is triggered in response to the interactive operation of the first terminal to the second terminal, and the second control state of the remote surgical robot is triggered in response to the first connection state; when the first control state and the second control state are triggered at the same time, one of the first control state and the second control state is selected from the first control state and the second control state according to a preset control state priority through at least one of the first terminal and the second terminal; according to the selected control state, the bandwidth of the communication connection between the first terminal and the second terminal is adjusted through the central server.

[0172] In one embodiment, in response to the first connection state, the second control state of the remote surgical robot is triggered; according to the second control state, the bandwidth of the communication connection between the first terminal and the second terminal is adjusted by the central server.

[0173] There are three ways to adjust the bandwidth based on the auxiliary surgical device: 1) directly triggering the second control state based on the interactive operation performed between the first terminal or the second terminal and the auxiliary surgical device, and using it to adjust the bandwidth; 2) directly triggering the first connection state based on the interactive operation, and using it to adjust the bandwidth; 3) further determining the second control state based on the first connection state, and then adjusting the bandwidth according to the second control state.

[0174] In this embodiment, the auxiliary surgical device includes: a first auxiliary surgical device detachably connected to the remote doctor console, and a second auxiliary surgical device detachably connected to the patient surgical platform; wherein, the first connection state includes: a second connection state triggered based on the interactive operation between the remote doctor console and the first auxiliary surgical device, and a third connection state triggered based on the interactive operation between the patient surgical platform and the second auxiliary surgical device, and a master-slave state priority is preset between the second connection state and the third connection state; through the remote doctor console or the patient surgical platform, one of the connection states is selected from the second connection state and the third connection state according to the master-slave state priority; according to the selected connection state, the bandwidth of the communication connection between the remote doctor console and the patient surgical platform is adjusted by the central server.

[0175] In this embodiment, the central server is only connected to the remote doctor console or the patient surgery platform for communication, and the second connection state and the third connection state are aggregated to one end of the two. One of the aggregated ends sends a first bandwidth adjustment request to the central server based on the selected connection state to adjust the bandwidth of the network channel.

[0176] Specifically, the remote doctor console also stores a second connection state triggered historically. When the patient surgery platform sends a third connection state to the remote doctor console, one of the connection states is selected to determine whether the bandwidth needs to be adjusted; or the patient surgery platform also stores a third connection state triggered historically, indicating the current surgical stage and surgical task of the remote surgical robot. When the remote doctor console sends the second connection state to the remote doctor console, one of the connection states is selected to determine whether the bandwidth needs to be adjusted.

[0177] In one embodiment, the master-slave state priority can also be expressed as the control state priority: in the second connection state and the third connection state, the connection state with the lower matched bandwidth value has a higher priority. For example, the third connection state represents the postoperative control state of the remote surgical robot, and the matched bandwidth is 20Mbps. The second connection state represents the intraoperative control state of the remote surgical robot, and the matched bandwidth is 60Mbps; indicating that the remote surgical robot should end the surgical operation at this time, the third connection state has a higher priority. The above examples of relative control state priority also apply to the examples of master-slave state priority here, so they will not be repeated here.

[0178] In addition, the central server can also communicate with the remote doctor console and the patient surgery platform at the same time. The second connection state and the third connection state are aggregated in the central server, and the bandwidth of the network channel is adjusted based on the connection state selected based on the master-slave state priority.

[0179] In this embodiment, the central server is communicatively connected to a preset network operator server; according to the first connection state, the bandwidth of the communication connection between the first terminal and the second terminal is adjusted by the central server, including: matching the bandwidth parameters corresponding to the first connection state through at least one of the first terminal and the second terminal, and sending a first bandwidth adjustment request to the central server based on the bandwidth parameters; and initiating a second bandwidth adjustment request to the network operator server through the central server based on the first bandwidth adjustment request, so as to adjust the bandwidth of the communication connection between the first terminal and the second terminal through the network operator server.

[0180] In this embodiment, the bandwidth parameter can be matched with the first control state as well as the first connection state, thereby reducing the steps of associating the first control state with the first connection state. The first connection state is matched to obtain the bandwidth parameter of the corresponding second control state.

[0181] Furthermore, at least one of the first terminal and the second terminal stores mapping information, and the bandwidth parameter includes multiple bandwidth levels in order of bandwidth, and the mapping information includes: a mapping relationship between multiple first connection states and multiple bandwidth levels; based on the bandwidth parameter, sending a first bandwidth adjustment request to the central server includes: through at least one of the first terminal and the second terminal, matching the target bandwidth level corresponding to the first connection state in the stored mapping information, and initiating a first bandwidth adjustment request to the central server based on the target bandwidth level.

[0182] In this embodiment, bandwidth levels are set for different first connection states to determine preset bandwidth sizes for different surgical stages and surgical tasks. The larger the bandwidth preset, the higher the level, and the smaller the bandwidth preset, the lower the level. For example, four bandwidth levels are set from low to high: 20Mbps, 30Mbps, 40Mbps, and 60Mbps, then: {pre-operative: 20Mbps; intra-operative: 60Mbps; post-operative: 20Mbps; surgical operation task: 60 Mbps; debugging task: 40Mbps; teaching task: 30Mbps}.

[0183] Each bandwidth level corresponds to a bandwidth value range or a bandwidth value. The first bandwidth adjustment request includes the bandwidth value range or the bandwidth value. After receiving the first bandwidth request, the central server queries the network operator server for the bandwidth size of the current network channel: if it is greater than the bandwidth value range, the bandwidth size is adjusted to the upper limit of the bandwidth value range; if it is less than the bandwidth value range, the bandwidth size is adjusted to the lower limit of the bandwidth value range; the bandwidth size of the current network channel is adjusted to the requested bandwidth value.

[0184] In addition, when both the remote doctor console and the patient surgery platform store mapping information, they respectively match their own target bandwidth levels of the first connection state, and initiate first bandwidth adjustment requests to the central server based on their respective target bandwidth levels.

[0185] When only the remote doctor console stores the mapping information, the patient surgery platform sends its first connection status to the remote doctor console. The remote doctor console uniformly matches the target bandwidth level of its own and the patient surgery platform's first connection status. At this time, the remote doctor console initiates a first bandwidth adjustment request to the central server based on the target bandwidth level.

[0186] When only the patient surgery platform stores the mapping information, the remote doctor console sends its first connection status to the patient surgery platform. The patient surgery platform uniformly matches the target bandwidth level of its own and the remote doctor console's first connection status. At this time, the patient surgery platform initiates a first bandwidth adjustment request to the central server based on the target bandwidth level.

[0187] In addition, the mapping information can also be stored in the central server, which will uniformly match the target bandwidth level corresponding to the first connection state of the remote doctor console and the patient surgery platform based on the stored mapping information, and directly initiate a second bandwidth adjustment request to the network operator server based on the target bandwidth level.

[0188] In this embodiment, the auxiliary surgical device includes a first auxiliary surgical device and a second auxiliary surgical device of different functional types; according to the first connection state triggered by the first auxiliary surgical device, the bandwidth of the communication connection between the first terminal and the second terminal is adjusted according to a preset first amplitude through the central server; according to the first connection state triggered by the second auxiliary surgical device, the bandwidth of the communication connection between the first terminal and the second terminal is adjusted according to a preset second amplitude through the central server.

[0189] In this embodiment, the first auxiliary surgical device has a function type that requires uninterrupted operation throughout the entire surgical procedure; the second auxiliary surgical device has a function type that does not require uninterrupted operation throughout the entire surgical procedure. Based on the aforementioned distinction between the two function types, specific examples are as follows:

[0190] A. Auxiliary surgical devices that can be detachably connected to the remote doctor's console include: endoscope control headsets, EEG monitoring headsets, eye tracking devices, large display screens, etc. If the EEG monitoring headset and eye tracking device are used to control the movement of the suction head of the liquid suction system, and the liquid suction system needs to be used throughout the current surgical project, then the endoscope control headset, EEG monitoring headset, and eye tracking device are considered the first auxiliary surgical device, and the large display screen is considered the second auxiliary surgical device. If the liquid suction system is not required throughout the current surgical project, then the endoscope control headset is considered the first auxiliary surgical device, and the EEG monitoring headset, eye tracking device, and large display screen are considered the second auxiliary surgical device.

[0191] B. Auxiliary surgical devices that are detachably connected to the patient's surgical platform include: surgical instruments, catheters, irrigation systems, operating tables, sterile drapes, puncture cards, energy platforms, monitors, fluid aspiration systems, and insufflators. The first auxiliary surgical device includes: operating tables, sterile drapes, puncture cards, and monitors; the second auxiliary surgical device includes: surgical instruments, catheters, irrigation systems, energy platforms, fluid aspiration systems, and insufflators.

[0192] Specifically, the duration of the first connection state triggered by the first auxiliary surgical device is monitored through at least one of the first terminal and the second terminal; when the duration exceeds the preset first duration, the bandwidth of the communication connection between the first terminal and the second terminal is adjusted according to the preset first amplitude through the central server.

[0193] Regarding the connection relationship between the first auxiliary surgical device, such as the operating table, sterile cover, puncture card, monitor and the patient surgical platform, when not connected, the first connection state is determined to be the unconnected state (belonging to the preoperative control state), and the bandwidth is adjusted to, for example, 10Mbps; when connected, the first connection state is determined to be the connected state (belonging to the intraoperative control state), and the bandwidth is adjusted to, for example, 60Mbps; when disassembled, the first connection state is determined to be the disassembled state (belonging to the postoperative control state), and the bandwidth is adjusted to, for example, 10Mbps.

[0194] Specifically, different first durations are set based on the different first connection states of the first auxiliary surgical device: 0, 17ms, 60ms, etc. in the disconnected and connected states, and 1 minute, 2 minutes, 5 minutes, 10 minutes, etc. in the detached state. The former provides real-time switching bandwidth to meet surgical needs, while the latter provides a clear time interval to prevent surgical interruptions and the need to continue.

[0195] For secondary surgical assistive devices, such as EEG monitoring headsets, eye tracking devices, large displays, surgical instruments, catheters, irrigation systems, energy platforms, fluid suction systems, and pneumoperitoneum machines, different second durations are set based on the device's different first connection states: In the disconnected and connected states, the second durations are set to shorter durations such as 0, 17ms, and 60ms. In the detached state, the second duration can be set to the required surgical duration for each procedure (i.e., static duration) or the remaining duration of the current procedure (i.e., dynamic duration). For example, if the required surgical duration for a procedure is 180 minutes, the second duration can be set directly to 180 minutes; if the current procedure has progressed for 50 minutes, the second duration can be set to 130 minutes. The former provides real-time bandwidth switching to meet surgical needs, while the latter requires a significant time interval because, even though the second surgical assistive device has been detached, it does not indicate the end of the procedure, so the bandwidth must be adjusted after the required duration has elapsed. Furthermore, the preset first duration is set to be less than the remaining surgical duration.

[0196] Specifically, the duration of the first connection state triggered by the second auxiliary surgical device is monitored through at least one of the first terminal and the second terminal; when the duration exceeds the preset second duration, the bandwidth of the communication connection between the first terminal and the second terminal is adjusted according to the preset second amplitude through the central server; wherein the first amplitude is greater than the second amplitude, and the preset first duration is less than the preset second duration.

[0197] For the first auxiliary surgical device, the connection status switching at different surgical stages involves a large increase or decrease in the transmission of data, so a larger first amplitude adjustment bandwidth is required. For example, the connection status switching between preoperative (10Mbps), intraoperative (60Mbps), and postoperative (10Mbps); or the connection status switching between power-on self-test (10Mbps), remote pairing / remote connection (20Mbps), remote interactive communication / remote surgical operation (60Mbps), remote pairing release / remote pairing switch (20Mbps), and power-off self-test (10Mbps).

[0198] For secondary surgical assistive devices, switching between connection states during the same surgical phase involves increasing or decreasing the transmission of smaller amounts of data, thus requiring a smaller second bandwidth adjustment. For example, during surgery, only data related to EEG monitoring headsets, eye tracking equipment, large displays, surgical instruments, catheters, flushing systems, energy platforms, fluid suction systems, and insufflator can be reduced. This data usage is relatively low, while video and voice data, which consume significantly more bandwidth (e.g., 90% of the bandwidth), continue to be transmitted.

[0199] 6. Dynamic bandwidth adjustment based on data type transmission

[0200] The bandwidth adjustment system includes: a first remote host and a second remote host, with a network channel established between the first remote host and the second remote host; a remote surgical robot, including a first terminal connected to the first remote host, and a second terminal connected to the second remote host, one end of the first terminal and the second terminal being a remote doctor console and the other end being a patient surgical platform; a central server, connected to at least one of the first remote host and the second remote host; a processor, configured to: generate first interaction data in response to a first interaction operation performed by the first terminal on the second terminal, and transmit the first interaction data to the second remote host through the network channel through the first remote host, so that the second terminal can perform motion control based on the first interaction data; identify the first data type of the first interaction data transmitted in the network channel through at least one of the first remote host and the second remote host; and adjust the bandwidth of the network channel through the central server according to the first data type.

[0201] Specifically, Figure 14 Example: When only the first terminal is connected to the central server for communication, the bandwidth adjustment process for the first terminal and the second terminal is as follows: The first terminal checks the first data type of the first interactive data transmitted in the network channel in real time according to the surgical process / according to the preset period, such as Figure 14 The first interactive data shown is generated by the first remote host and sent to the second remote host via the first remote host. The first data type is forwarded to the first remote host, so the first remote host can identify the first data type. When the first data type remains unchanged, no bandwidth adjustment is required. When the first data type changes, a bandwidth adjustment request is sent to the central server based on the changed first data type.

[0202] After receiving the request, the central server triggers a bandwidth adjustment request to the network operator server: first, the network operator server is queried for the node bandwidth of the requested first terminal, and the node bandwidth is returned based on the confirmation query information; the central server sends its own identity information or the identity information of the first terminal for authentication by the operator server to ensure the security of bandwidth adjustment; after the authentication is passed, the node bandwidth of the first terminal can be adjusted to the requested bandwidth; after receiving the adjusted bandwidth size, the central server forwards it to the first terminal, which can be prompted on the first terminal.

[0203] Please refer to Figure 15 A flow chart of a bandwidth adjustment method is provided, and an embodiment of the bandwidth adjustment method is provided, which is specifically as follows:

[0204] 1501. In response to a first interactive operation performed by the first terminal on the second terminal, first interactive data is generated, and the data is transmitted to the second remote host via the first remote host through the network channel, so that the second terminal can perform motion control based on the first interactive data.

[0205] In this embodiment, the first interactive data includes at least one of the following: endoscope video data, operating room pan / tilt video data, operating room gun video data, audio data, master-slave control signaling data, pairing data, energy control data, etc.

[0206] Exemplarily, the remote doctor console and the patient surgical platform both perform the following first interactive operation: turning on the endoscope video function, the operating room pan-tilt video function, the operating room gun camera video function, the voice call function, and the pairing function, and based on the turned-on functions, transmitting the corresponding video data, audio data, and pairing data; when the remote doctor console performs the first interactive operation on the foot pedal, operating parts such as the clamp / ring, and handle, it transmits control signaling and energy control data.

[0207] When at least one of the remote doctor console and the patient surgical platform performs a first interactive operation, the target end performs corresponding motion control. For example, if the endoscope video function, operating room pan / tilt video function, and operating room gun camera video function are simultaneously enabled on both ends, the remote doctor console displays the endoscope video, operating room pan / tilt video, and operating room gun camera video; if the voice call function is simultaneously enabled on both ends, voice messages are played on both ends; if the pairing function is enabled on one end, the other end displays a pairing confirmation interface, a pairing waiting interface, a pairing success interface, etc.; if the remote doctor console selects a control mode using a foot pedal and performs an operation using an operating component, the patient surgical platform's robotic arm and connected surgical instruments and energy devices will move accordingly.

[0208] 1502. Identify, through at least one of the first remote host and the second remote host, a first data type of first interaction data transmitted in the network channel;

[0209] In this embodiment, the first interactive data is transmitted between the remote doctor's console and the patient's surgical platform through a network channel established between the first remote host and the second remote host; therefore, the first data type of the first interactive data transmitted in the network channel can be directly identified by the first remote host and the second remote host.

[0210] The amount of data transmitted in the first interactive data transmission of each first data type varies, and therefore the bandwidth required for transmission in the network channel also varies. For example, when transmitted in the network channel, endoscopic video requires 30 Mbps of bandwidth, operating room pan / tilt video requires 10 Mbps of bandwidth, operating room bolt camera video requires 10 Mbps of bandwidth, audio data requires 20 Mbps of bandwidth, master-slave control signaling data requires 10 Mbps of bandwidth, pairing data requires 5 Mbps of bandwidth, and energy control data requires 5 Mbps of bandwidth. The actual required bandwidth value can be set based on demand and is not specifically limited here.

[0211] In this embodiment, at least one of the first remote host and the second remote host obtains historical interaction data types transmitted in the network channel, compares the first data type with the historical interaction data type, and determines a switching state of the first data type transmitted in the network channel; and adjusts the bandwidth of the network channel by the central server based on the switching state. The historical interaction data type refers to the most recent interaction data type transmitted last time.

[0212] For example, if the historical interaction data type is a pairing data type, and the first data type currently being transmitted is an endoscopic video data type, the switching state of the first data type being transmitted in the network channel is: the pairing data type is switched to the endoscopic video data type. Based on this switching state, the central server adjusts the bandwidth required for transmitting the pairing data through the network channel from 5 Mbps to 30 Mbps, which is the bandwidth required for transmitting the endoscopic video data. If the first data type currently being transmitted is the pairing data type, the switching state is: the pairing data type remains unchanged. The central server does not need to adjust the bandwidth of the network channel. Multiple first data types can be transmitted in the network channel, and the bandwidth of the network channel is adjusted based on the total bandwidth required for each first data type.

[0213] In one embodiment, the switching state includes a first switching state and a second switching state, wherein the first switching state includes a state of switching data types under different surgical stages or surgical tasks, and the second switching state includes a state of switching data types under the same surgical stage or surgical task; the duration of the first switching state is monitored, and when the duration exceeds a preset first duration, the bandwidth of the network channel is adjusted according to a preset first amplitude; or, the duration of the second switching state is monitored, and when the duration exceeds a preset second duration, the bandwidth of the network channel is adjusted according to a preset second amplitude; wherein the first amplitude is greater than the second amplitude, and the preset first duration is less than the preset second duration.

[0214] In this embodiment, the first switching state and the second switching state respectively preset the first data type included before and after the switching; a simple example is shown below:

[0215] 1) In the first switching state, the first data type before switching includes: a pairing data type; the first data type after switching includes at least one of the following: an endoscope video data type, an operating room pan / tilt video data type, an operating room gun camera video data type, an audio data type, and a master-slave control signaling data type.

[0216] 2) In the first switching state, the first data type before switching includes at least one of the following: endoscope video data type, operating room pan / tilt video data type, operating room gun camera video data type, audio data type, master-slave control signaling data type; the first data type after switching includes: pairing data type.

[0217] 3) In the second switching state, the first data type before switching includes at least one of the following: an endoscope video data type, an operating room pan-tilt video data type, an operating room gun camera video data type, an audio data type, and a master-slave control signaling data type; the first data type after switching includes at least one of the following: an endoscope video data type, an operating room pan-tilt video data type, an operating room gun camera video data type, an audio data type, and a master-slave control signaling data type.

[0218] In this embodiment, the switching of the first data type indicates that the current telesurgical robot has entered the next surgical phase or a different surgical task. In other words, the first switching state indicates a surgical phase or task switch. This involves a significant increase or decrease in the transmission of data and a timely switch to the next surgical phase, necessitating a shorter first duration and a larger first amplitude adjustment bandwidth.

[0219] For example, the first interactive data may represent preoperative pairing data (10 Mbps); intraoperative endoscopic video data, operating room pan / tilt video data, operating room gun video data, audio data, master-slave control signaling data, and energy control data (60 Mbps); and postoperative pairing data (10 Mbps). Therefore, the first switching state of the first data type before, during, and after surgery requires a shorter first duration (such as real-time switching, 17 ms, 60 ms, etc.) and a larger first amplitude adjustment bandwidth (such as ±50 Mbps).

[0220] In this embodiment, the first data type switch indicates that the remote surgical robot is currently in the same surgical phase or task. The second switching state, therefore, represents a state switch within the same surgical phase or task. This involves increasing or decreasing the transmission of a relatively small amount of data and preventing surgical interruptions to allow for continued operation before switching to the next surgical phase. Therefore, a longer second duration and a smaller second amplitude adjustment bandwidth are required.

[0221] For example, the first interactive data may include intraoperative endoscopic video data (30 Mbps), operating room pan / tilt video data (10 Mbps), operating room bolt camera video data (10 Mbps), audio data (20 Mbps), master-slave control signaling data (10 Mbps), and energy control data (5 Mbps). Each transmission process may involve one or more types of first interactive data. Temporarily suspending transmission of some first interactive data does not indicate the end of the surgery and the entry into the postoperative phase. Therefore, the second switching state of multiple types of first data during surgery requires a longer first duration (e.g., 3 hours for the duration of the surgery) and a smaller first amplitude adjustment bandwidth (e.g., 5 Mbps, 10 Mbps, 20 Mbps).

[0222] To provide a specific example, when the first interaction data is transmitted over time T (t1, t2, ..., tn, where t1-t2 is a standard duration), a first switching state M1 and a second switching state M2 are determined, as well as a third switching state M0 that maintains the original data type. The first duration includes zero standard durations, and the second duration includes i standard durations. Then:

[0223] t1: {first interaction data: pairing data; switching state: M0; bandwidth: 10Mbps};

[0224] t2: {First interactive data: endoscope video data, operating room pan / tilt video data, operating room bolt camera video data, audio data; switching state: M1; bandwidth: 70Mbps};

[0225] t3: {First interactive data: endoscope video data, operating room pan / tilt video data, operating room bolt camera video data; switching state: M2; bandwidth: 70Mbps};

[0226] t4: {First interactive data: endoscope video data, operating room pan / tilt video data, operating room bolt camera video data; switching state: M2; bandwidth: 70Mbps};

[0227] …

[0228] Ti+3: {first interaction data: endoscope video data; switching state: M2; bandwidth: 50 Mbps};

[0229] Ti+4: {first interaction data: endoscope video data, pairing data; switching state: M1; bandwidth: 10 Mbps}.

[0230] Furthermore, the surgical stage includes at least any one of the following: power-on self-test, remote pairing, remote connection, remote interactive communication, remote surgical operation, remote pairing release, and power-off self-test; the surgical task includes any one of the following: surgical operation task, teaching task, and debugging task.

[0231] In this embodiment, it is determined whether the first data type contains a preset target data type; the configuration information of the first interactive data of the preset target data type is detected through at least one of the first remote host and the second remote host; and according to the configuration information, the bandwidth of the network channel is adjusted through the central server.

[0232] Specifically, target data types include at least endoscopic video data, operating room pan / tilt video data, and operating room bolt camera video data. Video data configuration information includes at least image resolution, video encoding format, and video transmission bitrate. Examples of the correspondence between configuration information and bandwidth (the corresponding bandwidth includes the bandwidth occupied by other data types) include: {Image resolution: 4k, Video encoding format: H265, Video transmission bitrate: 6000; Bandwidth: 100Mbps}, {Image resolution: 4k, Video encoding format: H265, Video transmission bitrate: 3000; Bandwidth: 70Mbps}, {Image resolution: 2k, Video encoding format: H265, Video transmission bitrate: 2500; Bandwidth: 50Mbps}.

[0233] Furthermore, the correspondence between configuration information and bandwidth can be determined based on different surgical task types. The correspondence between the aforementioned configuration information and bandwidth can be determined based on the surgical operation task, and the teaching task can set a lower corresponding bandwidth based on the same configuration information: {image resolution: 4k, video encoding format: H265, video transmission bit rate: 6000; bandwidth: 20Mbps}, {image resolution: 4k, video encoding format: H265, video transmission bit rate: 3000; bandwidth: 10Mbps} {image resolution: 2k, video encoding format: H265, video transmission bit rate: 2500; bandwidth: 10Mbps}. The debugging task can be set to: the same correspondence between configuration information and bandwidth as the surgical operation task, or the same correspondence between configuration information and bandwidth as the teaching task.

[0234] In this embodiment, the first interaction data includes: third interaction data transmitted from the remote doctor console to the patient surgery platform, and fourth interaction data sent from the patient surgery platform to the doctor console, and a master-slave data type priority is preset between the third data type of the third interaction data and the fourth data type of the fourth interaction data; the central server is respectively communicated with at least one end of the remote doctor console and the patient surgery platform; according to the master-slave data type priority, one of the data types is selected from the third data type and the fourth data type; according to the selected third data type or fourth data type, the bandwidth of the network channel is adjusted by the central server.

[0235] In this embodiment, when the central server is only connected to the remote doctor console or the patient surgery platform for communication, the third interaction data and the fourth interaction data are aggregated to one end of the two. One of the aggregated ends sends a first bandwidth adjustment request to the central server through the remote host based on the selected data type to adjust the bandwidth of the network channel.

[0236] Specifically, the remote doctor console also stores historical third interaction data, indicating the current surgical stage and surgical task of the remote surgical robot; when the patient surgical platform sends fourth interaction data to the remote doctor console, one of the data types is selected to determine whether the bandwidth needs to be adjusted; or the patient surgical platform also stores historical fourth interaction data, indicating the current surgical stage and surgical task of the remote surgical robot; when the remote doctor console sends third interaction data to the patient surgical platform, one of the data types is selected to determine whether the bandwidth needs to be adjusted.

[0237] In one embodiment, the master-slave data type priority can also be expressed as the control state priority: in the third and fourth interactive data, the data type with the lower matching bandwidth value has a higher priority. For example, the fourth interactive data represents the postoperative stage, and the matching bandwidth is 20Mbps, while the third interactive data represents the intraoperative stage, and the matching bandwidth is 60Mbps; indicating that the remote surgical robot should end the surgical operation at this time, the fourth data type has a higher priority and the bandwidth is adjusted to 20Mbps. The above example of relative control state priority also applies to the example of master-slave data type priority here, so it will not be repeated here.

[0238] Wherein, one of the data types is selected from the third data type and the fourth data type according to the master-slave data type priority from high to low.

[0239] In this embodiment, the central server can also communicate with the remote doctor console and the patient surgery platform at the same time. The third interaction data and the fourth interaction data are collected in the central server, and the bandwidth of the network channel is adjusted based on the selected data type.

[0240] 1503. According to the first data type, adjust the bandwidth of the network channel through the central server.

[0241] In this embodiment, the central server is communicatively connected to a preset network operator server; the bandwidth parameters corresponding to the first data type are matched through at least one of the first remote host and the second remote host, and based on the bandwidth parameters, a first bandwidth adjustment request is sent to the central server; and based on the first bandwidth adjustment request, a second bandwidth adjustment request is initiated to the network operator server through the central server to adjust the bandwidth of the network channel through the network operator server.

[0242] In this embodiment, the bandwidth parameter may be matched with the first control state and the first connection state as well as the first data type, and the corresponding bandwidth parameter may be obtained based on the matching of the first connection state.

[0243] In one embodiment, the first interaction data includes multiple first data types, and the multiple first data types are preset with data type priorities; through at least one end of the first remote host and the second remote host, at least one first data type is selected from the multiple first data types according to the data type priority, and the corresponding bandwidth parameters are matched according to the selected first data type.

[0244] In this embodiment, the priority of the preset data type can be expressed as the control state priority: among the multiple first data types representing different surgical stages, the one with the lower matched bandwidth value has a higher priority. For example, if the paired data is transmitted in the postoperative stage, the bandwidth matched by the paired data type is 20Mbps, and if the endoscopic video data is transmitted in the intraoperative stage, the bandwidth matched by the endoscopic video data type is 60Mbps. This means that the remote surgical robot is currently releasing the pairing relationship and ending the surgical operation. The paired data type has a higher priority, and the bandwidth is adjusted to 20Mbps. The aforementioned example of relative control state priority also applies to the example of data type priority here, so it will not be repeated here.

[0245] Furthermore, data type priority can also be expressed as the importance of each first data type, with the higher the importance, the higher the priority. For example, the data types prioritized, from highest to lowest, include: endoscope video data type, master-slave control signaling data type, energy control data type, audio data type, operating room pan / tilt video data type, operating room bolt camera video data type, and paired data type. Based on the highest-priority data type, all data types corresponding to the surgical stage and surgical task are calculated, and the bandwidth parameters of all data types are matched.

[0246] In one embodiment, at least one of the first terminal and the second terminal stores mapping information, and the bandwidth parameter includes multiple bandwidth levels in order of bandwidth, and the mapping information includes: a mapping relationship between multiple first data types and multiple bandwidth levels; through at least one of the first remote host and the second remote host, the target bandwidth level corresponding to the first data type is matched in the mapping information stored in the corresponding first terminal and the second terminal, and a first bandwidth adjustment request is initiated to the central server based on the target bandwidth level.

[0247] In this embodiment, bandwidth levels of different first data types are set to determine preset bandwidth sizes for different surgical stages and surgical tasks. The larger the bandwidth preset, the higher the level, and the smaller the bandwidth preset, the lower the level. For example, four bandwidth levels are set from low to high: 20Mbps, 30Mbps, 40Mbps, and 60Mbps, then: {pre-operative: 20Mbps; intra-operative: 60Mbps; post-operative: 20Mbps; surgical operation task: 60 Mbps; debugging task: 40Mbps; teaching task: 30Mbps}.

[0248] Each bandwidth level corresponds to a bandwidth value range or a bandwidth value. The first bandwidth adjustment request includes the bandwidth value range or the bandwidth value. After receiving the first bandwidth request, the central server queries the network operator server for the bandwidth size of the current network channel: if it is greater than the bandwidth value range, the bandwidth size is adjusted to the upper limit of the bandwidth value range; if it is less than the bandwidth value range, the bandwidth size is adjusted to the lower limit of the bandwidth value range; the bandwidth size of the current network channel is adjusted to the requested bandwidth value.

[0249] In addition, when both the remote doctor console and the patient surgery platform store mapping information, they respectively match the target bandwidth level of the first data type output by themselves, and initiate a first bandwidth adjustment request to the central server based on their respective target bandwidth levels.

[0250] When only the remote doctor console stores the mapping information, the patient surgery platform sends its first data type to the remote doctor console, and the remote doctor console uniformly matches the target bandwidth level of the first data type output by itself and the patient surgery platform. At this time, the remote doctor console initiates a first bandwidth adjustment request to the central server based on the target bandwidth level.

[0251] When only the patient surgery platform stores the mapping information, the remote doctor console sends its output first data type to the patient surgery platform. The patient surgery platform uniformly matches the target bandwidth level of the first data type output by itself and the remote doctor console. At this time, the patient surgery platform initiates a first bandwidth adjustment request to the central server based on the target bandwidth level.

[0252] In addition, the mapping information can also be stored in the central server, which will uniformly match the target bandwidth level corresponding to the first data type output by the remote doctor console and the patient surgery platform based on the stored mapping information, and directly initiate a second bandwidth adjustment request to the network operator server based on the target bandwidth level.

[0253] In this embodiment, at least one auxiliary surgical device is further included, and at least one end of the first terminal and the second terminal is detachably connected to the at least one auxiliary surgical device; in response to a second interactive operation performed by the first terminal or the second terminal with the auxiliary surgical device, second interactive data is generated and transmitted between the first remote host and the second remote host via the network channel; through at least one of the first remote host and the second remote host, a second data type of the second interactive data transmitted in the network channel is identified, and according to a preset relative type priority between the first data type and the second data type, one of the first data type and the second data type is selected; according to the selected first data type or second data type, the bandwidth of the network channel is adjusted through the central server.

[0254] In one embodiment, the preset relative type priority can also be expressed as the control state priority: between the first data type and the second data type, the lower the matched bandwidth value, the higher its priority. The preset relative type priority can also be expressed as the control state priority: the importance of the first interaction data relative to the second interaction data is used to determine whether the bandwidth needs to be adjusted or the size of the adjusted bandwidth to meet the required bandwidth of the surgical project. The aforementioned example of relative control state priority is also applicable to the example of relative type priority here, so it will not be repeated. Among them, one of the data types is selected from the first data type and the second data type according to the relative type priority from high to low.

[0255] 7. Bandwidth Allocation for Multiple Remote Surgical Robots

[0256] The bandwidth allocation system includes: multiple groups of remote surgical robots, each group of the remote surgical robots includes a first terminal and a second terminal that are communicatively connected, wherein the first terminal and the second terminal of each group are communicatively connected in the same network channel, and one end is a remote doctor console and the other end is a patient surgery platform; a central server, which is communicatively connected to at least one of the first terminal and the second terminal of each group of the remote surgical robots; a processor, which is configured to: obtain interaction information between the first terminal and the second terminal of each group, and determine the surgical scene of each group of the remote surgical robots based on the interaction information; allocate bandwidth to each group of the remote surgical robots based on each surgical scene and the current bandwidth in the network channel, so that the central server can adjust the bandwidth of each group of the remote surgical robots.

[0257] For example, Figure 16 As shown, there are two groups of remote surgical robots, the first group of remote surgical robots includes a first terminal TX1 and a second terminal RX1, and the second group of remote surgical robots includes a first terminal TX2 and a second terminal RX2; when only the first terminal TX1 and the first terminal TX2 are respectively communicated with the central server, bandwidth is allocated to the two groups of remote surgical robots.

[0258] The two groups of remote surgical robots collect interaction information to the first terminal TX1 and the first terminal TX2, respectively. For example, the second terminal RX1 and the second terminal RX2 send the interaction information they generate to the first terminal TX1 and the first terminal TX2, respectively. The first terminal TX1 and the first terminal TX2 check the surgical scene in real time according to the surgical process / according to the collected interaction information according to a preset period. When the surgical scene remains unchanged, there is no need to request bandwidth adjustment. When the surgical scene changes, a request is made to the central server to adjust the bandwidth based on the changed surgical scene.

[0259] After receiving the request, the central server allocates bandwidth to the two groups of remote surgical robots according to the surgical scenario for which bandwidth adjustment is requested, and initiates a bandwidth adjustment request to the network operator server based on the allocated bandwidth: first, the network operator server is queried for the node bandwidth of the first terminal TX1 and the first terminal TX2, and the node bandwidth is returned based on the confirmation query information; the central server sends its own identity information, or the identity information of the first terminal TX1 and the first terminal TX2, for authentication by the operator server to ensure the security of bandwidth adjustment; after authentication is passed, the node bandwidth of the first terminal TX1 and the first terminal TX2 can be adjusted to the requested allocated bandwidth; after receiving the adjusted bandwidth size, the central server forwards it to the first terminal TX1 and the first terminal TX2, and can prompt the first terminal TX1 and the first terminal TX2.

[0260] Please refer to Figure 17 A flow chart of a bandwidth allocation method is provided, which provides an embodiment of the bandwidth allocation method, as shown below:

[0261] 1701. Obtain interaction information between the first terminal and the second terminal through each group of the remote surgical robots, and determine a surgical scenario for each group of the remote surgical robots based on the interaction information.

[0262] In this embodiment, the interaction information includes at least one of the following: a first control state of the telesurgical robot when the first terminal interacts with the second terminal, and interaction data transmitted in the network channel; and at least one auxiliary surgical device. The interaction information may also include a second control state of the telesurgical robot when the first terminal and / or the second terminal interacts with the detachably connected auxiliary surgical device; wherein at least one of the first and second terminals is detachably connected to the at least one auxiliary surgical device. A surgical scenario for each group of telesurgical robots is determined based on at least one of the first control state, the second control state, and the interaction data.

[0263] In this embodiment, the surgical scenario includes at least one of the following: scenarios under different surgical stages, scenarios under different surgical tasks, and scenarios under different surgical types, wherein the surgical stage includes at least any one of the following: power-on self-test, remote pairing, remote connection, remote interactive communication, remote surgical operation, remote pairing release, and power-off self-test; the surgical task includes at least any one of the following: surgical operation task, teaching task, and debugging task; the surgical type includes any one of surgical types with different degrees of urgency.

[0264] In one embodiment, each remote surgical robot group further includes a first voice acquisition device corresponding to the first terminal and a second voice acquisition device corresponding to the second terminal; and / or further includes an in-vivo image acquisition device corresponding to the patient surgical platform. Medical staff at the remote physician console and the patient surgical platform can communicate via the first and second voice acquisition devices.

[0265] Among them, the first voice acquisition device and the second voice acquisition device are used to acquire voice communication information during the operation, and the in-vivo image acquisition device is used to acquire anatomical scene images during the operation. The voice communication information and the anatomical scene image are transmitted through the network channel, and the interactive information includes the voice communication information and / or the anatomical scene image; based on the voice communication information, the scene of the corresponding remote surgical robot in any surgical stage, the scene under any surgical task, and / or the scene under any surgical type is determined; and / or, based on the anatomical scene image, the scene of the corresponding remote surgical robot in any surgical task, and / or the scene under any surgical type is determined.

[0266] Specifically, an in-vivo image acquisition device is installed at one end of the patient's surgical platform, and can be installed at the end of a surgical instrument. As the surgical instrument moves into the patient's body, it captures images of the anatomical scene within the patient's body. The patient's surgical platform is equipped with an image recognition algorithm; based on the anatomical scene images, the algorithm is applied to identify the scene under any surgical task and / or any surgical type. Image recognition algorithms include machine learning algorithms and deep learning algorithms, such as convolutional neural networks, recurrent neural networks, and support vector machines.

[0267] Specifically, at least one end of the patient's surgical platform and the remote doctor's console is equipped with a speech recognition algorithm, which extracts characteristic keywords based on the voice communication information during the communication process between medical staff at both ends, and identifies scenes in any surgical stage, any surgical task, and / or any surgical type based on the characteristic words.

[0268] For example, relevant feature keywords such as "cut," "suture," and "aspiration" can be used to identify the surgical scene currently in the intraoperative phase. Other feature keywords can be collected based on prior knowledge to establish a feature keyword library for identifying surgical scenes. Specific speech recognition algorithms can include those based on dynamic time warping (DTW), parameter-based hidden Markov models (HMMs), convolutional neural networks, and deep learning neural networks.

[0269] In addition, different types of surgeries are classified according to the urgency of the surgery; the classification is based on at least one of the following: preoperative evaluation information and intraoperative surgical information. Preoperative evaluation information such as patient age, surgical risk, surgical duration, surgical site, etc.; intraoperative surgical information such as patient bleeding status, whether an emergency accident occurred, surgical progress, etc.

[0270] 1702. Allocate bandwidth to each group of the remote surgical robots based on each surgical scenario and the current bandwidth in the network channel, so that the central server can adjust the bandwidth of each group of the remote surgical robots.

[0271] In this embodiment, the preset required bandwidth of each surgical scenario is obtained, and the total required bandwidth of each preset required bandwidth is determined; the total required bandwidth is compared with the current bandwidth in the network channel; if the total required bandwidth is higher than the current bandwidth in the network channel, at least one of the preset required bandwidth and the current bandwidth in the network channel of at least one group of remote surgical robots is adjusted so that the adjusted total required bandwidth is not higher than the current bandwidth in the network channel; based on the adjusted preset required bandwidth and / or current bandwidth, the bandwidth allocated to each group of remote surgical robots is determined.

[0272] For example, the first group of telesurgery robots performs preoperative surgery, with a preset bandwidth requirement of 20 Mbps. The second group performs intraoperative surgery, with a preset bandwidth requirement of 80 Mbps. The third group performs postoperative surgery, with a preset bandwidth requirement of 20 Mbps. The total bandwidth requirement is 120 Mbps. The current bandwidth of the network channel is 100 Mbps. In one solution, the total bandwidth requirement is adjusted to no higher than 100 Mbps; in another solution, the current bandwidth is adjusted to higher than 120 Mbps.

[0273] Among them, when the total required bandwidth is 120Mbps, if the total required bandwidth is adjusted to not higher than 100Mbps, it is necessary to reduce the preset required bandwidth of the first group of remote surgical robots (lower than 20Mbps), the preset required bandwidth of the second group of remote surgical robots (lower than 80Mbps), and / or the preset required bandwidth of the third group of remote surgical robots (lower than 20Mbps).

[0274] In one embodiment, multiple surgical scenes are preset with scene priorities, and the network channel is preset with a bandwidth threshold; if the total required bandwidth is higher than the current bandwidth in the network channel, the transmission frequency of at least one group of the remote surgical robots is reduced in sequence according to the scene priority; based on the reduced transmission frequency, the reduced preset required bandwidth corresponding to at least one of the surgical scenes is determined, and the reduced total required bandwidth is determined based on the reduced preset required bandwidth; if the reduced total required bandwidth is higher than the current bandwidth in the network channel and not higher than the bandwidth threshold, the current bandwidth in the network channel is increased to the reduced total required bandwidth.

[0275] In this embodiment, examples of bandwidth thresholds for network channel settings include 100 Mbps, 150 Mbps, 200 Mbps, 300 Mbps, and 500 Mbps. Bandwidth thresholds are set based on the bandwidth occupied by multiple groups of remote surgical robots that require compatibility. This embodiment describes how, when the total required bandwidth is higher than the current bandwidth, the transmission frequency of the remote surgical robot is prioritized to adjust the total required bandwidth, followed by the current bandwidth of the network channel.

[0276] In this embodiment, for different surgical stages, the scene priorities are set from high to low as: intraoperative stage (remote interactive communication, remote surgical operation), preoperative stage (power-on self-test, remote pairing, remote connection), and postoperative stage (remote pairing release, remote pairing switch, power-off self-test); for different surgical tasks, the scene priorities are set from high to low as: surgical operation tasks, teaching tasks, and debugging tasks; for different surgical types, the scene priorities are set from high to low as: urgency from high to low.

[0277] For example, for example, three groups of remote surgical robots are respectively in the preoperative stage, intraoperative stage and postoperative stage; then, according to the scene priority, the transmission frequency of the remote surgical robots in the postoperative stage is reduced in turn, and the required bandwidth of this group of remote surgical robots is reduced, so that the total required bandwidth is reduced; if the reduced total required bandwidth is still higher than the current bandwidth, the transmission frequency of the remote surgical robots in the preoperative stage is further reduced; and so on, and finally the transmission frequency of the remote surgical robots in the intraoperative stage continues to be reduced.

[0278] In one embodiment, multiple surgical scenarios are preset with scene priorities, and the network channel is preset with a bandwidth threshold; if the total required bandwidth is higher than the current bandwidth in the network channel and not higher than the bandwidth threshold, the current bandwidth in the network channel is increased to the total required bandwidth; if the total required bandwidth is higher than the current bandwidth in the network channel and the bandwidth threshold, the current bandwidth in the network channel is increased to the bandwidth threshold; according to the scene priority, the transmission frequency of at least one group of the remote surgical robots is reduced in turn; based on the reduced transmission frequency, the reduced preset required bandwidth corresponding to at least one of the surgical scenarios is determined, so that the reduced total required bandwidth is not higher than the bandwidth threshold of the network channel.

[0279] This embodiment further introduces that when the total required bandwidth is higher than the current bandwidth, the current bandwidth of the network channel is adjusted first, and then the transmission frequency of the remote surgical robot is adjusted to adjust the total required bandwidth.

[0280] In one embodiment, the interaction data includes multiple data types, and the multiple interaction data are preset with data type priorities according to different data types, and the network channel is preset with a bandwidth threshold; if the total required bandwidth is higher than the current bandwidth in the network channel and the bandwidth threshold, the current bandwidth in the network channel is increased to the bandwidth threshold of the network channel; according to the data type priority, the preset required bandwidth corresponding to at least one data type is reduced in each surgical scenario in turn, so that the process stops when the reduced total required bandwidth is no higher than the bandwidth threshold of the network channel.

[0281] This embodiment further introduces that when the total required bandwidth is higher than the bandwidth threshold, the current bandwidth of the network channel is adjusted first, and then the preset required bandwidth of the remote surgical robot is adjusted to adjust the total required bandwidth.

[0282] Exemplarily, the aforementioned data types include, from high to low priority: endoscope video data type, master-slave control signaling data type, energy control data type, audio data type, operating room pan / tilt video data type, operating room gun camera video data type, and pairing data type; the preset required bandwidth corresponding to one data type in all surgical scenarios can be reduced each time according to the data type priority from low to high.

[0283] For example, the first surgical scenario includes an endoscope video data type, an audio data type, and an operating room gun camera video data type; the second surgical scenario includes an endoscope video data type, a master-slave control signaling data type, an energy control data type, an audio data type, and an operating room pan / tilt video data type; the third surgical scenario includes a pairing data type, an audio data type, an operating room pan / tilt video data type, and an operating room gun camera video data type; the specific examples of the data type reduction process are as follows:

[0284] First, it reduces the required bandwidth of paired data types in the third surgical scenario;

[0285] The second reduction is to reduce the bandwidth required for the operating room PTZ video data type in the second and third surgical scenarios;

[0286] The third step is to reduce the bandwidth required for the operating room gun-mounted video data type in the first and third surgical scenarios;

[0287] The process continues in this way until the reduced total required bandwidth is no higher than the bandwidth threshold of the network channel.

[0288] In one embodiment, it also includes an in vitro image acquisition device, which is communicatively connected to a first terminal or a second terminal detachably connected to the auxiliary surgical device; the surgical scene image is acquired by the in vitro image acquisition device, wherein the surgical scene image includes the auxiliary surgical device and an image of the first terminal or the second terminal detachably connected to the auxiliary surgical device; according to the surgical scene image, the disassembly and assembly operation of the first terminal or the second terminal and the auxiliary surgical device is identified through the first terminal or the second terminal detachably connected to the auxiliary surgical device, and the second control state of the remote surgical robot is determined based on the disassembly and assembly operation.

[0289] Furthermore, the central server is communicatively connected to the network operator server; based on the current bandwidth adjusted in the network channel, a bandwidth adjustment request is initiated to the network operator server through the central server, so that the network channel is adjusted to the adjusted current bandwidth through the network operator server; wherein the adjusted current bandwidth includes the total required bandwidth, the adjusted total required bandwidth, or the preset bandwidth threshold of the network channel.

[0290] It should be noted that other sorting schemes that can be easily thought of by those skilled in the art within the technical scope disclosed in the present invention should also be within the scope of protection of the present invention, and they will not be described in detail here.

[0291] Those skilled in the art can clearly understand that for the convenience and brevity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the mobile terminal can be divided into different functional units or modules to complete all or part of the functions described above. The functional modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the modules in the above-mentioned mobile terminal can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0292] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0293] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0294] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0295] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0296] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if the described condition or event is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of the described condition or event" or "in response to detecting the described condition or event," depending on the context.

[0297] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0298] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0299] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0300] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0301] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0302] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0303] If the integrated module / unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the process steps in the above-mentioned method embodiments by using a computer program to instruct the relevant hardware. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. Computer-readable media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, removable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signals, telecommunications signals, and software distribution media. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased based on the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electric carrier signals and telecommunications signals.

[0304] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A bandwidth adjustment method for a remote surgical robot, characterized in that: include: A first remote host and a second remote host, wherein an Internet channel is established between the first remote host and the second remote host; A remote surgical robot, comprising a first terminal connected to the first remote host, and a second terminal connected to the second remote host, wherein one end of the first terminal and the second terminal is a remote doctor console, and the other end is a patient surgical platform, wherein the nodes representing the remote doctor console and the patient surgical platform in the cloud networking are used to communicate through the Internet channel; a central server connected to at least one of the first remote host and the second remote host; The bandwidth adjustment method comprises: In response to a first interactive operation performed by the first terminal on the second terminal, first interactive data is generated, and transmitted to the second remote host via the first remote host through the Internet channel, so that the second terminal can perform motion control based on the first interactive data; identifying, by at least one of the first remote host and the second remote host, a first data type of first interactive data transmitted in the internet channel; According to the first data type, adjusting the bandwidth of the Internet channel by the central server; According to the first data type, adjusting the bandwidth of the Internet channel through the central server includes: obtaining, through at least one of the first remote host and the second remote host, a historical interactive data type transmitted in the Internet channel, and comparing the first data type with the historical interactive data type to determine a switching state of the first data type transmitted in the Internet channel, and adjusting the bandwidth of the Internet channel through the central server according to the switching state; The switching state includes a first switching state and a second switching state, the first switching state includes a state of switching data types under different surgical stages or surgical tasks, and the second switching state includes a state of switching data types under the same surgical stage or surgical task; according to the switching state, the bandwidth of the Internet channel is adjusted by the central server, including: monitoring the duration of the first switching state, when the duration exceeds a preset first duration, adjusting the bandwidth of the Internet channel according to a preset first amplitude; or monitoring the duration of the second switching state, when the duration exceeds a preset second duration, adjusting the bandwidth of the Internet channel according to a preset second amplitude; the first amplitude is greater than the second amplitude, and the preset first duration is less than the preset second duration.

2. The bandwidth adjustment method according to claim 1, characterized in that: The surgical stage includes at least any one of the following: power-on self-test, remote pairing, remote connection, remote interactive communication, remote surgical operation, remote pairing release, and power-off self-test; The surgical task includes any one of the following: a surgical operation task, a teaching task, and a debugging task.

3. The bandwidth adjustment method according to claim 1 or 2, characterized in that: Also included is at least one auxiliary surgical device, at least one end of the first terminal and the second terminal being detachably connected to the at least one auxiliary surgical device; According to the first data type, adjusting the bandwidth of the Internet channel by the central server includes: In response to a second interactive operation performed between the first terminal or the second terminal and the surgical assistance device, second interactive data is generated and transmitted between the first remote host and the second remote host via the Internet channel; identifying, by at least one of the first remote host and the second remote host, a second data type of the second interactive data transmitted in the internet channel, and selecting one of the first data type and the second data type according to a preset relative type priority between the first data type and the second data type; The bandwidth of the Internet channel is adjusted by the central server according to the selected first data type or second data type.

4. The bandwidth adjustment method according to claim 1 or 2, characterized in that: According to the first data type, adjusting the bandwidth of the Internet channel by the central server includes: Determining whether the first data type includes a preset target data type; Detecting, through at least one of the first remote host and the second remote host, configuration information of first interactive data of the preset target data type; According to the configuration information, the bandwidth of the Internet channel is adjusted by the central server.

5. The bandwidth adjustment method according to claim 1 or 2, characterized in that: The first interaction data includes: third interaction data transmitted from the remote doctor console to the patient surgery platform, and fourth interaction data sent from the patient surgery platform to the remote doctor console, and a master-slave data type priority is preset between the third data type of the third interaction data and the fourth data type of the fourth interaction data; The central server is respectively connected to at least one end of the remote doctor console and the patient surgery platform for communication; According to the first data type, adjusting the bandwidth of the Internet channel by the central server includes: selecting one of the data types from the third data type and the fourth data type according to the master-slave data type priority; The bandwidth of the Internet channel is adjusted by the central server according to the selected third data type or fourth data type.

6. A bandwidth adjustment system for a remote surgical robot, using the bandwidth adjustment method for a remote surgical robot as claimed in any one of claims 1 to 5, characterized in that: The bandwidth adjustment system comprises: A first remote host and a second remote host, wherein an Internet channel is established between the first remote host and the second remote host; A remote surgical robot, comprising a first terminal connected to the first remote host, and a second terminal connected to the second remote host, wherein one end of the first terminal and the second terminal is a remote doctor console, and the other end is a patient surgical platform, wherein the nodes representing the remote doctor console and the patient surgical platform in the cloud networking are used to communicate through the Internet channel; a central server connected to at least one of the first remote host and the second remote host; The processor is configured as: In response to a first interactive operation performed by the first terminal on the second terminal, first interactive data is generated, and transmitted to the second remote host via the first remote host through the Internet channel, so that the second terminal can perform motion control based on the first interactive data; identifying, by at least one of the first remote host and the second remote host, a first data type of first interactive data transmitted in the internet channel; According to the first data type, adjusting the bandwidth of the Internet channel by the central server; According to the first data type, adjusting the bandwidth of the Internet channel through the central server includes: obtaining, through at least one of the first remote host and the second remote host, a historical interactive data type transmitted in the Internet channel, and comparing the first data type with the historical interactive data type to determine a switching state of the first data type transmitted in the Internet channel, and adjusting the bandwidth of the Internet channel through the central server according to the switching state; The switching state includes a first switching state and a second switching state, the first switching state includes a state of switching data types under different surgical stages or surgical tasks, and the second switching state includes a state of switching data types under the same surgical stage or surgical task; according to the switching state, the bandwidth of the Internet channel is adjusted by the central server, including: monitoring the duration of the first switching state, when the duration exceeds a preset first duration, adjusting the bandwidth of the Internet channel according to a preset first amplitude; or monitoring the duration of the second switching state, when the duration exceeds a preset second duration, adjusting the bandwidth of the Internet channel according to a preset second amplitude; wherein the first amplitude is greater than the second amplitude, and the preset first duration is less than the preset second duration.

7. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the bandwidth adjustment method of the remote surgical robot according to any one of claims 1 to 5 are implemented.

Images