Remote surgical robot system and method for interconnecting its equipment

Abstract

The present disclosure relates to a remote surgical robot system and a method for interconnecting its devices. The method includes displaying the first device information obtained and registered to the server on the configuration interface; obtaining the target first device information associated with the target first device determined based on the first device information; matching the second device information that can be paired with the second device based on the target first device information and displaying it; obtaining the target second device information associated with the target second device determined based on the second device information; obtaining the device network information of at least one of the target first device and the target second device; sending the at least one device network information to the opposite device associated with the device network information, so that the target first device and the target second device can establish pairing. The server directly obtains the remote doctor console and the patient surgical platform for pairing, and sends the device network information of at least one of the two to the opposite device to establish pairing, which can improve pairing efficiency and facilitate batch pairing.

Description

Technical Field

[0001] The present disclosure relates to the technical field of surgical robots, and in particular to a remote surgical robot system and a method for interconnecting devices thereof. Background Art

[0002] In recent years, surgical robot technology has flourished, resulting in surgical robots for different applicable scenarios such as orthopedic surgical robots, interventional surgical robots, and laparoscopic surgical robots. For doctors, they have the advantages of easy operation and high precision. For patients, they have the advantages of less trauma, less pain, and faster recovery, and are widely accepted by doctors and patients.

[0003] The surgical robot consists of a doctor's console and a patient operating platform. The doctor's console includes a display screen and an operating unit, while the patient operating platform includes one or more manipulator assemblies, each consisting of a manipulator and a medical device detachably mounted on the manipulator. During operation, the doctor manipulates the operating unit to generate control commands within the surgical field of view provided by the imaging device in the medical device, as displayed on the display screen. The manipulator then drives the medical device to perform the surgical operation based on these control commands.

[0004] With the increasing maturity of low-latency, high-bandwidth high-speed communication network technologies, such as 5G and the internet, telesurgery has emerged, combining the advantages of surgical robotics and high-speed communication networks. This technology allows surgeons to perform ultra-remote surgeries on patients in remote locations, for example, in different countries or provinces. This significantly reduces the constraints of space on surgical operations and effectively alleviates the uneven distribution of medical resources.

[0005] A telesurgery robotic system consists of a remote physician console and a patient surgical platform, both of which are typically deployed in different countries, provinces, or within different jurisdictions of the same city. Before ultra-telesurgical procedures can be performed, communication between the remote physician console and the patient surgical platform must be established. This is particularly true when a large number of either or both of these platforms are deployed. How to conveniently and reliably establish communication between the desired remote physician console and the desired patient surgical platform becomes a pressing technical challenge. Summary of the Invention

[0006] Based on this, it is necessary to provide a method for interconnecting a remote surgical robot system and its equipment that can conveniently and reliably establish a pairing between a remote doctor console and a patient surgical platform.

[0007] In one aspect, the present disclosure provides a method for interconnecting devices, the method being applied to a server in a remote surgical robotic system. The system includes the server, and a first device and a second device capable of communicating with the server. The first device includes one of a remote physician console and a patient surgical platform operable by the remote physician console, and the second device includes the other of the remote physician console and the patient surgical platform. The server includes a display screen. The method comprises:

[0008] Displaying the acquired first device information of the first device registered with the server in a configuration interface of the display screen;

[0009] Acquire target first device information associated with a target first device determined based on the first device information, where the target first device is used to pair with the second device;

[0010] Matching, based on the target first device information, second device information of a pairable second device registered with the server and capable of being paired with the target first device, and displaying the second device information on the configuration interface;

[0011] Acquire target second device information associated with the target second device determined based on the second device information;

[0012] Acquire device network information of at least one of the target first device and the target second device; and

[0013] The at least one device network information is sent to a peer device associated with the at least one device network information, so that the target first device and the target second device establish pairing to achieve direct communication.

[0014] In which, the first device information is displayed on the configuration interface through a list control, and different first device information is associated with different list options of the list control; the obtaining of target first device information associated with the target first device determined based on the first device information includes: in response to detecting that the target list option in the list options is selected, determining that the first device information of the first device associated with the target list option is the target first device information of the target first device for pairing.

[0015] Among them, the step of matching the second device information of the pairable second device registered to the server and capable of being paired with the target first device based on the target first device information, and displaying it in the configuration interface includes: in response to detecting that the target list option in the list options is selected, based on the target first device information associated with the target list option, matching the second device information of the pairable second device that can be paired with the target first device from the second devices registered to the server, and displaying it in the configuration interface through another list control, wherein different second device information is associated with different list options of the list control.

[0016] Among them, the step of obtaining the target second device information associated with the target second device determined based on the second device information includes: in response to detecting that the target list option in the other list option is selected, determining that the second device information of the second device associated with the target list option is the target second device information of the target second device to be paired.

[0017] Among them, the first device information is displayed on the configuration interface through a first drag control, different first device information is associated with different first drag controls, and the configuration interface includes a window control; the acquisition of target first device information associated with the target first device determined based on the first device information includes: in response to detecting that the target first drag control in the first drag control moves into the window of the window control, determining that the first device information of the first device associated with the target first drag control is the target first device information of the target first device for pairing.

[0018] Among them, the step of matching the second device information of the pairable second device registered with the server and capable of being paired with the target first device based on the target first device information, and displaying it in the configuration interface includes: in response to detecting that the target first drag control in the first drag control is moved into the window of the window control, matching the second device information of the pairable second device that can be paired with the target first device from the second device registered with the server based on the target first device information associated with the target first drag control, and displaying it in the configuration interface outside the window through a second drag control, different second device information is associated with different second drag controls.

[0019] Among them, the step of obtaining the target second device information associated with the target second device determined based on the second device information includes: in response to detecting that the target second drag control in the second drag control moves into the window, determining that the second device information of the second device associated with the target second drag control is the target second device information of the target second device to be paired.

[0020] In which, the window control includes a first area and a second area, the first area includes a first sub-area and a second sub-area, the first sub-area and the second sub-area are used to accommodate the first drag control, and the second area is used to accommodate the second drag control, the first sub-area and the second sub-area are associated with different control permissions or different pairing orders, and when the first drag control is associated with the remote doctor console and the second drag control is associated with the patient surgical platform, the method also includes: in response to the second drag control moving to the second area, one first drag control moving to the first sub-area, and another first drag control moving to the second sub-area, configuring the remote doctor console associated with different first drag controls to have different control permissions or pairing orders for the patient surgical platform associated with the second drag control.

[0021] Among them, the step of obtaining device network information of at least one of the target first device and the target second device includes: obtaining device network information of at least one of the target first device and the target second device in response to the triggering of an event, and the event includes the timing starting from determining the target second device information reaching a delay time, or a control in the configuration interface is triggered.

[0022] Among them, when the target first device includes one and the target second device includes at least one, the pairable second device includes the second device that supports the custom pairing protocol and is compatible with the custom pairing protocol supported by the target first device; or, when the target first device includes at least one and the target second device includes one, the pairable second device includes the second device that supports the custom pairing protocol and is compatible with the custom pairing protocol supported by each of the target first devices.

[0023] The method further includes: receiving a registration request sent by a corresponding device, and authenticating the registration request based on a judgment rule agreed upon between the server and the corresponding device, wherein the corresponding device includes the first device or the second device, and the judgment rule includes at least one of a custom protocol or an encryption algorithm agreed upon between the server and the corresponding device; when the authentication is successful, accepting the registration of the corresponding device and storing the registration information of the corresponding device included in the registration request.

[0024] On the other hand, the present disclosure provides a remote surgical robot system, which includes the server, and a first device and a second device that can communicate with the server, the first device includes one of a remote doctor console and a patient surgical platform that can be manipulated by the remote doctor console, and the second device includes the other of the remote doctor console and the patient surgical platform, and the server is configured to implement the method described in any one of the above embodiments.

[0025] On the other hand, the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions are executed on at least one processor, the method described in any one of the above embodiments is implemented.

[0026] The disclosed remote surgical robot system and the method for interconnecting its devices have the following beneficial effects:

[0027] The server directly obtains the remote doctor console and patient surgery platform for pairing, and sends the device network information of at least one of the two to the peer device to establish pairing, which can improve pairing efficiency and facilitate batch pairing. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a schematic structural diagram of an embodiment of a master-slave surgical robot suitable for a remote surgical robot system disclosed herein;

[0029] Figure 2 For Figure 1 A schematic structural diagram of an embodiment of a surgical instrument of a master-slave surgical robot is shown;

[0030] Figure 3 This is a schematic structural diagram of another embodiment of a slave robot in a master-slave surgical robot suitable for a remote surgical robot system disclosed herein;

[0031] Figure 4 This is a schematic structural diagram of another embodiment of a master-slave surgical robot suitable for a remote surgical robot system disclosed herein;

[0032] Figure 5 This is a network topology diagram of an embodiment of the telesurgery robot system disclosed herein;

[0033] Figure 6 This is a schematic diagram of a pairing scenario for the disclosed remote surgical robot system;

[0034] Figure 7 This is a flowchart of an embodiment of a device registration method disclosed herein;

[0035] Figure 8This is a schematic diagram of the structure of an embodiment of a custom protocol for inter-device communication disclosed herein;

[0036] Figure 9 A flowchart of an embodiment of a method for interconnecting devices disclosed herein;

[0037] Figure 10 A flowchart of another embodiment of the method for interconnecting devices disclosed herein;

[0038] Figure 11 A flowchart of another embodiment of the method for interconnecting devices disclosed herein;

[0039] Figure 12 A flowchart of another embodiment of the method for interconnecting devices disclosed herein;

[0040] Figures 13 to 15 A schematic diagram of an interactive interface according to an embodiment of a method for interconnecting devices disclosed herein;

[0041] Figures 16 to 19 A schematic diagram of an interactive interface of another embodiment of the method for interconnecting devices disclosed herein;

[0042] Figure 20 to Figure 22 A schematic diagram of an interactive interface of another embodiment of the method for interconnecting devices disclosed herein;

[0043] Figure 23 A flowchart of another embodiment of the method for interconnecting devices disclosed herein;

[0044] Figure 24 A flowchart of another embodiment of the method for interconnecting devices disclosed herein;

[0045] Figures 25 to 28 A schematic diagram of an interactive interface according to an embodiment of a method for interconnecting devices disclosed herein;

[0046] Figures 29 to 31 A schematic diagram of an interactive interface of another embodiment of the method for interconnecting devices disclosed herein;

[0047] Figures 32 to 34 A schematic diagram of an interactive interface of another embodiment of the method for interconnecting devices disclosed herein;

[0048] Figure 35 A flowchart of another embodiment of the method for interconnecting devices disclosed herein;

[0049] Figure 36 A flowchart of another embodiment of the method for interconnecting devices disclosed herein;

[0050] Figure 37 A flowchart of another embodiment of the method for interconnecting devices disclosed herein;

[0051] Figure 38A flowchart of another embodiment of the method for interconnecting devices disclosed herein;

[0052] Figure 39 A flowchart of another embodiment of the method for interconnecting devices disclosed herein;

[0053] Figure 40 The present invention is a flowchart of another embodiment of the method for interconnecting devices disclosed herein. DETAILED DESCRIPTION

[0054] To facilitate understanding of the present disclosure, a more comprehensive description of the present disclosure will be provided below with reference to the accompanying drawings. The accompanying drawings illustrate preferred embodiments of the present disclosure. However, the present disclosure can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the disclosure.

[0055] It should be noted that when an element is referred to as being "disposed on" another element, it may be directly on the other element or there may also be an element centered. When an element is considered to be "connected" to another element, it may be directly connected to the other element or there may also be an element centered. When an element is considered to be "coupled" to another element, it may be directly coupled to the other element or there may also be an element centered. The terms "vertical", "horizontal", "left", "right" and similar expressions used in this disclosure are for illustrative purposes only and do not represent the only implementation method. The terms "end" and "proximal end" used in this disclosure are directional words, which are conventional terms in the field of medical devices, where "end" refers to the end away from the operator during surgery and "proximal end" refers to the end close to the operator during surgery. The terms "first / second" and the like used in this disclosure refer to a component and a class of two or more components with common characteristics.

[0056] Unless otherwise defined, all technical and scientific terms used in this disclosure have the same meaning as commonly understood by those skilled in the art to which this disclosure belongs. The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit this disclosure. The term "and / or" as used in this disclosure includes any and all combinations of one or more of the associated listed items. The term "plurality" as used in this disclosure includes two or more.

[0057] The telesurgery robot system disclosed herein includes a master-slave surgical robot suitable for performing ultra-teleoperative surgery. The master-slave surgical robot includes a remote physician console and a patient surgical platform. The patient surgical platform can include different types of slave robots, including but not limited to single-port laparoscopic slave robots, multi-port laparoscopic slave robots, bronchial interventional slave robots, vascular interventional slave robots, and orthopedic slave robots. Different types of slave robots have different structural characteristics and may or may not be applicable to the same or different types of surgeries.

[0058] For example, in Figure 1 The master-slave surgical robot shown includes a remote physician console 100 and a patient surgical platform 200, which includes a single-port laparoscopic slave robot. The remote physician console 100 is used to send control commands to the patient surgical platform 200 based on the physician's operations to control the patient surgical platform 200. It is also used to display images captured by the patient surgical platform 200. The patient surgical platform 200 responds to control commands sent by the remote physician console 100 and performs corresponding operations. The patient surgical platform 200 is also used to capture images within the body.

[0059] The patient surgical platform 200 includes a robotic arm 210 and a manipulator assembly provided on the robotic arm 210. The manipulator assembly includes a manipulator 220 provided on the robotic arm 210 and a manipulator 220 provided on the manipulator 220. Figure 2 The surgical instrument 230 is shown. The patient surgical platform 200 also includes a puncture device 240 that is sheathed around the long shaft 231 of the surgical instrument 230. When the patient surgical platform 200 responds to control commands from the remote physician console 100, the robotic arm 210 is used to adjust the position of the surgical instrument 230, the manipulator 220 is used to drive the surgical instrument 230 to perform corresponding operations, and the end effector 232 of the surgical instrument 230 is used to extend into the body and perform surgical operations and / or obtain in-vivo images through its distal end instrument.

[0060] Figure 3Another patient surgical platform 200' is illustrated, which includes a multi-port laparoscope slave robot. The patient surgical platform 200' includes a robotic arm and multiple manipulator assemblies 230' disposed on the robotic arm. The robotic arm includes a main arm 210' and multiple adjustment arms 220' disposed on a directional platform 215' in the main arm 210'. Different manipulator assemblies 230' are disposed on different adjustment arms 220'. The main arm 210' can adjust the position of the adjustment arm 220' and the manipulator assembly 230', and the adjustment arm 220' can adjust the position of the manipulator assembly 230'. The manipulator assembly 230' includes a manipulator 240' and a medical device 250' detachably mounted on the manipulator 240'. The manipulator assembly 230' includes a parallelogram mechanism. Using the parallelogram principle, the manipulator 240' can be limited to perform rotational motion around a remote motion center (i.e., Remote Center, RC). Multiple medical devices 250 ′ can be inserted into the patient's body through different puncture devices 500 ′ respectively. Figure 1 The remote physician console 100 shown can also be used to operate Figure 3 The movement of manipulator assembly 230' is shown in patient surgical platform 200'.

[0061] For example, in Figure 4 An interventional surgical robot 300 is shown, which is also a natural orifice surgical robot, and includes a remote doctor console and a patient surgical platform 320. The remote doctor console includes a handle 310' and an imaging cart 330 that are interconnected, and / or the patient surgical platform 320 also includes an imaging cart 330. The patient surgical platform 320 is connected to a catheter instrument 340, a sensor system 350, and a control system 360 for achieving control between the catheter instrument 340, the sensor system 350, and the imaging cart 330. When a doctor performs various procedures on a patient next to the patient surgical platform 320, he or she can operate the handle 310' to trigger control instructions, which are sent to the patient surgical platform 320 for driving, thereby controlling the catheter instrument 340 to advance, retract, bend, and turn.

[0062] The patient surgical platform 320 can typically be moved to the side of the operating table to engage the catheter instrument 340 and, under control commands, control the catheter instrument 340 to move vertically, horizontally, or in non-vertical and non-horizontal directions, thereby providing a better preoperative preparation angle for the operation of the catheter instrument 340. The control commands can be triggered by the doctor operating the patient surgical platform 320, or by the doctor directly clicking or pressing a button on the patient surgical platform 320. Of course, in other embodiments, the control commands can also be voice control or commands triggered by a force feedback mechanism.

[0063] like Figure 4 As shown, the patient surgical platform 320 may further include a base 321, a sliding base 322 that can be raised and lowered along the base 321, and two robotic arms 323 fixedly connected to the sliding base 322. The robotic arms 323 may include multiple arm segments connected at joints, each of which provides the robotic arms 323 with multiple degrees of freedom, for example, seven degrees of freedom corresponding to the seven arm segments. A manipulator (not shown) is mounted at the end of the robotic arm 323. The manipulator of the robotic arm 323 is used to engage the catheter instrument 340 and, under the action of the manipulator, controls the distal end of the catheter instrument 340 to bend and turn accordingly. The two robotic arms 323 may have identical or partially identical structures, with one robotic arm 323 used to engage the inner catheter instrument 341 and the other robotic arm 323 used to engage the outer catheter instrument 342. During installation, the outer catheter device 342 may be installed first, and after the outer catheter device 342 is installed, the catheter of the inner catheter device 341 may be inserted into the catheter of the outer catheter device 342 .

[0064] The sensor system 350 has one or more subsystems for receiving information about the catheter device 340. The subsystems may include: a position sensor system; a shape sensor system for determining the position, orientation, speed, velocity, pose, and / or shape of the tip of the catheter device 340 and / or one or more segments along a catheter that may comprise the catheter device 340; and / or a visualization system for capturing images from the tip of the catheter device 340.

[0065] The imaging cart 330 may be equipped with a display system 331 and an irrigation system (not shown). The display system 331 is used to display images or representations of the surgical site and catheter instruments 340 generated by the subsystems of the sensor system 350. Real-time images of the surgical site and catheter instruments 340 captured by the visualization system may also be displayed. Pre-operative or intra-operative images of the surgical site may also be presented using image data from imaging technologies such as computed tomography (CT), magnetic resonance imaging (MRI), optical coherence tomography (OCT), and ultrasound.

[0066] The preoperative or intraoperative image data can be presented as a two-dimensional, three-dimensional, or four-dimensional (e.g., time-based or rate-based) image and / or as an image from a model created based on the preoperative or intraoperative image data set. A virtual navigation image can also be displayed in which the actual position of the catheter device 340 is registered with the preoperative image to present a virtual image of the catheter device 340 within the surgical site to the operator from the outside.

[0067] The control system 360 includes at least one memory and at least one processor. It is understood that the control system 360 can be integrated into the patient surgical platform 320 or the imaging cart 330, or can be independently provided. The control system 360 can support wireless communication protocols such as IEEE 802.11, IrDA, Bluetooth, HomeRF, DECT, and wireless telemetry. The control system 360 can transmit one or more signals instructing the catheter instrument 340 to move, which are used by the manipulator to move the catheter instrument 340. The catheter instrument 340 can be extended to the surgical site in the body through the opening of the patient's natural cavity or surgical incision.

[0068] Furthermore, the control system 360 may include a mechanical control system (not shown) and an image processing system (not shown). The mechanical control system is used to control the movement of the catheter instrument 340 and, therefore, may be integrated into the patient surgical platform 320. The image processing system is used for virtual navigation path planning and, therefore, may be integrated into the imaging vehicle 330. Of course, the various subsystems of the control system 360 are not limited to the specific ones listed above and can be reasonably configured according to actual circumstances.

[0069] The image processing system can use the aforementioned imaging techniques to image the surgical site based on preoperative or intraoperative images of the surgical site. Software used in conjunction with manual input can also be used to convert the recorded images into two-dimensional or three-dimensional composite images of a portion or entire anatomical organ or region. During a virtual navigation procedure, the sensor system 350 can be used to calculate the position of the catheter instrument 340 relative to the patient's anatomy. This position can be used to generate external tracking images and internal virtual images of the patient's anatomy, achieving registration of the actual position of the catheter instrument 340 with the preoperative images, thereby allowing a virtual image of the catheter instrument 340 within the surgical site to be presented to the operator from the outside.

[0070] The inner catheter device 341 and the outer catheter device 342 have substantially the same structural composition, and each comprises a slender and flexible inner catheter 41 and an outer catheter 42, wherein the diameter of the outer catheter 42 is slightly larger than that of the inner catheter 41, so that the inner catheter 41 can pass through the outer catheter 42 and provide a certain degree of support for the inner catheter 41, so that the inner catheter 41 can reach the target location in the patient's body, so as to facilitate operations such as tissue or cell sampling from the target location.

[0071] Certain movements of the handle 310 can cause corresponding movements of the catheter instrument 340. For example, when a doctor operates the directional lever of the handle 310 to move upward or downward, the movement of the directional lever of the handle 310 can be mapped to a corresponding pitch movement of the distal end of the catheter instrument 340. When the doctor operates the directional lever of the handle 310 to move left or right, the movement of the directional lever of the handle 310 can be mapped to a corresponding yaw movement of the distal end of the catheter instrument 340. In this embodiment, the handle 310 can control the movement of the distal end of the catheter instrument 340 within a 360-degree spatial range.

[0072] The remote surgical robot system disclosed herein also includes a server. Figure 5 As shown, remote doctor consoles A1-A3 and patient surgical platforms B1-B3 of the same or different master-slave surgical robots are each connected to a server S for communication. Server S manages information sent from these consoles and relays information between them. Server S also facilitates device interconnection (i.e., pairing) between these consoles. Pairing involves connecting two devices to enable communication.

[0073] In ultra-tele-surgery, the remote doctor console and the patient surgery platform are deployed in different locations. For example, they can be deployed in different countries, different provinces, or different jurisdictions in the same city.

[0074] The server may also be deployed at a different location than either the remote physician console or the patient surgical platform. For example, the server, remote physician console, and patient surgical platform may be deployed in different cities. The server may also be deployed at the same location as either the remote physician console or the patient surgical platform. For example, the server may be deployed independently of the remote physician console or the patient surgical platform located at the same location, or the server may be integrated with the remote physician console or the patient surgical platform located at the same location. It is understood that the server may also be deployed in the cloud.

[0075] In the telesurgery robot system disclosed herein, at least one remote doctor console, one patient surgery platform, and one server are deployed respectively.

[0076] There may be at least one remote doctor console, at least one patient operation platform, and at least one server.

[0077] When there is a single server, each remote physician console and each patient surgical platform is connected to the single server. When there are multiple servers, the servers are connected via a network topology, and different remote physician consoles and different patient surgical platforms can be connected to the same or different servers. The network topology between the multiple servers includes at least one of a star topology, a ring topology, a bus topology, a tree topology, a mesh topology, a virtual local area network, and a wireless topology.

[0078] The remote doctor console and the server, the patient surgical platform and the server, and the servers themselves can be connected via the same or different types of high-speed communication networks. For example, these types of high-speed communication networks may include at least one of the following: broadband Internet, a dedicated Internet network, a 5G network, and a dedicated 5G network.

[0079] For example, the remote doctor consoles A1~A3 and the patient operating platforms B1~B3 can realize one-to-one, one-to-many and many-to-one communication under the action of the server S. The paired remote doctor consoles and the patient operating platforms can communicate end-to-end without the need for the server's transfer function. For example, from the perspective of the remote doctor console, these types of pairings can be understood. Figure 6 As shown:

[0080] One-to-one pairing refers to pairing one remote physician console with one patient surgical platform. For example, remote physician consoles A2 and A3 are each paired with only one patient surgical platform, such as A2 paired with B2 or A3 paired with B1. Therefore, for A2 and A3, this is a one-to-one pairing.

[0081] One-to-many pairing refers to pairing a remote physician console with multiple patient surgical platforms. For example, remote physician console A1 is paired with both patient surgical platforms B2 and B3, so for A1, this is a one-to-many pairing.

[0082] Many-to-one pairing refers to pairing multiple remote physician consoles with one patient surgical platform. For example, remote physician consoles A1 and A2 are paired with one patient surgical platform B2, so for A1 and A2, they are a many-to-one pairing.

[0083] <Device Registration>

[0084] Remote doctor consoles and patient surgical platforms come in many different types and models, often with counterfeit and substandard products. Inappropriate pairing of remote doctor consoles and patient surgical platforms can easily lead to surgical risks. To minimize the surgical risks associated with inappropriate pairing between remote doctor consoles and patient surgical platforms, device registration can be used to filter out remote doctor consoles or patient surgical platforms that are not permitted to be paired (i.e., not allowed to be used). In other words, only remote doctor consoles and patient surgical platforms that have been registered with the server are allowed to be paired.

[0085] The method of registering the remote doctor console and the patient surgery platform to the server is the same. To simplify the description, any remote doctor console and any patient surgery platform can be described as a device. In some embodiments, refer to Figure 7 , the method of registering the device to the server includes:

[0086] 101. After the device establishes a connection with the server, the device sends a registration request to the server.

[0087] The process of establishing a connection between a device and a server includes: the device sending a connection request to the server; the server receiving the connection request, attempting to establish a connection with the device, and returning the connection status to the device. If the returned connection status indicates a successful connection, a persistent connection is established between the device and the server; otherwise, a persistent connection is not established between the device and the server. If the returned connection status indicates a failed connection, the connection process can be repeated.

[0088] 102. The server receives the registration request sent by the device and authenticates the registration request based on a judgment rule agreed upon between the server and the device.

[0089] 103. In response to the server determining that the authentication is successful, the server accepts the registration of the device and returns a registration success status to the device.

[0090] The server may maintain registration information of registered devices through a registration list or other means, where the registration information includes device information of these devices.

[0091] 104. In response to the server determining that the authentication fails, the server rejects the registration of the device and returns a registration failure status to the device.

[0092] In some embodiments, in step 102, the agreed judgment rule includes a first judgment rule, which includes a custom protocol agreed upon between the device and the server. A custom protocol includes a protocol header and a protocol body. The protocol header is the message header, and the protocol body is the message data portion. Different custom protocols typically have the same format for the protocol header, but different formats for the protocol body. Differences in the protocol body can be reflected in at least one difference in the number of fields, field type, field order, and data exchange format. Figure 8 It illustrates a custom protocol based on TCP protocol.

[0093] The registration request is a datagram encapsulated according to the format specified by the custom protocol. After receiving the registration request, the server decapsulates the registration request according to the format specified by the custom protocol. If the decapsulation is successful, authentication is considered successful; if the decapsulation fails, authentication is considered unsuccessful.

[0094] In the initial state of a device, for example, before it is registered with a server, each device only stores its own device information and the server's device information. A device registration request includes both the server's device information and the local device information. The server's device information includes at least one of the server's IP address and port number. The local device information includes basic device information and network information. Basic device information includes at least one of the device's unique identifier (ID or UDID), device name, device type, manufacturer name, function, operation type, and device location. Network information includes at least one of the device's IP address and port number.

[0095] The device registration request is encapsulated into a datagram according to the format specified by the custom protocol. For example, if the custom protocol is TCP, the server port number and the device's local port number are encapsulated in the protocol header, and other local device information is encapsulated in the protocol body. The protocol body includes multiple fields, each containing local device information. For example, these fields may include at least one of the following: the device's unique identifier (ID or UDID), device name (name), device type (type), device manufacturer name (manufacture), device function (function), and IP address. The protocol body may also include a timestamp field to store the time when the device's registration request was sent to the server.

[0096] In some embodiments, different devices can communicate with each other through the same or different custom protocols. For example, the remote doctor console and the server can communicate through an agreed first custom protocol; the patient surgery platform and the server can communicate through an agreed second custom protocol; and the remote doctor console and the patient surgery platform can communicate through an agreed third custom protocol. The three custom protocols can be the same or different. In some embodiments, a remote doctor console can be configured to support communication with multiple patient surgery platforms using the same or different custom protocols, and a patient surgery platform can also be configured to support communication with multiple remote doctor consoles using the same or different custom protocols.

[0097] In some embodiments, for example, when a remote physician console or patient surgical platform communicates with a server via an agreed-upon custom protocol, the protocol body of the custom protocol may include at least one field for storing custom protocols supported for pairing by the remote physician console or patient surgical platform. In step 103, when the server accepts device registration, the custom protocols supported for pairing by the corresponding device may also be stored on the server.

[0098] The field device function in the protocol body is explained with examples. For the remote doctor console, the device function includes operable functions; for the patient surgery platform, the device function includes functions that can be operated. The device functions of different remote doctor consoles or different patient surgery platforms may be the same or different. The remote doctor console and the patient surgery platform that are expected to be used in pairing must at least match the device core functions in the device functions before they are allowed to be paired. For example, the device core functions include at least one of the motion control function and the energy control function, both of which are related to the device hardware conditions. Only with the corresponding device hardware conditions can the corresponding core functions be implemented. For example, a patient surgical platform includes a hand robot and an energy platform. The patient surgical platform supports motion control functions based on the hand robot and energy control functions based on the energy platform. A remote physician console includes a hand operation unit and a foot pedal unit. The remote physician console supports motion control functions based on the hand operation unit and energy control functions based on the foot pedal unit. For the patient surgical platform, if the remote physician console cannot support motion control functions or energy control functions due to its device hardware conditions, it may not be allowed to pair with the patient surgical platform. Or, from the perspective of control permissions, at least it may not be allowed to be configured with full operator control permissions for the patient surgical platform. In this case, it may have observer permissions. In some embodiments, the motion control function or energy control function can be further classified, and the device core function corresponds to a more specific core function. The remote physician console and the patient surgical platform to be paired must at least match in the more specific core function before pairing is allowed. Of course, this is also closely related to the device hardware conditions and will not be elaborated in detail here. Among them, the device functions including the device core function under the device function field can be recorded by associated coding, for example, using digital records. Among the device functions, different flag records can be added to the device core functions and the device non-core functions to facilitate the matching of the device core functions.

[0099] In step 103, the server accepts the device's registration and stores the device's registration information. This information includes data stored in at least some fields of the custom protocol body, such as device information and one or more custom protocols supported by the device for pairing. After step 103, the server can assign a login password to the device. This login password is used for simple authentication when the device subsequently logs into the server, eliminating the need for the server to authenticate the device based on agreed-upon rules, such as custom protocols or encryption algorithms.

[0100] In some embodiments, the agreed-upon judgment rule may include a second judgment rule, which includes an encryption algorithm agreed upon between the device and the server. The registration request may be a data message encapsulated according to a format specified by a custom protocol, or a data message encapsulated according to a format specified by a universal protocol. In some embodiments, the encryption algorithm may encrypt at least a portion of the data in the protocol body of the data message to generate a first check value, which is stored in an authentication field of the protocol body. The data in the protocol body may not include the data stored in the authentication field. After receiving the registration request, the server decapsulates the registration request according to the format specified by the custom protocol or universal protocol to obtain all the data in the data message. The server may then encrypt the data in the protocol body based on the agreed-upon encryption algorithm to generate a second check value. The server detects whether the first check value and the second check value are identical. If they are identical, the data is complete and valid, and authentication is considered successful. If they are different, the data is missing or invalid, and authentication is considered unsuccessful.

[0101] Alternatively, when the encryption algorithm is reversible, the server may not generate the second verification value, but instead decrypt the first verification value stored in the authentication field obtained by directly decapsulating the registration request based on the inverse solution of the encryption algorithm to obtain verification data. The server compares the decrypted verification data with at least part of the corresponding data in the protocol body. If they are the same, it indicates that the data is complete and legal, and the authentication can be determined to be successful; if they are different, it indicates that the data is missing or illegal, and the authentication can be determined to be failed.

[0102] Exemplarily, the encryption algorithm can be a base64 encryption algorithm or a hash algorithm. The data used for encryption in the protocol body can be data from all fields in the protocol body, or data from fields selected according to a certain rule. Such a rule can, for example, be fields selected at intervals, or the first few consecutive fields, or the last few consecutive fields. In one example, one or more of the device information of the associated device in the protocol body can be selected for encryption. For example, the unique identifier, device name, and timestamp in the protocol body can be selected as the data of the field to be encrypted. In one example, the data of the selected fields can be concatenated into a string and then subjected to the encryption process.

[0103] In some embodiments, the registration request can be double-authenticated in combination with the first judgment rule and the second judgment rule, which can prevent unauthorized devices from being allowed to access the remote surgical robot system described in the present disclosure, thereby ensuring the safety and reliability of remote surgery.

[0104] In the scenario of double authentication, the registration request is usually a data message encapsulated according to the format specified by the custom protocol. The server decapsulates the registration request based on the format specified by the custom protocol and performs the first authentication. If the decapsulation is successful, it is determined that the first authentication is successful and enters the second authentication. If the decapsulation fails, it is determined that the first authentication has failed and no longer enters the second authentication. The server will reject the device registration. In the second authentication, the server encrypts the data in the protocol body based on the agreed encryption algorithm to generate a second check value. The server detects whether the first check value and the second check value stored in the authentication field in the data message corresponding to the registration request are the same. If they are the same, it can be determined that the second authentication is successful. When both authentications are successful, the server will accept the device registration. If they are different, it can be determined that the second authentication has failed and the server will reject the device registration.

[0105] Alternatively, in the second authentication, as described above, the server may not generate the second verification value, but instead decrypt the first verification value stored in the authentication field obtained by directly decapsulating the registration request based on the inverse solution of the encryption algorithm to obtain verification data. The server compares the decrypted verification data with at least part of the corresponding data in the protocol body. If they are the same, the authentication is determined to be successful; if they are different, the authentication is determined to have failed.

[0106] In some embodiments, the patient surgical platform may include multiple types. When the same remote doctor console manipulates different patient surgical platforms to perform the same or different types of operations, the types of data exchanged between the remote doctor console and the patient surgical platform may be different. The remote doctor console may also include multiple types. When different remote doctor consoles manipulate the same patient surgical platform to perform the same or different types of operations, the types of data exchanged between the remote doctor console and the patient surgical platform may also be different. Among them, the different types of data exchanged between the remote doctor console and the patient surgical platform can be reflected in the different field types in the protocol body of the custom protocol. It can be understood that due to the different field types in the protocol body, the custom protocol including the protocol body is also different.

[0107] In some embodiments, the server can be used as an open platform for a variety of remote doctor consoles and a variety of patient surgery platforms. The server and different devices can agree on different judgment rules for authentication during registration. As long as the device can be successfully authenticated, it can be allowed to register and subsequently paired for ultra-remote surgery. For example, when the judgment rules include custom protocols, different judgment rules correspond to different custom protocols. For another example, when the judgment rules include encryption algorithms, different judgment rules correspond to different encryption algorithms, where using encryption algorithms with the same name to encrypt data corresponding to different field types can be understood as different encryption algorithms.

[0108] Taking the example of the server using a custom protocol to authenticate the registration request sent by a device, when the server receives the registration request sent by any device, it does not know whether the server and the device have agreed on a custom protocol, nor does it know which custom protocol is agreed upon. Therefore, the server usually needs to call all its known custom protocols to try to decapsulate the registration request sent by the current device. If the registration request can be decapsulated based on a certain custom protocol, it is determined that the authentication is successful; if the registration request cannot be decapsulated based on any custom protocol, it is determined that the authentication has failed.

[0109] Exemplarily, after receiving a registration request, the server may sequentially attempt to decapsulate the registration request using all known custom protocols. In some embodiments, it is assumed that the device includes a first device and a second device, the judgment rule agreed upon between the first device and the server is a first custom protocol, and the judgment rule agreed upon between the second device and the server is a second custom protocol, and all known custom protocols include the first custom protocol and the second custom protocol. Upon receiving a registration request, the server may first attempt to decapsulate the currently received registration request using the first custom protocol. If the decapsulation fails, it may then attempt to decapsulate the currently received registration request using the second custom protocol.

[0110] For example, after the server receives a first registration request sent by a first device, the server first decapsulates the first registration request based on the first custom protocol. At this time, the decapsulation is successful, and the authentication is determined to be successful. For another example, after the server receives a second registration request sent by a second device, the server first decapsulates the second registration request based on the first custom protocol. At this time, the decapsulation fails. The server then decapsulates the second registration request based on the second custom protocol. At this time, the decapsulation is successful, and the authentication is determined to be successful. For another example, the server may also receive a third registration request sent by a third device. However, the third device may not have agreed on a custom protocol with the server. In this case, after the server receives the third registration request sent by the third device, the server first decapsulates the third registration request based on the first custom protocol. At this time, the decapsulation fails. The server then decapsulates the third registration request based on the second custom protocol. At this time, the decapsulation still fails, and the authentication is determined to be failed.

[0111] In some embodiments, the core equipment of the patient surgical platform includes a slave robot, an imaging platform, and an energy platform. The slave robot includes a manipulator and medical instruments mounted on the manipulator. The remote physician console controls the movement of the manipulator, thereby controlling the movement of the medical instruments. The imaging platform processes multimedia data, such as video or audio. Video can originate from, for example, imaging devices within medical instruments such as endoscopes; audio can originate from, for example, the remote physician console or input from the slave robot, including voice intercom. The energy platform controls the energy instruments within the medical instruments. Energy instruments can include, for example, electrosurgical scalpels, ultrasonic scalpels, microwave ablation scalpels, radiofrequency ablation scalpels, staplers, plasma scalpels, and the like. The imaging platform and energy platform can be located separately or integrated on a single, movable trolley. For ease of description, the imaging platform and energy platform are collectively referred to as the image energy processing platform. In some embodiments, the patient surgical platform may also include auxiliary equipment, such as monitors, ventilators, CT scanners, electromagnetic positioning systems, pneumoperitoneum machines, operating tables, and shadowless lamps.

[0112] Generally, the type of telesurgery system can depend on the type of slave robot in the master-slave surgical robot. Different slave robots typically have different structural characteristics. Various types of slave robots include, but are not limited to, single-port laparoscopic slave robots, multi-port laparoscopic slave robots, bronchial interventional slave robots, vascular interventional slave robots, and orthopedic slave robots.

[0113] In some embodiments, the type of remote doctor console may depend on the type of operating unit, which is used to control the actions of medical instruments in the hand robot, including motion control, clamping control, cutting control, coagulation control, suturing control, etc. The operating unit includes a hand operating unit, and the types of hand operating units include but are not limited to game controller type operating units, mechanical linkage type operating units, magnetic navigation type operating units, trackball type operating units, etc.

[0114] In some embodiments, remote physician consoles and patient surgical platforms that are not registered with the server or have different communication protocols are generally prohibited from pairing. Pairing requires the same communication protocol, which generally refers to the same custom protocol, mainly including the same format specified by the custom protocol.

[0115] In some embodiments, remote physician consoles and patient surgical platforms that are registered with the server and use the same communication protocol are generally allowed to be paired. Regardless of the type of remote physician console or patient surgical platform, a remote physician console and patient surgical platform can be paired if the basic conditions are met: the remote physician console and patient surgical platform are registered with the server and use the same communication protocol.

[0116] In some embodiments, a custom protocol can be agreed upon between a type of remote physician console and a type of patient surgical platform, so that the type of remote physician console is specifically used to be paired with the type of patient surgical platform, that is, the remote physician console is specifically used to operate a specific type of patient surgical platform.

[0117] In some embodiments, the same custom protocol can be agreed upon between one type of remote physician console and multiple types of patient surgical platforms, or multiple custom protocols can be agreed upon separately, so that this type of remote physician console can also be universally used in pairing with multiple types of patient surgical platforms, that is, the remote physician console can be universally used to manipulate multiple specific types of patient surgical platforms.

[0118] Exemplarily, a remote physician console manufactured by a manufacturer can be configured to operate one or more types of patient surgical platforms manufactured by the manufacturer or another manufacturer. For example, manufacturer A manufactures a universal remote physician console A1 and manufactures a first type of patient surgical platform A2 and a second type of patient surgical platform A3. The first type of patient surgical platform A2 is, for example, a patient surgical platform with a single-port laparoscopic slave robot, and the second type of patient surgical platform A3 is, for example, a patient surgical platform with a multi-port laparoscopic slave robot. The same custom protocol can be agreed upon between the remote physician console A1 and patient surgical platforms A2 and A3, or different custom protocols can be agreed upon respectively. The remote physician console A1 can be configured to operate one or more of patient surgical platforms A2 and A3.

[0119] Exemplarily, remote doctor consoles manufactured by different manufacturers can be configured to be specifically used to operate one or more types of patient surgical platforms manufactured by each manufacturer. For example, manufacturer B manufactures a remote doctor console B1, and manufactures a first type of patient surgical platform B2 and a second type of patient surgical platform B3. The same custom protocol, such as a first custom protocol, can be agreed upon between the remote doctor console B1 and the patient surgical platforms B2 and B3. Manufacturer C manufactures a remote doctor console C1, and manufactures a first type of patient surgical platform C2 and a second type of patient surgical platform C3. The same custom protocol, such as a second custom protocol, which is different from the first custom protocol, can be agreed upon between the remote doctor console C1 and the patient surgical platforms C2 and C3. For example, remote doctor consoles B1 and C1 are both remote doctor consoles with mechanical linkage-type operating units, patient surgical platforms B2 and C2 are both, for example, single-port laparoscopic slave robots, and patient surgical platforms B3 and C3 are both, for example, multi-port laparoscopic slave robots. The remote physician console B1 manufactured by manufacturer B can only be configured to operate one or more of its manufactured patient surgical platforms B2 and B3, and cannot be configured to operate the patient surgical platforms C2 and C3 manufactured by manufacturer C. At the same time, the remote physician console C1 manufactured by manufacturer C can only be configured to operate one or more of its manufactured patient surgical platforms C2 and C3, and cannot be configured to operate the patient surgical platforms B2 and B3 manufactured by manufacturer B.

[0120] Exemplarily, remote doctor consoles manufactured by different manufacturers can be configured to be used to manipulate one or more types of patient surgical platforms manufactured by different manufacturers. Assume that manufacturer B manufactures a remote doctor console B1, and manufactures a patient surgical platform B2 of the first type and a patient surgical platform B3 of the second type; manufacturer C manufactures a remote doctor console C1, and manufactures a patient surgical platform C2 of the first type and a patient surgical platform C3 of the second type; remote doctor consoles B1 and C1 are, for example, remote doctor consoles with mechanical linkage type operating parts, patient surgical platforms B2 and C2 are, for example, single-port laparoscope slave robots, and patient surgical platforms B3 and C3 are, for example, multi-port laparoscope slave robots. For example, a first custom protocol is agreed upon between the remote doctor console B1 and patient surgical platforms B2, B3, C2, and C3, and the first custom protocol is established in the remote A second custom protocol is agreed upon between the doctor console C1 and the patient surgical platforms B2, B3, C2, and C3. The first custom protocol may be the same as or different from the second custom protocol. The remote doctor console B1 manufactured by manufacturer B may be configured to be used to manipulate one or more of the patient surgical platforms B2 and B3 manufactured by it, and at the same time, may also be configured to be used to manipulate one or more of the patient surgical platforms C2 and C3 manufactured by manufacturer C; the remote doctor console C1 manufactured by manufacturer C may be configured to be used to manipulate one or more of the patient surgical platforms C2 and C3 manufactured by it, and at the same time, may also be configured to be used to manipulate one or more of the patient surgical platforms B2 and B3 manufactured by manufacturer B.

[0121] In some embodiments, a custom protocol may be agreed upon between one type of patient surgical platform and one type of remote physician console, or multiple custom protocols may be agreed upon between one type of patient surgical platform and multiple types of remote physician consoles, so that the type of patient surgical platform may also be configured to be operated by one or more remote physician consoles of the same type.

[0122] For example, a patient surgical platform manufactured by a manufacturer can be configured to be operated by one or more types of remote physician consoles manufactured by the manufacturer or other manufacturers. For example, manufacturer A manufactures a first type of patient surgical platform A1, and manufacturer A also manufactures a first type of remote physician console A2 and a second type of remote physician console A3. The first type of remote physician console A2 can be, for example, a remote physician console with an operating unit in the form of electromagnetic navigation, and the second type of remote physician console A3 can be, for example, a remote physician console with an operating unit in the form of a mechanical linkage. The same custom protocol can be agreed upon between the patient surgical platform A1 and the remote physician consoles A2 and A3, or different custom protocols can be agreed upon respectively. The patient surgical platform A1 can be configured to be operated by one or more of the remote physician consoles A2 and A3.

[0123] Exemplarily, patient surgical platforms manufactured by different manufacturers can be configured to be manipulated only by one or more types of patient surgical platforms manufactured by each manufacturer. For example, manufacturer B manufactures a first type of patient surgical platform B1, and also manufactures a first type of remote physician console B2 and a second type of remote physician console B3. A common custom protocol, such as a first custom protocol, can be agreed upon between patient surgical platform B1 and remote physician consoles B2 and B3. Manufacturer C manufactures a first type of patient surgical platform C1, and also manufactures a first type of remote physician console C2 and a second type of remote physician console C3. A common custom protocol, such as a second custom protocol that is different from the first custom protocol, can be agreed upon between patient surgical platform C1 and remote physician consoles C2 and C3. For example, patient surgical platforms B1 and C1 can both be single-port laparoscopic slave robots, remote physician consoles B2 and C2 can both be remote physician consoles with electromagnetic navigation operating units, and remote physician consoles B3 and C3 can both be remote physician consoles with mechanical linkage operating units. Patient surgical platform B1 can be configured to be manipulated only by one or more of remote physician consoles B2 and B3. Meanwhile, the patient surgical platform C1 can only be configured to be manipulated by one or more of the remote physician consoles C2 and C3 .

[0124] For example, patient surgical platforms manufactured by different manufacturers can be configured to be operated by one or more types of remote doctor consoles manufactured by different manufacturers. Assume that manufacturer B manufactures the first type of patient surgical platform B1, and manufacturer B also manufactures the first type of remote doctor console B2 and the second type of remote doctor console B3; manufacturer C manufactures the first type of patient surgical platform C1, and manufacturer B also manufactures the first type of remote doctor console C2 and the second type of remote doctor console C3; patient surgical platforms B1 and C1 can be, for example, single-port laparoscopic slave robots, remote doctor consoles B2 and C2 can be, for example, remote doctor consoles with an operating part in the form of electromagnetic navigation, and remote doctor consoles B3 and C3 can be, for example, remote doctor consoles with an operating part in the form of mechanical linkage. For example, in the patient surgical platform B1 and remote doctor consoles B2, B3, C2, C 3, and a second custom protocol is agreed upon between the patient surgery platform C1 and the remote doctor consoles B2, B3, C2, and C3. The first custom protocol may be the same as or different from the second custom protocol. The patient surgery platform B1 manufactured by manufacturer B can be configured to be capable of being operated by one or more of the remote doctor consoles B2 and B3 manufactured by it, and at the same time, it can also be configured to be capable of being operated by one or more of the remote doctor consoles C2 and C3 manufactured by manufacturer C; the patient surgery platform C1 manufactured by manufacturer C can be configured to be capable of being operated by one or more of the patient surgery platforms C2 and C3 manufactured by it, and at the same time, it can also be configured to be capable of being operated by one or more of the remote doctor consoles B2 and B3 manufactured by manufacturer B.

[0125] Understandably, whether the remote physician console and the patient surgical platform can be paired depends primarily on whether they are registered with the server and use the same communication protocol. Even if the remote physician console and the patient surgical platform are manufactured by different manufacturers, they can generally be paired as long as they are registered with the server and use the same communication protocol. During pairing, at least observer permissions are permitted. This disclosure provides examples of further conditions for pairing below.

[0126] <Equipment Maintenance>

[0127] When remote physician consoles and patient surgical platforms successfully register with the server, the server will store their device information. However, as the number of successfully registered remote physician consoles and patient surgical platforms increases, if the server does not regularly maintain these remote physician consoles and patient surgical platforms, the existence of a large number of inactive remote physician consoles or patient surgical platforms will not only waste server storage space, but also reduce the efficiency of pairing between active remote physician consoles and patient surgical platforms.

[0128] In some embodiments, the server can periodically clear the registration information of inactive remote physician consoles and patient surgical platforms registered on the server. For example, the registration information of these inactive devices can be removed from the device list maintained by the server. When clearing inactive devices, the server can first detect whether the device to be cleared is currently in a paired state. If it is not currently in a paired state, a clearing process can be initiated. If it is currently in a paired state, the device to be cleared can be converted to an active device. Alternatively, the clearing process can be initiated after the pairing state of the device to be cleared is released. This prevents the device to be cleared from being paired and used when the clearing process is initiated, thereby preventing the ultra-teleoperative surgery from being interrupted and affecting the surgical procedure.

[0129] Inactivity includes inactive login and inactive pairing. Inactive login includes that the login frequency is less than the preset login frequency or the login duration is less than the preset login duration within the inactive period set by the server. Inactive pairing includes that the successful pairing frequency is less than the preset pairing frequency or the usage duration after successful pairing is less than the preset pairing usage duration within the inactive period. Among them, from the perspective of the server's timing, the server can be cleared once a week, for example; the inactive period is the time from the successful registration of the device to the start of the server, such as 1 year; the preset login frequency is the number of times the device logs in to the server as recorded by the server, such as 1 time; the preset login duration is the cumulative time the device logs in to the server, such as 1 hour; the preset pairing frequency is the number of times the device is paired with other devices as recorded by the server, such as 1 time; the preset pairing usage duration is the cumulative time from the establishment of pairing with other devices to the unpairing as recorded by the server, such as 1 hour.

[0130] The server effectively saves server resources by regularly clearing the registration information of inactive remote doctor consoles and patient surgery platforms, and can improve the efficiency of pairing between remote doctor consoles and patient surgery platforms. In addition, it also prevents these inactive remote doctor consoles and patient surgery platforms from being illegally used, affecting the safety of ultra-teleoperative surgery.

[0131] <Device Interconnection>

[0132] In some embodiments, the registered remote doctor console and the patient surgical platform can be interconnected, that is, paired. This disclosure discusses three pairing modes, and the pairing initiators of different pairing modes are different. Among them, the first mode is a two-end call mode, which means that both the remote doctor console and the patient surgical platform initiate pairing. The second mode is a single-end call mode, which means that one end of the remote doctor console and the patient surgical platform initiates pairing. The third mode is an administrator direct connection mode, which means that the server initiates pairing.

[0133] In the first and second modes, the pairing initiator must log in to the server before initiating pairing. The pairing initiator can quickly log in to the server using the login password assigned by the server upon registration. The pairing initiator includes at least one of the remote physician console and the patient surgical platform. In the third mode, the pairing initiator is the server. The user must log in to the server with an administrator account and password to obtain administrator privileges. This administrator privilege can be used to configure the judgment rules agreed upon between the server and each device, or to directly pair the device.

[0134] To simplify the description, we can refer to one of the remote doctor console and the patient surgery platform as the first device and the other as the second device. The following describes the pairing methods in each of the three modes.

[0135] <First Mode - Two-way Calling>

[0136] In some embodiments, see Figure 9 , the method comprising:

[0137] 201. The first device sends a first request to the server.

[0138] The first request is used to request the server to send second device information of a second device that may be paired with the first device.

[0139] In one embodiment, the communication protocol, such as a custom protocol, agreed upon between all first devices and all second devices registered with the server is the same. Any first device and any second device can communicate after pairing. In this case, the first request simply requests the server to send second device information that can be paired with the second device.

[0140] In one embodiment, the communication protocols, such as custom protocols, agreed upon between at least some of the first and second devices registered with the server are different. Communication between the first and second devices with different communication protocols is not possible even after pairing. In this case, to improve compatibility and pairing efficiency, the server can match the custom protocols supported by the second devices based on the relevant information carried in the first request, such as the custom protocols supported by the first device for pairing, and select the second devices associated with the custom protocols supported by the first device as pairable second devices, and provide the second device information of the pairable second devices.

[0141] The pairable second device may be a second device that satisfies certain conditions among the second devices. For example, the pairable second device includes an online second device, i.e., a second device that is logged into the server. For example, the pairable second device includes a second device that is in an unpaired state.

[0142] The first request is encapsulated into a data message in a format specified by a custom protocol agreed upon by the first device and the server and sent to the server.

[0143] 202. The server receives the first request and sends second device information that can be paired with the second device to the first device based on the first request.

[0144] After receiving the first request, the server decapsulates the first request based on the custom protocol agreed upon with the first device to obtain a custom protocol supported by the first device for pairing. The custom protocol supported by the first device for pairing is used for communicating with the second device and is different from the custom protocol agreed upon with the server.

[0145] Based on the obtained custom protocol supported by the first device for pairing, the server matches a second device with the same custom protocol supported for pairing from the registered second devices as a pairable second device, and obtains second device information associated with the pairable second device.

[0146] In one embodiment, the second device information includes at least one type of device information for the corresponding second device, and the at least one type of device information uniquely identifies the corresponding second device. For security and simplicity, the second device information may only include a unique identifier for the corresponding second device, without providing the device network information. This unique identifier can be named in conjunction with the device location information, for example, "XX Hospital, Patient No. XX Surgical Platform." This provides more useful information without compromising communication security.

[0147] 203. The first device receives second device information that can be paired with the second device, sent by the server.

[0148] After the first device receives the second device information, it can display the second device information on the display screen of the first device in at least one of a variety of forms such as a list, so that the doctor can choose to pair the second device.

[0149] 204 : The first device obtains target second device information of the target second device determined based on the second device information, and sends a first pairing request to the server.

[0150] The target second device is a second device among the pairable second devices that the physician desires to pair with the first device. The first pairing request is used to request pairing with the target second device. The first pairing request includes device information of the first device and target second device information of the target second device. In one embodiment, both the device information of the first device and the target second device information may include only unique identifiers of the respective devices.

[0151] The first pairing request is also encapsulated into a data message in the format specified by the custom protocol agreed upon by the first device and the server and sent to the server.

[0152] 205. The server receives a first pairing request sent by the first device.

[0153] After receiving the first pairing request, the server may return a signaling indicating receipt of the first pairing request to the first device.

[0154] 206. The target second device sends a second request to the server.

[0155] The second request is used to request the server to send first device information of a pairable first device that may be paired with the target second device.

[0156] The pairable first device may include at least some first devices registered with the server. Similarly, to improve pairing efficiency, the pairable first device may include first devices registered with the server and using the same communication protocol as the second device. To achieve this, the second request may include a custom pairing protocol supported by the second device. The second request is encapsulated into a data packet in the format specified by the custom protocol agreed upon between the target second device and the server and sent to the server.

[0157] 207 : The server receives the second request, and sends first device information that can be paired with the first device to the target second device based on the second request.

[0158] After receiving the second request, the server decapsulates the second request based on the custom protocol agreed upon with the target second device, and obtains the custom protocol supported by the second device for pairing.

[0159] Based on the acquired custom protocol supported for pairing by the second device, the server matches a first device with the same custom protocol supported for pairing from the registered first devices as a pairable first device, and acquires first device information associated with the pairable first device.

[0160] The first device information includes at least one of a plurality of device information of the corresponding first device. For example, it may only include a unique identifier of the corresponding first device without providing device network information of the device.

[0161] 208. The target second device receives the first device information that can be paired with the first device sent by the server.

[0162] The description of 208 can refer to the description of 203 and will not be repeated here.

[0163] 209 : The target second device obtains target first device information of the target first device to be paired with, which is determined based on the first device information, and sends a second pairing request to the server.

[0164] The target first device is the first device among the pairable first devices that the physician desires to pair with the target second device. The second pairing request is used to request pairing with the target first device. The second pairing request includes target second device information of the target second device and target first device information of the target first device. In one embodiment, the first device information includes at least one type of device information of the corresponding first device. The at least one type of device information can uniquely identify the corresponding first device. For example, the first device information may include only a unique identifier of the corresponding device.

[0165] The second pairing request is also encapsulated into a data message in the format specified by the custom protocol agreed upon by the target second device and the server and sent to the server.

[0166] 210. The server receives a second pairing request sent by a target second device.

[0167] After receiving the second pairing request, the server may return a signaling indicating receipt of the second pairing request to the target second device.

[0168] Among them, since the first device and the target second device it requests to pair with are registered with the server, and the target second device and the target first device it requests to pair with are registered with the server, the received pairing request returned by the server to the first device or the target second device can indicate that the pairing request is allowed, and the server can use paired request ok signaling to indicate the return to the device that initiates the pairing request.

[0169] 211. The server determines whether two peer devices to be paired match based on the first pairing request and the second pairing request.

[0170] The two peer devices refer to the first device and the target second device, and the two are peer devices to each other. The server detects whether the first pairing request matches the second pairing request.

[0171] In step 211, the server may determine whether the two peer devices to be paired match based on the information included in the first pairing request and the information included in the second pairing request. For example, the server may determine whether the two peer devices to be paired match based on whether the target first device information matches the first device information of the first device. If the target first device information matches the first device information of the first device, the two peer devices to be paired match; otherwise, the two peer devices to be paired do not match.

[0172] 204 may include determining at least one target second device information associated with the target second device. For example, if the target second device includes multiple target second device information, the first device may send multiple first pairing requests to the server, each first pairing request being used to request pairing with a target second device. 209 may include determining at least one target first device information associated with the target first device. For example, if the target second device includes multiple target first device information, the target second device may send multiple second pairing requests to the server, each second pairing request being used to request pairing with a target first device.

[0173] In 211 , if the server determines that the two peer devices to be paired match, the process proceeds to 212 ; otherwise, the process proceeds to 214 .

[0174] 212. The server sends device network information of the opposite device to at least one of the first device and the target second device.

[0175] The device network information may include at least one of an IP address, a port number, and a custom protocol supported by the device for pairing.

[0176] In 212, the server may send the device network information of the target second device to the first device. Alternatively, the server may send the device network information of the first device to the target second device. Alternatively, the server may send the device network information of the target second device to the first device and send the device network information of the first device to the target second device.

[0177] 213 , at least one of the first device and the target second device receives the device network information of the opposite device sent by the server, and establishes pairing with the opposite device based on the device network information.

[0178] The first device receives the device network information of the target second device and establishes a pairing with the target second device based on the device network information. Alternatively, the target second device receives the device network information of the first device and establishes a pairing with the first device based on the device network information. Alternatively, both are performed simultaneously.

[0179] After pairing of the first device and the target second device is completed, the first device and the target second device may directly communicate based on any custom protocol that both devices support for pairing.

[0180] 214. The server sends a pairing failure notification to the first device and the target second device.

[0181] In the above 204 or 209, the corresponding device can send a corresponding pairing request to the server in response to the triggering of an event. The event, for example, includes a control for sending a pairing request being triggered, or the corresponding target device information is determined to have reached a preset delay time. The preset delay time can be 0 or other time that is not 0.

[0182] In some embodiments, the pairing implemented at 213 is real-time pairing. After real-time pairing, the first device and the target second device can communicate instantly, without being relayed through a server, and can communicate directly end-to-end. In some embodiments, the pairing implemented at 213 is scheduled pairing. The first device and the target second device enter and implement real-time pairing after meeting scheduled conditions. Exemplary scheduled conditions include the arrival of the scheduled pairing time, the physical condition of the remote physician console or the patient's surgical platform meeting the pairing requirements, etc.

[0183] In some embodiments, after 213, after the first device and the target second device are paired, a pairing success status is returned to the server. The server receives the pairing success status and sends a pairing success notification to the first device and the target second device. In some embodiments, after 211, if the server determines that the intended paired devices do not match, the server may send a pairing failure notification to the first device and the target second device. The pairing success or pairing failure notification may be displayed via a UI interface or sound.

[0184] In some embodiments, two devices can send pairing requests to the server simultaneously or in different time periods. When any device sends a pairing request to the server, the server can start a timer to count the time it takes for the other device expected by the device to send a pairing request to the server. If the other device also sends a pairing request to the server within the set waiting time range, and the server determines that the pairing request sent by the device matches the pairing request sent by the other device, it determines that the device and the other device match, and the pairing is considered successful; if the timeout expires or the pairing requests do not match, it is determined that the device and the other device do not match, and the pairing is considered unsuccessful. For example, the set time can be 60s, 120s, 180s, etc. Setting an appropriate waiting time range can improve the user experience and avoid unnecessary time waste during pairing.

[0185] <Second Mode - Single-Ended Call>

[0186] In some embodiments, see Figure 10 , the method comprising:

[0187] 301. The first device sends a first request to the server.

[0188] 302. The server receives the first request and sends second device information that can be paired with the second device to the first device based on the first request.

[0189] 303. The first device receives second device information that can be paired with the second device, sent by the server.

[0190] 304 : The first device obtains target second device information of the target second device determined based on the second device information, and sends a first pairing request to the server.

[0191] 305. The server receives a first pairing request sent by the first device.

[0192] After receiving the first pairing request, the server may return a signaling indicating receipt of the first pairing request to the first device.

[0193] The principles of 301 to 305 in the second mode are the same or similar to those of 201 to 205 in the first mode, and are not repeated here.

[0194] 306. The server sends a first pairing request to the target second device associated with the target second device information.

[0195] 307 : The target second device receives the first pairing request sent by the server, and obtains a response instruction generated in response to the first pairing request.

[0196] The target second device may generate a notification based on the first pairing request and play the notification to facilitate the doctor to respond to the pairing request based on the notification, ie, respond.

[0197] The content of the notification is to inform the target second device whether to accept the pairing initiated by the first device. In some embodiments, the notification can be played in the form of a UI interface or a sound.

[0198] In some embodiments, the user can make a decision on whether the target second device accepts or rejects the pairing of the first device based on the notification. For example, when playing in the form of a UI interface, operable controls of "Accept pairing of the first device" and "Reject pairing of the first device" can be displayed on the display screen of the second target device. When the user operates the operable control of "Accept pairing of the first device", it indicates acceptance of the pairing of the first device, or, when the user operates the operable control of "Reject pairing of the first device", it indicates rejection of the pairing of the first device. For example, when playing in the form of sound, "Whether to accept pairing of the first device" can be played in the audio device of the second target device. The user can use the voice recognition device set by the second target device to recognize the user's audio input of "Accept pairing of the first device" and generate a response instruction to accept the pairing of the first device, or recognize the user's audio input of "Reject pairing of the first device" and generate a response instruction to reject the pairing of the first device.

[0199] In some embodiments, the second target device may also make the decision to accept or reject the pairing of the first device, that is, the decision is made by the device itself. For example, when the target second device receives the first pairing request sent by the server, a timer may be started to count. When the response waiting time range is reached and the target second device does not obtain the user's response to the notification, the second target device generates a response instruction to reject the pairing of the first device by default. The target second device sends the response instruction to the server. After receiving the response instruction, the server sends a notification of pairing failure to the first device. For another example, the target second device may be configured to store a whitelist, which is associated with one or more first devices. The target second device allows the first device associated with the whitelist to pair with it, and the user does not need to respond to the notification. For example, in a whitelist scenario, if the first device is in the whitelist of the target second device, the user can actively respond to the notification within the response waiting time range. The response includes the target second device accepting or rejecting the pairing of the first device. When the preset conditions are met, such as when the response waiting time range is reached and the target second device does not obtain the user's response to the notification, the second target device automatically generates a response instruction to accept the pairing of the first device. The target second device sends the response instruction to the server. After receiving the response instruction, the server sends a notification of successful pairing to the first device.

[0200] 308. The target second device sends a response instruction to the server.

[0201] 309 : In response to receiving the affirmative response instruction, the server sends the device network information of the opposite device to at least one of the first device and the target second device.

[0202] If the response instruction indicates that the target second device accepts pairing with the first device, the response instruction is an affirmative response instruction.

[0203] 310 , at least one of the first device and the target second device receives device network information of the opposite device sent by the server, and establishes pairing with the opposite device based on the device network information.

[0204] For example, the server sends the device network information of the target second device to the first device. The first device receives the device network information and establishes pairing with the target second device based on the device network information.

[0205] 311. The server sends a prompt indicating successful pairing to the first device and the target second device.

[0206] 312. The first device receives the successful pairing prompt sent by the server and plays the successful pairing prompt; the target second device receives the successful pairing prompt sent by the server and plays the successful pairing prompt.

[0207] 313. In response to receiving the negative response instruction, the server sends a pairing failure prompt to the first device.

[0208] If the response instruction indicates that the target second device rejects pairing with the first device, the response instruction is a negative response instruction.

[0209] In addition, the server may also simultaneously send a pairing failure prompt to the target second device.

[0210] 314. The first device receives the pairing failure prompt sent by the server and plays the pairing failure prompt.

[0211] In some embodiments, the first device is a remote physician console, and the target second device is a patient surgical platform, suitable for use when a physician proactively matches a patient. In some embodiments, the first device is a patient surgical platform, and the target second device is a remote physician console, suitable for use when a patient proactively matches a physician. These two applicable scenarios maximize the utilization of high-quality medical resources.

[0212] In the above embodiments, the request sent by the local device to the server for requesting a pairing with the opposite device, such as the first request sent by the first device to the server, or the second request sent by the second device to the server, may include or not include a screening condition.

[0213] When these requests do not include any filtering conditions, in order to facilitate pairing of the local device with the target peer device among the pairable peer devices, the server can obtain various status information of the peer device and send it to the local device for prompting, such as display. This status information may include, for example, at least one of the peer device's custom protocol information that supports pairing, online status information, device type information, device function information, applicable surgery type information, network status information, device location information, power-on self-test status information, pairing status information, scheduled pairing time information, and control authority allocation status information. The doctor can select the target peer device to be paired based on this status information. However, too much status information is not conducive to quickly and accurately performing the desired pairing. Therefore, in order to further improve pairing efficiency, these requests can be configured to include filtering conditions so that the server can accurately provide pairable peer devices based on the filtering conditions, so that the doctor can select the target peer device to be paired.

[0214] In some embodiments, filtering conditions can be configured based on the above-obtained status information of the local device or the opposite device. Some filtering conditions are associated with the status information of both the local device and the opposite device, while some filtering conditions are associated only with the status information of the opposite device. The filtering conditions can include at least one of the following:

[0215] For example, the filtering criteria include the custom pairing protocol supported by the local device. The server can obtain this filtering criteria from the local device's device information and, based on this filtering criteria, match the peer devices registered with the server with those that match the custom pairing protocol supported by the local device as pairable peer devices. By setting this filtering criteria, a list of truly pairable and usable peer devices can be obtained.

[0216] For example, the screening condition includes the device type of the local device. The registration information of the local device or the opposite device stored in the server may include the device type of the device. For the opposite device being a patient surgical platform, the device type may be reflected in the type of slave robot, including single-port laparoscopic slave robot, multi-port laparoscopic slave robot, bronchial interventional slave robot, vascular interventional slave robot, or orthopedic slave robot, etc.; for the opposite device being a remote doctor console, the device type may be reflected in the type of operating part, including a game controller-type operating part, a mechanical linkage-type operating part, a magnetic navigation-type operating part, or a trackball-type operating part, etc. Different types of remote doctor consoles may be suitable for different types of patient surgical platforms. The server may obtain the screening condition from the device information of the local device, and based on the screening condition, match the opposite devices registered to the server with the opposite devices that match the device type of the local device as the pairable opposite device for registration.

[0217] For example, the filtering criteria include the device functions supported by the local device. The registration information of the local device or the peer device stored on the server may include the device functions supported by the device, including core device functions. Pairing is permitted only when the remote physician console and the patient surgical platform match in core device functions. Exemplarily, core device functions include motion control functions or energy control functions. This is particularly important when the control permissions to be configured are (full) operator permissions. If the control permissions to be configured are only observer permissions, regardless of whether the core device functions match, as long as both devices to be paired include observation functions, a match is possible and pairing is permitted. The server can obtain the filtering criteria from the device information of the local device and, based on the filtering criteria, match the peer devices registered with the server with those that match the device functions supported by the local device as pairable peer devices. By setting these filtering criteria, a list of truly pairable and usable peer devices can be obtained.

[0218] For example, the filtering criteria include the types of surgeries supported by the local device. The registration information of the local or peer device stored on the server can include the types of surgeries that the device is suitable for. Different slave robots may be suitable for the same or different types of surgeries. For example, for thoracic or abdominal surgeries, a (single-port or multi-port) laparoscopic slave robot may be suitable; for bronchial, vascular, or urological surgeries, an (bronchial or vascular) interventional slave robot may be suitable; and for orthopedic surgeries, an orthopedic slave robot may be suitable. Different remote physician consoles may be suitable for the same or different types of surgeries. The surgical type can also include more specific surgical procedures, such as urological surgery, lung surgery, liver surgery, kidney surgery, and knee replacement surgery. For urological surgery, the slave robot can be matched with a (vascular) interventional slave robot; for lung surgery, a (bronchial) interventional slave robot; for liver or kidney surgery, a (single-port or multi-port) laparoscopic slave robot; and for knee replacement surgery, an orthopedic slave robot. The server can obtain the filtering condition from the device information of the local device and, based on the filtering condition, match the peer devices registered with the server with the surgery type that matches the local device as the pairing peer devices. By setting the filtering condition, the server can obtain the pairing peer devices that can be used.

[0219] For example, the screening conditions include the scheduled pairing time of the local device. The scheduled pairing time of the local device can be input by the doctor and obtained by the server. The doctor can input the operation planning time of the peer device and obtain it by the server. The operation planning time includes an occupied time period and an idle time period. Based on the scheduled pairing time of the local device, the server can match the paired peer devices whose scheduled pairing time does not conflict with the idle time period from the operation planning time associated with the peer device obtained by the server. Among them, the peer device that is in an unpaired state and has not been configured with an operation planning time can be considered to have an idle time period as its operation planning time.

[0220] For example, the filtering criteria include the desired online status of the peer device. Upon detecting that the peer device is currently logged in, the server can mark the peer device as online; upon detecting that the peer device is not currently logged in, the server can mark the peer device as offline. Doctors can configure the desired online status as a filtering criterion, allowing the server to match paired peer devices that meet this online status, facilitating rapid acquisition of peer devices suitable for real-time pairing.

[0221] For example, the filtering conditions include the desired network status of the peer device. The server can obtain the network status between the peer device and the server. The network status can include the network type and network communication status, and the network communication status can be normal or abnormal. Doctors can configure the desired network status as a filtering condition, so that the server can match the peer device that meets the network status.

[0222] For example, the screening conditions include the expected pairing status of the peer device. The server can obtain and maintain the pairing status of the local device or the peer device, and the pairing status includes the paired state and the unpaired state. To ensure privacy and security, the pairable peer device usually comes from the peer device in the unpaired state. In the remote teaching scenario, multiple local devices can be paired with one peer device. It is possible that the peer device is in the paired state. In this scenario, the pairable peer device is also allowed to come from the peer device in the paired state. The doctor can configure the expected pairing status as a screening condition for the server to match the appropriate pairable peer device.

[0223] For example, the screening conditions include the device positioning expected by the peer device. The registration information of the local device or the peer device stored in the server may include the device positioning of the device. The device positioning can be obtained through a positioning module configured for the device, such as a GPS positioning module, a Beidou positioning module, a Starlink positioning module, etc., or can be obtained through IP address resolution, or can be written manually. A variety of positioning methods can be used for mutual verification. If the verification is successful, it is used for screening. If the verification fails, the user can be prompted to manually modify the device positioning for screening. Device positioning can include multiple levels, such as continental, national, provincial, municipal, district, street, or building levels. Doctors can configure the expected device positioning as a screening condition for the server to match a pairable peer device that matches the device positioning.

[0224] For example, the filtering conditions include the desired control authority allocation status of the peer device. The server can obtain the control authority allocation status configured on the peer device, which includes the state where all control authorities are allocated, the state where some control authorities are allocated, and the state where control authorities are not allocated. The state where all control authorities are allocated is more suitable for pairing in substitute teaching scenarios. The state where some control authorities are allocated is more suitable for pairing in collaborative surgery scenarios. The state where control authorities are not allocated is suitable for pairing in any scenario. Doctors can configure the desired control authority as a filtering condition so that the server can match a suitable pairable peer device.

[0225] For example, the filtering criteria include the desired power-on self-test (PT) status of the peer device. When communicating with a local or peer device, the server can obtain the device's PT status. This PT status can be classified into multiple levels, such as whether all hardware and software functions are functioning properly or whether all core hardware and software functions are functioning properly. Typically, the paired peer device is expected to have all hardware and software functions functioning properly. However, if the desired paired peer device does not meet the PT requirement for full hardware and software functioning properly, to ensure ultra-teleoperative surgery can be performed, the peer device can be paired as long as its core hardware and software functions are functioning properly. The PT status can be associated with the patient's condition. For example, if the peer device is a surgical platform, its hardware and software functions include core devices and auxiliary devices. Core devices include, for example, a hand robot and an energy platform, while auxiliary devices include, for example, a monitor or ventilator. If the patient's condition is not serious and does not require auxiliary devices, even if the auxiliary device's hardware and software functions abnormally, as long as the core device's hardware and software functions properly, the device can be paired. This meets the minimum pairing requirements and increases the pairing success rate. The server can determine whether auxiliary devices are needed based on patient condition information input from the local or peer device. This information can include definitions of the core and auxiliary devices to be used. Doctors can configure their desired power-on self-test status, allowing the server to match appropriate peer devices.

[0226] Among them, the above-mentioned screening conditions can be used individually or in combination, and the opposite-end device that meets one or more of the above-mentioned screening conditions can be used as a pairable opposite-end device.

[0227] <Third Mode - Administrator Direct Connection>

[0228] In some embodiments, the server can retrieve all registered first and second devices and perform desired pairing based on the retrieved first and second devices. This allows for batch configuration of devices across multiple locations, improving pairing efficiency. However, in practice, these first and second devices may have different communication protocols. Because these protocols are not visible, pairing initiated by doctors is often tentative, resulting in low pairing efficiency.

[0229] In some embodiments, the server can configure different labels for different custom protocols supported for pairing by the corresponding devices. When the device information of the corresponding device is displayed on the display screen of the device or the server, the label can be displayed together to facilitate the doctor to identify devices that can be paired with each other. The custom protocols supported for pairing by each device may include different labels that can be configured for the custom protocols of the device. For example, a first device and a second device with the same label can be paired with each other, while a first device and a second device with different labels cannot be paired with each other. Different labels can be represented by different digital codes, characters, text, colors, or symbols.

[0230] In some embodiments, the doctor can use the server itself for direct administrator connection; or, the doctor can use any device that can log in to the server with administrator privileges, including a remote doctor console, a patient surgery platform, a desktop computer or a portable mobile terminal for direct administrator connection, and the portable mobile terminal includes a mobile phone, a tablet computer or a laptop computer. In some embodiments, any of these devices that can be directly connected by the administrator can be devices registered to the server, and the registration of these devices can be authenticated through a custom protocol or encryption algorithm as described above to improve security. In some embodiments, the mobile terminal device can be bound to the remote doctor console or the patient surgery platform, and participate in the pairing configuration of the above-mentioned multiple modes as a remote doctor console or a patient surgery platform.

[0231] In some embodiments, in order to facilitate the use of a server for administrator direct connection, the present disclosure provides a method, see Figure 11 , the method comprising:

[0232] 401. Obtain first device information of a first device registered with a server.

[0233] 402 : Acquire target first device information associated with the target first device determined based on the first device information.

[0234] The target first device is used to pair with the second device.

[0235] 403 : Based on the target first device information, second device information of a pairable second device that can be paired with the target first device is matched from second devices registered with the server.

[0236] As mentioned above, the custom pairing protocol supported by the pairable second device that can be paired with the target first device is generally compatible with the custom pairing protocol supported by the target first device.

[0237] 404 : Acquire target second device information associated with the target second device determined based on the second device information.

[0238] 405 : Obtain device network information of at least one of the target first device and the target second device.

[0239] 406 , sending the device network information to a peer device associated with the network information.

[0240] 407 , the opposite device receives the network information and establishes a pairing with the local device associated with the network information based on the network information to achieve direct communication.

[0241] For example, in step 406 , when the device network information includes the device network information of the target first device, the device network information is sent to the target second device. Furthermore, in step 407 , the target second device establishes a pairing with the target first device based on the device network information of the target first device.

[0242] For example, in step 406 , when the device network information includes the device network information of the target second device, the device network information is sent to the target first device. Then, in step 407 , the target first device establishes pairing with the target second device based on the device network information of the target second device.

[0243] For example, in step 406, if the device network information includes the device network information of the first target device and the device network information of the second target device, the device network information of the first target device is sent to the second target device, and the device network information of the second target device is sent to the first target device. Furthermore, in step 407, the second target device establishes a pairing with the first target device based on the device network information of the first target device, and the first target device establishes a pairing with the second target device based on the device network information of the second target device.

[0244] The above steps 401 to 406 can help to quickly perform pairing and improve pairing efficiency.

[0245] After 406, after the target first device and the target second device are paired, at least one of the target first device and the target second device returns a pairing success status to the server. The server receives the pairing success status and updates the pairing status it maintains. The server may send a pairing success notification to the target first device and the target second device.

[0246] In some embodiments, the target first device may include one, the target second device may include at least one, and the pairable second device may include a second device that supports a custom pairing protocol compatible with the custom pairing protocol supported by the target first device.

[0247] For example, assuming that the target first device supports a custom pairing protocol A, the second device can be used as a pairable second device if the custom pairing protocol supported by the second device includes A.

[0248] In some embodiments, the target first device includes at least one, the target second device includes one, and the pairable second device includes a second device that supports a custom pairing protocol and is compatible with the custom pairing protocol supported by each target first device.

[0249] For example, assuming that a target first device supports pairing with a custom protocol A and another target first device supports pairing with a custom protocol B, the second device can be used as a pairable second device if the custom pairing protocols supported by the second device are compatible with A and B.

[0250] For example, when the custom protocol supported for pairing in the second device is compatible with at least one custom protocol supported for pairing in each target first device, the second device may serve as the pairable second device.

[0251] For example, assuming that a target first device supports pairing custom protocols A and B, and another target first device supports pairing custom protocols C and D, the second device can serve as a pairable second device if the custom protocol supported for pairing is at least compatible with at least one of A and B, and compatible with at least one of C and D. For example, a second device that supports pairing custom protocols including A and C, or including A and D, or including B and C, or including B and D can serve as a pairable second device. Of course, a second device that includes A, B and C, or including A, C and D, or including B, C and D, or including A, B, C and D is also within the aforementioned scope and can serve as a pairable second device.

[0252] For example, assuming that a target first device supports pairing custom protocols A and B, and another target first device supports pairing custom protocols B and C, the second device can serve as a pairable second device if the custom protocol supported for pairing is compatible with at least one of A and B, and compatible with at least one of B and C. For example, a second device that supports pairing custom protocols including B, or including A and B, or including B and C, or including A and C can serve as a pairable second device. Of course, a second device that includes A, B and C is also within the aforementioned scope and can serve as a pairable second device.

[0253] In some embodiments, when determining the first device for pairing and the second device for pairing with the first device, filtering conditions may be configured to obtain a suitable device. For descriptions of these filtering conditions, reference may be made to the foregoing text and will not be repeated here.

[0254] Based on the pairing methods described in the first to third modes, one-to-one, one-to-many, or many-to-one pairings can be quickly achieved between any remote doctor console and the patient's surgical platform. This simplifies user operations, saves labor and time, reduces the complexity of remote surgery, improves the universality of remote surgery, and contributes to the establishment of an ultra-remote surgery medical network.

[0255] In some embodiments, in the various pairing modes described above, the server may store pairing information for the first and second devices to be paired, and assign unique pairing identifiers to both devices. The first and second devices may attempt to establish pairing and communicate based on the pairing identifier, but communication with each other is prohibited until pairing is successful. Each subsequent communication signaling (i.e., data) may include the pairing identifier for communication authentication. For devices with mismatched pairing identifiers, the recipient has the right to deny communication service.

[0256] In some embodiments, the server can obtain the pairing status of all devices in real time, and can maintain a pairing table based on the pairing status, which records the status of the device as paired, pairing or unpaired. The server can set the status of the device that has completed real-time pairing to paired. The server can set the status of the device that has scheduled pairing or the device that is establishing real-time pairing to pairing. The server can also delete the pairing information of the device that failed to pair or the device that was unpaired, and set the status of the device that failed to pair or the device that was unpaired to unpaired. Among them, as the pairing table is updated, the second device information of the second device that can be paired, or the first device information of the first device that can be paired with each other and the second device information of the second device can be updated and sent to the relevant devices, for example, the former is updated and sent to the first device, and the latter is updated and sent to the server to facilitate pairing.

[0257] In some embodiments, the first device, the second device, or the server may include a display screen and an input device, which can be used to quickly achieve visual pairing of the first and second devices. The display screen and input device can be independent components, and the input device can include, for example, a mouse and a keyboard. On the remote physician console side, its operating unit can also be used as a mouse during the pairing phase. The display screen and input device can also be integrated components, such as a touch screen.

[0258] <UI interaction for two-way calls>

[0259] In some embodiments, the present disclosure provides a method for pairing, see Figure 12 , the method comprising:

[0260] 501. The first device displays a first configuration interface on its first display screen, and in response to obtaining a trigger of a first control in the first configuration interface, sends a first request to a server.

[0261] 502. The server receives a first request sent by a first device, and sends second device information that can be paired with a second device to the first device based on the first request.

[0262] 503. The first device receives the second device information that can be paired with the second device sent by the server, and displays the second device information in the first configuration interface.

[0263] 504 : The first device obtains target second device information of a target second device to be paired with determined based on the second device information, and sends a first pairing request to the server.

[0264] After determining the target second device information, the first device can automatically trigger the issuance of a first pairing request to the server. After determining the target second device information, the first device can issue the first pairing request to the server in response to an event. The event includes the first device reaching a delay time from the time the target second device information is determined, or a control in the first configuration interface being triggered, the control being used to control the sending of the pairing request. Upon detecting that the delay time has been reached or that the control has been triggered, the first device issues the first pairing request to the server.

[0265] 505. The server receives a first pairing request sent by the first device.

[0266] After receiving the first pairing request, the server may return a signaling indicating receipt of the first pairing request to the first device.

[0267] 506 , the target second device displays the second configuration interface on its second display screen, and in response to obtaining the triggering of the first control in the second configuration interface, sends a second request to the server.

[0268] 507 : The server receives the second request sent by the target second device, and sends the first device information that can be paired with the first device to the target second device based on the second request.

[0269] 508. The target second device receives the first device information that can be paired with the first device sent by the server, and displays the first device information in the second configuration interface.

[0270] 509 : The target second device obtains target first device information determined based on the first device information, and sends a second pairing request to the server.

[0271] After determining the target first device information, the target second device may automatically trigger the issuance of a second pairing request to the server. After determining the target first device information, the target second device may issue the second pairing request to the server in response to an event. The event includes the target second device reaching a delay time from the time the target first device determines the target first device information, or the triggering of a control in the second configuration interface, which is used to control the sending of the pairing request. Upon detecting that the delay time has been reached or that the control has been triggered, the target second device issues the second pairing request to the server.

[0272] 510. The server receives a second pairing request sent by a target second device.

[0273] After receiving the second pairing request, the server may return a signaling indicating receipt of the second pairing request to the target second device.

[0274] 511. The server determines whether two peer devices to be paired match based on the first pairing request and the second pairing request.

[0275] If the server determines that the two peer devices to be paired match, the process proceeds to 512 ; otherwise, the process proceeds to 515 .

[0276] 512. The server sends the device network information of the opposite device to at least one of the first device and the target second device.

[0277] 513 , at least one of the first device and the target second device receives the device network information of the opposite device sent by the server, and establishes pairing with the opposite device based on the device network information.

[0278] 514 , after the first device and the target second device are paired, a message indicating that the pairing is successful is displayed on the first configuration interface and the second configuration interface respectively.

[0279] 515. The server sends a notification of pairing failure to the first device and the target second device.

[0280] 516 , the first device and the target second device receive the notification of pairing establishment failure sent by the server, and display a message of pairing establishment failure on the first configuration interface and the second configuration interface respectively.

[0281] Exemplarily, the message of successful or failed pairing establishment may be displayed by popping up a dialog box on each configuration interface, or by a fixed or floating status bar on each configuration interface.

[0282] In some embodiments, combined Figures 13 to 15 Refer, in 501 or 506, the first control 1 can be a button. When button 1 is clicked, it means that the first control 1 is triggered, and the corresponding device sends a corresponding request to the server to request the device information of the paired peer device. In 503 or 508, the corresponding device information sent by the server can be displayed on the corresponding configuration interface through the second control 2. The second control 2 can be a list control. The list control 2 includes at least one list option. One list option is associated with the device information of one device. In 504 or 509, when detecting that one or more list options are selected, the corresponding device can determine that the device information associated with the one or more list options is the target device information, that is, determine the target peer device that the corresponding device expects to pair with. For example, take the configuration interface of a remote doctor console, such as the remote doctor console of Hospital A, as an example. Figure 13 , the first control 1 is triggered, and the remote doctor console of Hospital A sends a request to request that the peer device be paired; Figure 14At this time, the server matches the patient surgery platforms of hospitals A~E and displays them in the configuration interface in list control 2; Figure 15 When the list option associated with Hospital B's patient surgery platform is selected and confirmed, Hospital B's patient surgery platform is the desired peer device for Hospital A's remote physician console, and a pairing request is issued. Similarly, if Hospital A's remote physician console is identified as the desired peer device in the corresponding configuration interface of Hospital B's patient surgery platform and another pairing request is issued, the two devices match and pairing can be established; otherwise, the two devices do not match and pairing cannot be established.

[0283] In this embodiment, when the corresponding device detects that the pairing has failed, it can automatically restore the list options to their initial state, such as an unselected state, to facilitate subsequent pairing of the corresponding device with other peer devices. The corresponding device can also cancel pairing with the peer device when the list options are restored to their initial state, such as an unselected state.

[0284] In some embodiments, combined Figures 16 to 19 Refer, in 501 or 506, the first control 1' can be a window control, and the corresponding device can generate a second control 2' in the respective configuration interface according to its device information. The second control 2' can be a drag control, which has an initial position in the corresponding configuration interface. When the second control 2' is moved into the window of the window control 1', it means that the first control 1' is triggered, and the corresponding device sends a corresponding request to the server to request the device information of the pairable peer device. In 503 or 508, the corresponding device information can be displayed in the corresponding configuration interface through the third control 3'. The third control 3' can be a drag control, which has an initial position in the corresponding configuration interface. The number of third controls 3' is the same as the number of pairable peer devices, and each third control 3' corresponds to a peer device. In 504 or 509, the corresponding device can determine that the device information associated with the third control 3' is the target device information when detecting that the third control 3' is moved into the window of the window control 1', that is, determine the target peer device that the corresponding device expects to pair with. For example, take the configuration interface of a remote doctor console, such as the remote doctor console of Hospital A, as an example. Figure 16 and Figure 17 When the second control 2' representing the remote doctor console of Hospital A moves into the window of the window control 1', the remote doctor console of Hospital A sends a request to request that the peer device be paired; Figure 18 At this time, the server matches the patient operation platform of hospital A~E and displays it in the configuration interface by dragging the control 3'; Figure 19When drag control 3' associated with Hospital B's patient surgery platform is moved into the window of window control 1' and confirmed, Hospital B's patient surgery platform is the desired peer device for Hospital A's remote physician console, and a pairing request is issued. Similarly, if Hospital B's patient surgery platform's corresponding configuration interface determines that Hospital A's remote physician console is the desired peer device for pairing and issues another pairing request, the two devices match and pairing can be established; otherwise, the two do not match and pairing cannot be established.

[0285] In this embodiment, see Figures 20 to 22 The window of the window control 1' may include a first area 11' and a second area 12'. The first area 11' includes a first sub-area 11a' and a second sub-area 11b', which are respectively used to accommodate the second control 2', and the second area 12' is used to accommodate the third control 3', wherein these areas can also be implemented using windows. In the case where one or more remote doctor consoles can be paired with a patient surgical platform, the second control 2' is associated with the remote doctor console, and the third control 3' is associated with the patient surgical platform. The first sub-area 11a' and the second sub-area 11b' can be associated with different control permissions, or can also be associated with different pairing orders. Exemplarily, in response to the third control 3' moving to the second area 12', the patient surgical platform associated with the third control 3' sends a first request to the server, which can match a remote doctor console that can be paired with the patient surgical platform and display it in the configuration interface of the patient surgical platform. The pairable remote doctor console can be displayed outside the window in the form of a drag control.

[0286] For example, the first sub-area 11a' can be associated with the remote doctor console accommodated therein for the first control authority over the patient surgical platform accommodated in the second area 12', and the second sub-area 11b' can be associated with the remote doctor console accommodated therein for the second control authority over the patient surgical platform accommodated in the second area 12', and the first control authority is different from the second control authority. Exemplarily, the first control authority is full operator authority, and the second control authority is observer authority. When the second control 2' associated with the first remote doctor console is moved to the first sub-area 11a', the first remote doctor console is configured to have full operator authority over the patient surgical platform accommodated in the second area 12'; when the second control 2' associated with the second remote doctor console is moved to the second sub-area 11b', the second remote doctor console is configured to have observer authority over the patient surgical platform accommodated in the second area 12'. As Figure 22For the same patient surgical platform, such as the one at Hospital A, the remote physician console at Hospital A has different control permissions or pairing order than the remote physician console at Hospital B. In this example, the second sub-area 11b' can include one or more remote physician consoles suitable for quickly pairing a patient surgical platform with a full operator permission and multiple observer permissions. This allows for rapid allocation of control permissions during pairing.

[0287] For example, a first sub-area may be associated with a first pairing order between a remote physician console housed therein and a patient surgical platform housed in a second area, while a second sub-area may be associated with a second pairing order between a remote physician console housed therein and a patient surgical platform housed in the second area, with the first pairing order and the second pairing order being different. Exemplarily, the first pairing order takes precedence over the second pairing order. When a second control associated with the first remote physician console is moved to the first sub-area, the first remote physician console is configured to have the first pairing order with the patient surgical platform housed in the second area; when a second control associated with the second remote physician console is moved to the second sub-area, the second remote physician console is configured to have the second pairing order with the patient surgical platform housed in the second area. In this example, the first area can include more sub-areas, with different sub-areas having different pairing orders. In this way, the pairing order can be quickly set during pairing. In this example, the first area can also be used to accommodate different patient surgical platforms, while the second area is used to accommodate remote physician consoles. Similarly, the pairing order between different patient surgical platforms and the same remote physician console can be quickly set. For example, in response to the drag control associated with the patient surgical platform being moved to the second area, the patient surgical platform sends a first request to the server, which can match a pairable remote doctor console that can be paired with the patient surgical platform and display it in the configuration interface of the patient surgical platform. The pairable remote doctor console can be displayed outside the window in the form of a drag control.

[0288] Whether the second or third control has been moved into the first control can be determined by detecting whether the pixel coordinates of the boundary of the second or third control fall within the pixel coordinate range of the boundary of the first control. Whether the second or third control has been moved into the corresponding area can also be determined by detecting whether the pixel coordinates of the boundary of the second or third control fall within the pixel coordinate range of the boundary of the corresponding area.

[0289] In this embodiment, the drag control may have the same or different sizes before and after being moved. For example, the drag control may have a larger or smaller size when it is moved into the first control relative to before being moved.

[0290] In this embodiment, when the corresponding device detects a pairing failure, it can automatically remove at least the third of the second and third controls from the first control and restore it to its pre-movement position, facilitating subsequent pairing of the corresponding device with other peer devices. The corresponding device can also unpair with the peer device upon detecting that the second and third controls have been removed from the first control. When the second or third control is manually removed, it can automatically return to its pre-movement position once it is removed from the first control.

[0291] In some embodiments, in step 501 or 506, the first control may be a button. When the button is clicked, the first control is triggered, and the corresponding device sends a corresponding request to the server for device information of a pairable peer device. The corresponding device may generate a second control on its respective configuration interface based on its device information. In step 503 or 508, the corresponding device information sent by the server may be displayed on the corresponding configuration interface via a third control. The number of third controls is the same as the number of pairable peer devices, and each third control corresponds to a peer device. The second and third controls are fixed in position on the configuration interface. In 504 or 509, when the display screen of the corresponding device is a touch screen, when the corresponding device detects that the contact of the finger on the touch screen starts to move from one of the second control and the third control, and maintains the contact and continues to move to the other of the second control and the third control, the corresponding device determines that the device information associated with the third control corresponding to the start or end of maintaining the contact and continuing to move is the target device information, that is, determines the target peer device with which the corresponding device expects to be paired; or, when the display screen of the corresponding device is a touch screen or a non-touch screen, when the corresponding device detects that the cursor starts to move from one of the second control and the third control on the display screen, and continues to move to the other of the second control and the third control, the corresponding device determines that the device information associated with the third control corresponding to the start or end of the continuous movement is the target device information, that is, determines the target peer device with which the corresponding device expects to be paired. In 504 or 509, when the display screen of the corresponding device is a touch screen, the corresponding device may also determine that the device information associated with the third control included in the closed track is the target device information when it detects that the finger maintains contact on the touch screen and continuously moves around the third control to form a closed track such as a circle, square, triangle, etc., that is, determines the target peer device with which the corresponding device is expected to be paired; or, when the display screen of the corresponding device is a touch screen or a non-touch screen, the corresponding device may also determine that the device information associated with the third control included in the closed track is the target device information when it detects that the cursor continuously moves around the third control on the display screen to form a closed track, that is, determines the target peer device with which the corresponding device is expected to be paired. In 504 or 509, the corresponding device may also determine that the peer device is the target peer device when it detects that the number of clicks or the dwell time of the finger or cursor on the space representing the peer device reaches a threshold.

[0292] In this embodiment, the movement track of the finger or cursor on the corresponding display screen can be displayed on the corresponding configuration interface to identify the pairing relationship.

[0293] In some embodiments, each configuration interface may include a control for configuring the aforementioned filtering conditions to accurately match the pairable peer devices. The control may include a text box, a list, or a check box. For example, a text box may be used to enter filtering conditions. A list may be used to select filtering conditions. A check box may be used to check filtering conditions. Before executing 501 or 506, the doctor may first configure the filtering conditions. In 501 or 506, the corresponding device may encapsulate the configured filtering conditions into a corresponding request and send it to the server, so that the server can filter out matching peer devices based on the corresponding request.

[0294] UI Interaction for Single-Ended Calls

[0295] In some embodiments, the present disclosure provides a method for pairing, see Figure 23 , the method comprising:

[0296] 531. The first device displays a first configuration interface on its first display screen, and in response to obtaining a trigger of a first control in the first configuration interface, sends a first request to the server.

[0297] 532. The server receives the first request sent by the first device, and sends second device information that can be paired with the second device to the first device based on the first request.

[0298] 533. The first device receives the second device information that can be paired with the second device sent by the server, and displays the second device information in the first configuration interface.

[0299] 534 , the first device obtains target second device information of a second target device to be paired with determined based on the second device information, and sends a first pairing request to the server.

[0300] 535. The server receives a first pairing request sent by the first device.

[0301] After receiving the first pairing request, the server may return a signaling indicating receipt of the first pairing request to the first device.

[0302] 536. The server sends a first pairing request to the target second device associated with the target second device information.

[0303] 537 , the target second device receives the first pairing request sent by the server, displays a second configuration interface on a second display screen of the target second device, and generates a first control and a second control in the second configuration interface based on the first pairing request.

[0304] The first control may be, for example, a pop-up dialog box for notifying the doctor whether to accept the pairing of the first device. The second control may be, for example, a button including a first button and a second button, wherein the first button is triggered to accept the pairing of the first device, and the second button is triggered to reject the pairing of the first device.

[0305] 538. The target second device obtains the trigger for the second control and sends the response instruction generated by triggering the second control to the server.

[0306] If it is detected that the first button is triggered, it is determined that the target second device accepts pairing with the first device, and the process proceeds to 539 ; if it is detected that the second button is triggered, it is determined that the target second device rejects pairing with the first device, and the process proceeds to 543 .

[0307] 539. The server sends the device network information of the opposite device to at least one of the first device and the target second device.

[0308] 540 , at least one of the first device and the target second device receives the device network information of the opposite device sent by the server, and establishes pairing with the opposite device based on the device network information.

[0309] 541. The server sends a pairing success message to the first device and the target second device.

[0310] 542. The first device receives a pairing success message sent by the server and displays the pairing success on the first configuration interface. The target second device receives a pairing success message sent by the server and displays the pairing success on the second configuration interface.

[0311] 543. The server sends a pairing failure message to the first device.

[0312] 544. The first device receives a pairing failure message sent by the server and displays the pairing failure on the first configuration interface.

[0313] A dialog box may pop up on the corresponding configuration interface to indicate whether pairing is successful or failed.

[0314] The methods of steps 531 to 535 are the same as those of steps 501 to 505 in the above embodiment, and are not described in detail here. The methods for pairing failure and unpairing can also be the same as those described in the above embodiment, and are not described in detail here.

[0315] <Admin direct connection UI interaction>

[0316] In some embodiments, the present disclosure provides a method for pairing, see Figure 24 , the method comprising:

[0317] 601. Display the acquired first device information of the first device registered with the server in a configuration interface of the display screen.

[0318] 602. Obtain target first device information associated with the target first device determined based on the first device information.

[0319] 603 , based on the target first device information, match the second device information of a pairable second device registered with the server and capable of being paired with the target first device, and display the second device information in the configuration interface.

[0320] 604 : Acquire target second device information associated with the target second device determined based on the second device information.

[0321] 605 : Obtain device network information of at least one of the target first device and the target second device.

[0322] 606 , sending the device network information to a peer device associated with the network information.

[0323] 607 , the opposite device receives the network information and establishes a pairing with the local device associated with the network information based on the network information to achieve direct communication.

[0324] In some embodiments, see Figures 25 to 28 The first device information can be displayed on the configuration interface via the list control 1a. Different first device information is associated with different list options in the list control 1a. When it is detected that a target list option is selected in the list options, the first device information associated with the target list option can be determined as the target first device information of the target first device expected for pairing.

[0325] At the same time or with a slight delay, when it is detected that the target list option in the list option 1a is selected, the second device information of the pairable second device that can be paired with the target first device can be matched from the second devices registered to the server based on the target first device information associated with the target list option. The matched second device information of the pairable second device can be displayed in the configuration interface through another list control 1b for the doctor to configure. Different second device information is associated with different list options of the another list control 1b. When it is detected that the target list option in the another list option 1b is selected, it can be determined that the second device information associated with the target list option is the target second device information of the target second device expected to be paired.

[0326] In some embodiments, after determining the target first device and the target second device, the device network information of at least one of the target first device and the target second device can be obtained in response to the triggering of an event. The event includes the server reaching a delay time from the time the target second device is determined, or a control in the configuration interface is triggered, the control being used to trigger the acquisition of the device network information of at least one of the target first device and the target second device. For example, the control is Figure 28 The server obtains the device network information of at least one of the target first device and the target second device when it obtains that the timing reaches the delay time or when it obtains that the control is triggered.

[0327] In some embodiments, after the target first device and the target second device are paired, at least one of the target first device and the target second device may return a pairing success status to the server. The server receives the pairing success status, displays a pairing success message on the configuration interface, and sends a pairing success message to the target first device and the target second device. The configuration interfaces of the target first device and the target second device may display the pairing success message. For example, this may be displayed via a pop-up dialog box.

[0328] In some embodiments, see Figures 29 to 31 , the first device information can be displayed on the configuration interface via the first drag control 1a'. Different first device information is associated with different first drag controls 1a'. The configuration interface may include a window control 1c' having a window. Upon detecting that the target first drag control in the first drag control 1a' has moved to the window of the window control 1c', the first device information associated with the target first drag control can be determined as the target first device information for pairing.

[0329] At the same time or with a slight delay, when it is detected that the target first drag control in the first drag control 1a' moves to the window of the window control 1c', the second device information of the pairable second device that can be paired with the target first device can be matched from the second device registered to the server based on the target first device information associated with the target first drag control. The matched second device information of the pairable second device can be displayed in the configuration interface outside the window through the second drag control 1b', and different second device information is associated with different second drag controls 1b'. When it is detected that the target second drag control in the second drag control 1b' moves to the window 1c', it can be determined that the second device information of the second device associated with the target second drag control is the target second device information of the target second device to be paired. For example, Figure 29 , the first device that can be used for pairing includes the remote doctor console of Hospital A~D; Figure 30When the first drag control 1a' associated with the remote doctor console of Hospital A is moved to the window 1c', the server automatically matches the second device information of the second device that is compatible with the custom protocol supported by the remote doctor console of Hospital A, such as the patient surgery platform of Hospitals A~E; Figure 31 When the second drag control 1b' associated with the patient surgery platform of Hospital B is moved to the window 1c', it is determined that the patient surgery platform of Hospital B is a device that is expected to be paired with the remote doctor console of Hospital A.

[0330] In the above embodiment, see Figures 32 to 34 , the window control 1c' may include a first area 11c' and a second area 12c'. The first area 11c' may include a first sub-area 111c' and a second sub-area 112c', the first sub-area 111c' and the second sub-area 112c' are used to accommodate the first drag control 1a', the second area 12c' is used to accommodate the second drag control 1b', and the first sub-area 111c' and the second sub-area 112c' are configured to be associated with different control permissions or different pairing orders. In which, when the first drag control 1a' is associated with the remote doctor console and the second drag control 1b' is associated with the patient surgical platform, when the second drag control 1b' moves to the second area 12c', a first drag control 1a' moves to the first sub-area 111c', and another first drag control 1a' moves to the second sub-area 112c', the remote doctor console associated with different first drag controls 1a' may be configured to have different control permissions or pairing orders for the patient surgical platform associated with the second drag control 1b'. For example, Figure 32 , the first device that can be used for pairing includes the remote doctor console of Hospital A~D; Figure 33 When the first drag control 1a' associated with the remote doctor console of Hospital A and the remote doctor console of Hospital B are moved to the first sub-area 111c' and the second sub-area 112c' respectively, the server automatically matches the second device information of the second device that is compatible with the custom protocol supported by the remote doctor console of Hospital A and the remote doctor console of Hospital B, such as the patient surgery platform of Hospitals A~C; Figure 34 When the second drag control 1b' associated with the patient surgery platform of Hospital B is moved to the second area 12c', the patient surgery platform of Hospital B is determined to be a device that is intended to be paired with the remote physician console of Hospital A and the remote physician console of Hospital B. If the first sub-area 111c' and the second sub-area 112c' are associated with different control permissions or different pairing orders, the remote physician console of Hospital A and the remote physician console of Hospital B are configured to have different control permissions or different pairing orders for the patient surgery platform of Hospital B.

[0331] <Pairing successful detection>

[0332] In some embodiments, when the remote physician console and the patient surgical platform establish real-time pairing, a successful pairing test can be performed to ensure that the remote surgery can proceed smoothly. This test can be performed using at least one network channel (i.e., communication link) established during the real-time pairing between the remote physician console and the patient surgical platform to determine whether the pairing is truly successful.

[0333] During an ultra-teleoperative procedure, multiple types of network information may be exchanged between the remote physician console and the patient's surgical platform. For example, these types of network information include first-type network information exchanged between the remote physician console and core devices on the patient's surgical platform, and second-type network information exchanged between the remote physician console and auxiliary devices on the patient's surgical platform.

[0334] The first network information can include different types of network information, such as at least one of motion control information, energy control information, multimedia interaction information, and human-machine interface interaction information. Motion control information can typically be sent by the remote doctor's console and executed by the slave robot, such as in master-slave follower control. Motion control information can also be sent by the slave robot and executed by the remote doctor's console. For example, when the remote doctor's console includes an operating unit in the form of a mechanical linkage, the slave robot sends motion control information about the posture of its medical device, and the operating unit of the remote doctor's console performs alignment control of the medical device. Energy control information is typically sent by the remote doctor's console and executed by the energy platform, such as controlling the conduction of a unipolar circuit, bipolar circuit, ultrasonic transducer, radio frequency circuit, or microwave circuit on the energy platform. Multimedia interaction information includes video interaction information and audio interaction information. Video interaction information is typically collected by an imaging device, such as an endoscope, processed by the imaging platform, and then sent to the remote doctor's console's display screen for display. Audio interaction information is typically collected by audio intercom devices installed at both the remote doctor's console and the slave robot, processed by the imaging platform, and then sent to the remote device for playback. Human-computer interaction information is usually operated by the remote doctor console on the touch screen of the display screen, and the interface changed after the operation is displayed on the display screen of the slave hand robot. The operation can be, for example, image annotation.

[0335] The second network information may also include different types of network information. The remote physician console typically interacts with different auxiliary devices through different network information interactions. The remote physician console interacts with the same auxiliary device through at least one type of network information interaction. For example, when the auxiliary device includes a monitor, the interaction information may include at least one of heart rate, pulse, and blood pressure information. For example, when the auxiliary device includes an operating table, the remote physician console may be configured to control the position or posture of the operating table. The interaction information may include, for example, motion control information of the operating table by the remote physician console.

[0336] In some embodiments, when establishing a pairing between a remote physician console and a patient surgical platform, at least one of the remote physician console and the patient surgical platform creates a network channel for communication between the two based on device network information sent by a server. During ultra-remote surgery, different network channels can be created to meet the transmission requirements of different types of network information between the remote physician console and the patient surgical platform. Creating different network channels for transmitting different types of network information can provide multiple advantages, such as facilitating bandwidth optimization, facilitating priority management, facilitating data isolation to improve security, facilitating fault isolation to improve network fault tolerance, and ensuring the compliance of specific data transmissions.

[0337] By configuring the port numbers of the remote doctor's console and the patient's surgical platform, different network channels can be created to differentiate the transmission of different types of network information. For example, a motion control network channel can be created to transmit motion control information; an energy control network channel can be created to transmit energy control information; a multimedia interaction network channel can be created to transmit multimedia interaction information; and a human-machine interface interaction network channel can be created to transmit human-machine interface interaction information.

[0338] In some embodiments, the remote doctor console or the patient surgery platform may create different network channels for transmitting network information of corresponding data types in the remote doctor console and the patient surgery platform according to the different configured control permissions, wherein different control permissions may be associated with the transmission of different data types. In some embodiments, different network channels may be created for transmitting network information between corresponding device types in the remote doctor console and the patient surgery platform according to the different medical and patient information input in the remote doctor console or the patient surgery platform, wherein different medical and patient information may be associated with the transmission of network information between different device types. Exemplarily, medical and patient information includes, for example, the type of surgery or the surgical equipment to be used, and different medical and patient information may need to be used in conjunction with different core devices or auxiliary devices. Therefore, different network channels may be created based on the number or type of devices that need to be connected.

[0339] In some embodiments, the present disclosure also provides a method for detecting whether pairing is successful. Figure 35 , the method comprising:

[0340] 711. The server obtains information about a target network channel to be established between the first device and the second device, and based on the obtained target network channel information, sends device network information of the opposite device associated with the target network channel to at least one of the first device and the second device.

[0341] The first device and the second device are paired. Different network channels typically have different port numbers. The server sends a port number associated with a target network channel to at least one of the first device and the second device.

[0342] 712 : At least one of the first device and the second device receives device network information of the opposite device associated with the target network channel sent by the server, and creates the target network channel with the opposite device based on the device network information.

[0343] 713 : At least one of the first device and the second device evaluates a communication status of the target network channel.

[0344] In some embodiments, the communication status of the target network channel can be evaluated by detecting network connectivity. For example, a heartbeat mechanism can be used to detect the network connectivity of the target network channel. The use of the heartbeat mechanism requires the creation of a sender and a receiver at both ends of the target network channel. One of the first device and the second device can be used as the sender and the other can be used as the receiver. When the sender and the receiver are used as a pair, they can use the TCP protocol or the UDP protocol. When using the heartbeat mechanism for detection, the sender sends a message to the receiver via the target network channel. If the receiver returns a signal indicating that the message has been received within a predetermined time, it indicates that the network connection between the sender and the receiver is normal, that is, the network connectivity of the target network channel between the two is normal. If the receiver does not return a signal indicating that the message has been received within a predetermined time, it indicates that the network connection between the sender and the receiver is disconnected, that is, the network connectivity of the target network channel between the two is abnormal.

[0345] In some embodiments, the target network channel includes one or more. When there are multiple target network channels, different senders and receivers can be created for different target network channels. In other words, when the target network channels include a first target network channel and a second target network channel, for the first target network channel, the first device can serve as the sender and the second device can serve as the receiver; for the second target network channel, the second device can serve as the sender and the first device can serve as the receiver.

[0346] In some embodiments, the communication status of the target network channel can also be evaluated by detecting the network communication quality. For example, the network communication quality can be evaluated by detecting at least one of network delay, jitter, packet loss, and out-of-order transmission of messages in the target network channel. For example, the network communication quality can be evaluated by evaluating the network delay. Similarly, a sending end and a receiving end are created at both ends of the target network channel. The sending end sends a preset type of instruction to the receiving end through the target network channel. The preset type of instruction can be related to the type of network information to be transmitted by the target network. The instruction is encapsulated into a message for transmission according to a custom protocol agreed upon between the devices at both ends of the target network channel. The message includes a sending timestamp. After receiving the message, the receiving end returns a signaling to the sending end through the target network channel. The signaling includes a receiving timestamp when the receiving end receives the message. The sending end calculates the difference between the receiving timestamp and the sending timestamp and compares the difference with a preset delay threshold corresponding to the target network channel. Exemplarily, when the difference is less than the preset delay threshold, the network communication quality is determined to be excellent; when the difference exceeds a first range of the preset delay threshold, the network communication quality is determined to be good; and when the difference exceeds a second range of the preset delay threshold, the network communication quality is determined to be poor. In some embodiments, when the network communication quality is a first quality, such as excellent or good, the network communication quality of the target network channel can be determined to be normal; and when the network communication quality is a second quality, such as poor, the network communication quality of the target network channel can be determined to be abnormal.

[0347] 714 , when the communication status of the evaluated target network channel is normal, at least one of the first device and the second device determines that pairing of the first device and the second device is successful.

[0348] 715 , when the communication state of the evaluated target network channel is abnormal, at least one of the first device and the second device determines that pairing of the first device and the second device fails.

[0349] The target network channels may include multiple ones. Whether the pairing between the first device and the second device is successful usually requires comprehensive consideration of the communication status of the multiple target network channels.

[0350] For example, network connectivity is a key indicator for evaluating communication status. If the network connectivity of all target network channels is normal, it can be determined that the pairing of the first device and the second device is successful. If the network connectivity of one or more target network channels is abnormal, it can be determined that the pairing of the first device and the second device has failed. For example, network communication quality is a key indicator for evaluating communication status. If the network communication quality of all target network channels is normal, it can be determined that the pairing of the first device and the second device is successful. If the network communication quality of one or more target network channels is abnormal, it can be determined that the pairing of the first device and the second device has failed.

[0351] In some embodiments, for example, when the first device or the second device is configured to have operator authority, a comprehensive evaluation of whether the pairing of the first device and the second device is successful can be made based on the priority of the network information transmitted by the target network channel and the communication status of the target network channel. Some target network channels are used to transmit high-priority network information, such as motion control information, energy control information, and video information; some target network channels are used to transmit lower-priority network information, such as audio information and human-computer interaction information. In order to improve the pairing success rate, the system can be made to have a higher degree of redundancy. For example, as long as the communication status of all target network channels used to transmit high-priority network information is normal, even if the communication status of all target network channels used to transmit low-priority network information is abnormal, it can be determined that the pairing of the first device and the second device is successful. For example, as long as the communication status of any target network channel used to transmit high-priority network information is abnormal, it can be determined that the pairing of the first device and the second device has failed.

[0352] After 714 or 715 , a pairing success or pairing failure prompt may be provided in the first device and the second device.

[0353] In the above-described step 711, at least one of the first device and the second device may obtain the configured control authority or the medical-patient information input by the doctor, and may determine information about the target network channel to be established between the first device and the second device based on the control authority or the medical-patient information, and transmit the determined target network channel information to the server. The server may also obtain the configured control authority or the medical-patient information input by the doctor, and may determine information about the target network channel to be established between the first device and the second device based on the control authority or the medical-patient information.

[0354] In some embodiments, alternatively to 711, the server sends device network information of the peer device to at least one of the first device and the second device, where the device network information includes all network channels that may be created.

[0355] 712 Alternatively, at least one of the first device and the second device receives device network information of the peer device, and can determine device network information associated with the target network channel from the device network information based on the configured control authority or medical-patient information, and then create a target network channel with the peer device based on the device network information of the target network channel for communication.

[0356] Alternatively, at least one of the first and second devices receives device network information from the peer device and, based on the device network information, establishes all network channels with the peer device for communication. Alternatively, at least one of the first and second devices may determine a target network channel from all network channels based on configured control permissions or patient information, and then evaluate the communication status of the target network channel. In this embodiment, the first and second devices establish all network channels but may only detect the communication status of the necessary network channels, i.e., the target network channels.

[0357] In some embodiments, when it is determined that the first device and the second device are successfully paired, the corresponding device, such as the first device or the second device, may allow or prohibit the use of corresponding control permissions based on the different network communication qualities of multiple target network channels; it may also allow or prohibit the implementation of corresponding surgical procedures based on the different network communication qualities of multiple target network channels. For example, when the network communication quality of multiple target network channels for transmitting high-priority network information, such as motion control information, energy control information, and video information, is better, relatively dangerous surgical procedures may be allowed, such as surgery on arteries, and organs such as lungs, liver, kidneys, and stomach; for example, when the network communication quality of any target network channel among multiple target network channels for transmitting motion control information, energy control information, and video information is average, only relatively safe surgical procedures may be allowed, such as surgery on skin and fat.

[0358] In some embodiments, the remote doctor console's control authority over the patient's surgical platform includes, but is not limited to, control authority over at least one of a hand robot, an imaging platform, and an energy platform.

[0359] Exemplarily, control permissions include operator permissions and observer permissions, with operator permissions being of a higher level than observer permissions. Operator permissions overwrite observer permissions, meaning observer permissions are part of operator permissions. Therefore, the network channels required for operator permissions include those required for observer permissions, and the number of network channels required for operator permissions is greater than the number of network channels required for observer permissions.

[0360] In some embodiments, observer permissions apply to the observer role, primarily in teaching scenarios. Interns can learn by observing doctors performing surgeries, and doctors can also provide guidance by observing interns performing surgeries. Observer permissions are primarily embodied in the imaging platform. Configuring observer permissions typically requires establishing a network channel between the remote physician console and the imaging platform. The imaging platform can process multimedia interaction information and human-computer interface interaction information. Multimedia interaction information includes, for example, video and audio information. Human-computer interface interaction information includes, for example, annotations or labeling of surgical images captured by the endoscope and displayed on the display screen of the remote physician console. Video information can originate from an endoscope or an external camera mounted on a slave robot and electrically connected to the imaging platform. The external camera is used to monitor the operating status of at least some of the equipment on the patient's surgical platform. After being processed by the imaging platform, the video information and human-computer interface interaction information are transmitted to the remote physician console via the network channel between the imaging platform and the remote main console, where they are displayed on the remote physician console's display screen. The audio information can come from an audio device installed on the slave robot and electrically connected to the imaging platform. The audio device includes an input device and an output device. The input device is such as a microphone and the output device is such as a speaker. The remote doctor console also includes an audio device. The input device of the remote doctor console and the slave robot can collect sound input, such as voice conversations between users at both ends, and after processing by the imaging platform such as denoising, it is transmitted to the output device of the opposite device through the network channel between the remote doctor console and the imaging platform for playback. Among them, depending on the type of multimedia interactive information, the same or different network channels can be created between the remote doctor console and the imaging platform to transmit different types of multimedia interactive information. For example, under the observer authority, when creating different network channels, three network channels can be created respectively, one for transmitting video information, one for transmitting human-computer interface interaction information, and one for transmitting audio information.

[0361] In some embodiments, operator permissions apply to the identity of the operator, including the physician performing the surgery. Operator permissions can be reflected in various devices on the patient surgical platform, including the imaging platform, the slave robot, and the energy platform. The surgical procedure performed by the physician may include motion control of medical devices (including energy devices) mounted on the slave robot within the surgical field of view provided by the imaging platform, and energy control of the energy devices mounted on the slave robot through control of the energy platform. During the procedure, the physician may annotate or label the surgical images and communicate with the physician or patient on the slave robot side via voice. Therefore, under operator permissions, the remote physician console may establish multiple network channels with the imaging platform, slave robot, energy platform, and auxiliary devices to transmit different types of network information. In some embodiments, multiple network channels can be established between the remote physician console and any device based on the specific signal types transmitted or the physical communication links between the two devices. In some embodiments, for example, for a laparoscopic slave robot or an interventional slave robot, the slave robot includes multiple manipulators, each of which is used to assemble a medical device. The remote physician console's motion control of the medical device mainly relies on the motion control of the manipulators, and the remote physician console's motion control of different manipulators is independent of each other, and their physical communication links are different. Therefore, when creating a motion control network channel between the remote physician console and the slave robot, multiple motion control network channels associated with different manipulators can be created. In some embodiments, the energy platform may include processing of different types of energy control signals, and when creating an energy control network channel between the remote physician console and the energy platform, multiple energy control network channels associated with different types of energy control signals can be created.

[0362] In some embodiments, different control permissions can be configured in a more detailed manner. When a specific permission is allowed, any device can open or create a network channel corresponding to that permission. When a specific permission is prohibited, any device can close or destroy the network channel corresponding to that permission. Disabling a specific permission can help shield interference on the one hand; on the other hand, it can help free up bandwidth resources when network bandwidth is insufficient. For example, when a visual permission that consumes a lot of network bandwidth is disabled, the released bandwidth resources can help ensure the transmission of other information.

[0363] In some embodiments, observer permissions may include viewing permissions, audio and video permissions, and annotation permissions, corresponding to a first network channel for transmitting video information, a second network channel for transmitting audio information, and a third network channel for transmitting human-computer interface interaction information. For example, observer permissions may be configured to include viewing permissions and audio and video permissions, enabling or creating the first and second network channels while simultaneously disabling or destroying the third network channel.

[0364] In some embodiments, operator permissions may include partial operator permissions and full operator permissions, both of which can override observer permissions. A remote physician console can obtain partial operator permissions for a patient surgical platform, which is applicable when multiple remote physician consoles are required to collaboratively operate a patient surgical platform to perform a surgery. A remote physician console can also obtain full operator permissions for a patient surgical platform, which is applicable when a remote physician console is required to independently operate a patient surgical platform to perform a surgery.

[0365] In some embodiments, partial operator authority includes, for example, authority to control some manipulators or some types of medical devices in a slave robot. For example, the slave robot may include a first manipulator, a second manipulator, a third manipulator, and a fourth manipulator, each manipulator may be equipped with one medical device, and the medical devices equipped with different manipulators may be the same or different. Partial operator authority may be configured so that a remote doctor console can only control the first and second manipulators, while the unconfigured third and fourth manipulators may be assigned to one or more other remote doctor consoles for control. Configuration of partial operator authority may be achieved based on a unique identifier of the manipulator, such as the manipulator serial number. For another example, medical devices may include first, second, third, and fourth types of medical devices. The types of medical devices are recorded in their storage devices. When the medical devices are installed on the manipulator of the slave hand robot, they are identified by the read-write device set in the manipulator and can be sent to the remote doctor console through the motion control network channel between the slave hand robot and the remote doctor console. The remote doctor console can determine whether it can control the medical device based on the acquired type of medical device. Part of the operator authority can be configured so that a remote doctor console can only control medical devices of the first and second types, and unconfigured medical devices of the third and fourth types can be configured to be controlled by one or more other remote doctor consoles. For example, the first type of medical device is surgical forceps, the second type of medical device is an endoscope, the third type of medical device is an electrosurgical unit, and the fourth type of medical device is a stapler.

[0366] In some embodiments, full operator authority includes, for example, authority to control all manipulators in a hand robot or all types of medical instruments.

[0367] In some embodiments, after the above 715, when a prompt is given for pairing failure, a prompt may be given for the target network channel with abnormal communication status, prompting the doctor to detect or maintain the target network channel, or prompting the doctor to reconfigure the control authority to avoid the image of pairing failure caused by the target network with abnormal communication status. For the latter, for example, in a substitute teaching scenario, when the doctor learns from the prompt of pairing failure that the network channel corresponding to the observer's visual permission is normal, while the network channel corresponding to the auditory and audible permission is abnormal, the doctor can reconfigure the observer's permission to include only the visual permission to ensure successful pairing, and even if the network channel corresponding to the auditory and audible permission is closed or destroyed, there will be no major obstacle to the implementation of the substitute teaching function.

[0368] In some embodiments, when creating a target network channel with a peer device, target network channels with different communication protocol standards can be created based on the different control permissions obtained. For example, when the control permission is observer permission, a target network channel with a UDP protocol standard can be created, such as multimedia network channels and human-machine interface interaction network channels that adopt the UDP protocol standard, which has a faster signaling transmission speed. For another example, when the control permission is operator permission, for observer permissions covered by operator permissions, a target network channel with a UDP protocol standard can be created, and for other permissions, a target network channel with a TCP protocol standard can be created, such as motion control network channels and energy control network channels that adopt the TCP protocol standard, which has more reliable signaling transmission, with very little packet loss and disorder, and can ensure the reliable implementation of remote surgery. Of course, each target network channel can also unify the communication protocol standard.

[0369] In some embodiments, a target network channel is created based on medical and patient information. The medical and patient information may include the type of surgery the doctor or patient desires to perform. For example, when the surgery type is laparoscopic, a target network channel is created between the remote doctor's console and the corresponding equipment of the (single-port / multi-port) laparoscopic slave robot; when the surgery type is (bronchial / vascular) interventional surgery, a target network channel is created between the remote doctor's console and the corresponding equipment of the (bronchial / vascular) interventional slave robot. Medical and patient information may include the surgical equipment required for the surgery. For example, when the medical and patient information includes that the surgical equipment requires a slave robot, a monitor, and a ventilator, a target network channel is created between the remote doctor's console and the slave robot, the monitor, and the ventilator; when the medical and patient information includes that the surgical equipment requires a slave robot but does not require a monitor and ventilator, a target network channel is created between the remote doctor's console and the slave robot.

[0370] In some embodiments, the entire pairing phase may also include the configuration of control permissions. Whether control permissions are successfully configured may affect the reliable implementation of ultra-teleoperative surgery and is an important reference for measuring the success of pairing. Among control permissions, the successful configuration of operator permissions is particularly important for ensuring the reliable implementation of ultra-teleoperative surgery. In most cases, an ultra-teleoperative surgery primarily involves controlling the core devices in the patient surgical platform, including both motion control of the slave robot by the remote physician console and energy control of the energy platform by the remote physician console. Therefore, operator permissions typically involve both motion control permissions and energy control permissions. However, in some cases, there may be one or more remote physician consoles that are paired with the patient surgical platform in real time. When multiple remote physician consoles are included, one or more remote physician consoles may be configured with only observer permissions, while one or more remote physician consoles may be configured with operator permissions. Among these, one remote physician console may be configured with full operator permissions, meaning that it has both motion control permissions for the slave robot and energy control permissions for the energy platform. A remote doctor console may be configured to have partial operator authority, for example, including one of the motion control authority over the slave hand robot and the energy control authority over the energy platform being obtained by the remote doctor console; or, including partial motion control authority over the slave hand robot being obtained by the remote doctor console; or, including partial energy control authority over the energy platform being obtained by the remote doctor console, wherein the energy platform may include multiple energy output links, and different energy output links may be controlled by different remote doctor consoles, that is, multiple energy output links may include multiple energy control authorities.

[0371] In a master-slave surgical robot system, operator permissions are typically transparent to both the operator and the executor. This means both are aware of their respective operator permissions. For example, the operator knows which operations they can control the executor to perform, and the executor knows which operations controlled by the operator can be performed. One of the operator and executor can be a remote physician console, and the other can be a patient surgical platform. For example, in master-slave control, the operator is the remote physician console, and the executor is the slave robot or energy platform of the patient surgical platform. In master-slave alignment, the operator is the slave robot, and the executor is the remote physician console.

[0372] In some embodiments, the successful configuration of operator permissions includes at least two aspects. The first aspect includes that the configuration is successful if the permissions are possessed (i.e., allowed), indicating that the desired operation can be performed; the second aspect includes that the configuration is successful if the permissions are not possessed (i.e., prohibited), indicating that the desired operation cannot be performed. Whether the operator permissions are successfully configured includes detecting at least one of the two aspects. Based on this, for a remote doctor console and a patient surgical platform that have established real-time pairing, the present disclosure provides a method for detecting whether the operator permissions are successfully configured, the method comprising:

[0373] 721 , at least one of the remote doctor console and the patient surgery platform obtains the ownership permission among the configured operator permissions.

[0374] The ownership authority mainly includes authority attribution, and authority attribution includes association with the controlled object. The operating end can only control the controlled object in the execution end, but cannot control the uncontrolled object. From the perspective of the remote doctor's console, the controlled object may include a specific manipulator component in the slave hand robot that is expected to be controlled or a manipulator component associated with a specific type of medical device; the controlled object may include a specific energy output link in the energy platform that is expected to be controlled. From the perspective of the patient's surgical platform, the controlled object may include a specific manipulator component in the slave hand robot that is expected to be controlled by the remote doctor's console or a manipulator component associated with a specific type of medical device; the controlled object may include a specific energy output link in the energy platform that is expected to be controlled by the remote doctor's console. Therefore, whether it is viewed from the remote doctor's console or from the patient's surgical platform, the two ends are corresponding in terms of authority attribution. Among them, "specific" includes all or part.

[0375] Different permission types can be closely linked to the operator's permission type. For example, when the controlled object includes a manipulator component, the permission type is typically associated with motion control permission. Another example is when the controlled object includes a specific energy output link, the permission type is typically associated with energy control permission. In other words, when configuring operator permissions, it's usually sufficient to configure the permission type, and the associated permission type can be configured by default based on the permission type.

[0376] In addition, different types of control instructions are usually required for the controlled object based on its permission type. For example, if the permission type is motion control permission, motion control instructions need to be sent to control the controlled object; if the permission type is energy control permission, energy control instructions need to be sent to control the controlled object.

[0377] 722. The remote doctor console determines the controlled object from the patient's surgical platform based on the ownership of the authority, and determines the target control instruction associated with the controlled object from the preset control instruction set.

[0378] Even if the slave robots in two patient surgical platforms are of the same type, differences in machining and assembly may result in slightly different kinematics. Therefore, a preset control instruction set can be designed for each patient surgical platform. The preset control instruction set includes motion control instructions simulating the motion of the operator's console and energy control instructions simulating the activation of the pedal assembly of the remote physician console. Both sets of control instructions may include at least one control instruction.

[0379] For example, in a laparoscopic slave robot, different motion control instructions can be designed for different manipulator components within the slave robot. These motion control instructions primarily control the medical device mounted on the manipulator by controlling the manipulators within the manipulator components. These motion control instructions are typically posture instructions, which include at least one of a position instruction and a posture instruction.

[0380] In some embodiments, the slave robot includes a main arm and multiple manipulators connected to the distal end of the main arm. When the slave robot is powered on and initialized, the main arm unfolds to drive each manipulator to a first initial position fixed relative to a base coordinate system. When a medical device is assembled to the manipulator, the manipulator performs initialization operations on the medical device, which also allows the medical device to be fixed relative to the base coordinate system and maintain a second initial position, which is different from the first initial position. Medical devices include endoscopes and surgical instruments. Typically, the movement of an endoscope is relative to a base coordinate system, and the movement of a surgical instrument is relative to an endoscope coordinate system. However, when detecting whether the motion control authority is successfully configured, the control method can be simplified. For example, the remote doctor console can be defined to control the movement of all types of medical devices relative to the base coordinate system.

[0381] In the manipulator assembly of the same slave robot, the kinematic models of the manipulators are basically the same, and the kinematic models of the medical devices may be the same or different.

[0382] Assuming that the kinematic models of different medical devices suitable for the slave robot are basically the same, when each medical device is in the second initial position relative to the base coordinate system, the current joint parameters of each joint in the manipulator and the target joint increments of at least some of the joints can be combined, and the target position of the medical device relative to the base coordinate system can be determined based on the kinematic model of the manipulator and the kinematic model of the medical device. The target joint increment is pre-designed and can be a small value. For example, when the joint parameters are all joint angles, the target joint increment can be designed to be 0.1°, 0.5°, 1°, 1.5°, 2°, or other values ​​between any two of the aforementioned values, so as to reduce the amplitude of the movement used to verify whether the configuration of the ownership permission is successful. Of course, the target position of each manipulator component can also be a directly specified position relative to the base coordinate system, and the target position is different from its corresponding second initial position. For example, the target position is an upward, downward, left, right, forward or backward position relative to the second initial position. In this case, the target position of each manipulator component will not change due to the installation of different medical devices on the manipulator, that is, each manipulator component is associated with a relatively fixed target position. The target pose designed here is the target motion control instruction used to detect whether the motion control authority has been successfully configured. Furthermore, in 722, the steps of determining a controlled object from the patient surgical platform and determining a target control instruction associated with the controlled object from a preset control instruction set include: determining a controlled manipulator component from the slave hand robot, and determining a target motion control instruction associated with the controlled manipulator component from the preset control instruction set based on the controlled manipulator component.

[0383] Assuming that the kinematic models of different medical devices suitable for the slave robot are substantially different, the target poses for different medical devices mounted on different manipulators can be pre-designed based on a similar method described above, with the manipulator and medical device in corresponding initial pose states. When the same medical device is mounted on different manipulators, the target poses of the manipulator assemblies are different; when different medical devices are mounted on the same manipulator, the target poses of the manipulator assemblies are also different. In other words, the target pose of the manipulator assembly is associated with the medical device itself and the manipulator on which the medical device is mounted. Furthermore, in 722, the steps of determining a controlled object from the patient surgical platform and determining target control instructions associated with the controlled object from a preset control instruction set include: determining a controlled manipulator assembly from the slave robot, identifying the medical device in the controlled manipulator assembly, and determining target motion control instructions associated with the controlled manipulator assembly from the preset control instruction set based on the controlled manipulator assembly and the medical device mounted thereon.

[0384] In some embodiments, the design of the energy control instructions is relatively simple. When the energy platform includes multiple energy output links, the energy control instructions include multiple energy control instructions corresponding to the multiple energy output links. These energy control instructions may include switch control instructions, current regulation instructions, voltage regulation instructions, power regulation instructions, or frequency regulation instructions. Furthermore, in 722, the steps of determining a controlled object from the patient surgical platform and determining a target control instruction associated with the controlled object from a preset control instruction set include: determining a controlled energy output link from the energy platform, and determining a target energy control instruction associated with the controlled energy output link from the preset control instruction set.

[0385] At 723, the remote doctor console sends target control instructions to the patient's surgical platform.

[0386] The communication between the remote doctor console and the patient surgery platform is achieved through a network channel established between the two. The communication includes the transmission of target control instructions and the transmission of the actual execution results of the target control instructions.

[0387] 724 , the patient surgery platform receives the target control instruction, and the controlled object of the patient surgery platform executes the target control instruction.

[0388] In some embodiments, a doctor on the controlled object side can observe the controlled object's execution of the target control instruction and notify the remote doctor console or the patient's surgical platform of the observation results. The observation results include successful execution or failed execution. Successful execution may indicate successful configuration of the ownership authority in the operator's authority, and failed execution may indicate failed configuration of the ownership authority in the operator's authority.

[0389] In some embodiments, see Figure 36 , the method may further include:

[0390] 725. After the controlled object executes the target control instruction, the patient surgery platform returns the actual execution result to the remote doctor console.

[0391] If the target control instruction is a motion control instruction, the actual execution result may include the actual position and posture of the corresponding manipulator component relative to the base coordinate system. If the target control instruction is an energy control instruction, the actual execution result may include the actual energy output state of the corresponding energy transmission link, including the actual switching state, actual current output value, actual voltage output value, actual power output value, or actual frequency output value.

[0392] 726 , the remote doctor console receives the actual execution result and detects whether the actual execution result matches the expected execution result.

[0393] The expected execution result may be an execution result determined based on the target control instruction under ideal conditions. If the target control instruction is a motion control instruction, the expected execution result may include an expected position of the corresponding manipulator component relative to the base coordinate system. If the target control instruction is an energy control instruction, the expected execution result may include an expected energy output state of the corresponding energy transmission link, an expected switching state of the expected energy output state, an expected current output value, an expected voltage output value, an expected power output value, or an expected frequency output value.

[0394] 727, when it is detected that the actual execution result matches the expected execution result, it is determined that the permission configuration is successful.

[0395] The actual execution result matches the expected execution result, which includes at least one meaning. This meaning includes the controlled object responding to the target control instruction action. This meaning includes the controlled object moving in response to the motion control instruction, or the controlled object outputting energy in response to the energy control instruction.

[0396] The actual execution result matches the expected execution result, which can also include another layer of meaning. This layer of meaning includes that the controlled object accurately responds to the target control instruction action. This layer of meaning includes that when the target control instruction is a motion control instruction, in the same coordinate system, the deviation between the actual posture and the expected posture is within the deviation threshold; or, when the target control instruction is an energy control instruction, the actual energy output state is the actual switching state and the expected energy output state is the expected switching state, the actual energy output state and the expected energy output state should be consistent, for example, the open state; when the actual energy output state is the actual current output value, the actual voltage output value, the actual power output value or the actual frequency output value, and the expected energy output state is the expected current output value, the expected voltage output value, the expected power output value or the expected frequency output value, the deviation between the actual energy output state and the expected energy output state is within the deviation threshold.

[0397] 728, when it is detected that the actual execution result does not match the expected execution result, it is determined that the permission configuration fails.

[0398] Through 725~728, it is possible to more intelligently determine whether the ownership permissions in the operator permissions are configured successfully.

[0399] In some embodiments, see Figure 37 , it is also possible to further detect whether the unowned permissions in the operator permissions are configured successfully, and the method may include:

[0400] 731, the remote doctor console and the patient surgery platform obtain unowned permissions among the configured operator permissions.

[0401] Operator authority includes possessed authority and non-possessed authority. Non-possessed authority can be obtained independently or based on possessed authority in operator authority. As mentioned above, the authority of possessed authority is associated with the controlled objects in the patient surgical platform, and the authority of non-possessed authority is associated with the uncontrolled objects in the patient surgical platform. Assuming that different uncontrolled objects can be controlled, corresponding control instructions need to be sent. For example, when the uncontrolled object is a manipulator component, motion control instructions need to be sent. When the uncontrolled object is an energy output link, energy control instructions need to be sent.

[0402] 732. The remote doctor console determines the non-controlled object from the patient's surgical platform based on the ownership of the authority that is not possessed, and determines the target control instruction associated with the non-controlled object from the preset control instruction set.

[0403] The design of the preset control instruction set for the non-controlled object can refer to the design of the preset control instruction set for the controlled object, and will not be repeated here.

[0404] When the kinematic models of different medical devices suitable for the slave robot are basically the same, in 732, the steps of determining the uncontrolled object from the patient surgical platform and determining the target control instructions associated with the uncontrolled object from the preset control instruction set include: determining the uncontrolled manipulator component from the slave robot, and determining the target motion control instructions associated with the uncontrolled manipulator component from the preset control instruction set based on the uncontrolled manipulator component.

[0405] When the kinematic models of different medical devices suitable for the slave robot are basically different, in 732, the steps of determining the uncontrolled object from the patient surgical platform and determining the target control instructions associated with the uncontrolled object from the preset control instruction set include: determining the uncontrolled manipulator component from the slave robot, identifying the medical device in the uncontrolled manipulator component, and determining the target motion control instructions associated with the uncontrolled manipulator component from the preset control instruction set based on the uncontrolled manipulator component and the medical device assembled thereon.

[0406] In 732, the steps of determining an uncontrolled object from the patient surgical platform and determining a target control instruction associated with the uncontrolled object from a preset control instruction set include: determining an uncontrolled energy output link from the energy platform, and determining a target energy control instruction associated with the uncontrolled energy output link from the preset control instruction set.

[0407] 733, the remote doctor console sends target control instructions to the patient's surgical platform.

[0408] 734 , the patient surgery platform receives the target control instruction, and the non-controlled objects of the patient surgery platform execute the target control instruction.

[0409] If the permission configuration is unsuccessful, the non-controlled object will typically not receive the target control instruction, or even if it receives the target control instruction, it will not execute the target control instruction. Reasons for not receiving the target control instruction include: the network channel between the remote physician console and the non-controlled object is not established according to the control permission configuration; or, even if the network channel between the remote physician console and the non-controlled object is established, the target control instruction cannot be transmitted between the two according to the control permission configuration. Therefore, in step 734, the patient surgical platform may not receive the target control instruction, or its non-controlled object may not execute the target control instruction. In other words, regardless of whether the permission configuration is successful, the state of the non-controlled object will not change, including the motion state or energy output state. If the permission configuration fails, the state of the non-controlled object will change, including the motion state or energy output state. In abnormal situations, the reasons for the failure of the permission configuration may include interference with the transmission of the target control instruction in the network channel, or the incorrect establishment of the network channel between the remote physician console and the patient surgical platform.

[0410] Whether the state of the uncontrolled object occurs can be determined by the doctor's observation on the uncontrolled object side. The doctor can notify the remote doctor console or the patient's surgery platform of the observation result, so as to manually determine whether the unauthorized configuration is successful.

[0411] In some embodiments, see Figure 17 , the method may further include:

[0412] 735. After the non-controlled object executes the target control instruction, the patient surgery platform returns the actual execution result to the remote doctor console.

[0413] Regardless of whether the non-controlled object executes the target control instruction, the patient surgery platform can return the actual execution result to the remote doctor console. The actual execution result includes whether the state of the non-controlled object has changed.

[0414] 736 , the remote doctor console receives the actual execution result and detects whether the actual execution result matches the expected execution result.

[0415] When authorization is not granted, the expected execution result associated with the target control instruction typically includes the state of the uncontrolled object remaining unchanged. This includes the motion state or energy output state remaining unchanged. A motion state that remains unchanged means the position and orientation of the uncontrolled manipulator component remains unchanged; an energy output state that remains unchanged means the switch state, current output value, voltage output value, power output value, or frequency output value of the uncontrolled energy platform remains unchanged.

[0416] 737: When it is detected that the actual execution result matches the expected execution result, it is determined that the permission configuration is not successful.

[0417] For example, when the actual execution result corresponds to the state of the non-controlled object not changing, which matches the expected execution result corresponding to the state of the non-controlled object not changing, it can be determined that the permission configuration is not successful.

[0418] 738. When it is detected that the actual execution result does not match the expected execution result, it is determined that the permission configuration fails.

[0419] For example, when the state of the non-controlled object corresponding to the actual execution result changes, it does not match the expected execution result that the state of the non-controlled object does not change, and it can be determined that the permission configuration fails.

[0420] Through 735~738, it is possible to more intelligently determine whether the unowned permissions in the operator permissions are configured successfully.

[0421] In some embodiments, when one or more remote physician consoles are paired with a patient surgical platform in real time, for any remote physician console, if both the configuration of permissions and the configuration of non-permissions are successful, it can be confirmed that the remote physician console has been successfully paired with the patient surgical platform; if the configuration of permissions or non-permissions fails, it can be confirmed that the remote physician console has failed to be paired with the patient surgical platform. This pairing success detection can fully ensure the smooth implementation of ultra-teleoperative surgery.

[0422] <Switching Pairing>

[0423] In some embodiments, a device (the local device) can pair with multiple peer devices. For example, a remote physician console can pair with multiple patient surgical platforms. Another example is a patient surgical platform that can pair with multiple remote physician consoles. A physician can configure a single device to pair with multiple peer devices simultaneously. These pairings can be scheduled. The device can obtain pairing information with multiple peer devices from a server. This information includes the pairing order of the peer devices and their network information. Since the device can record the order in which it pairs with other peer devices, the pairing order can also be determined by the device itself. The pairing order can be defaulted, for example, pairs configured earlier will have an earlier pairing order, while pairs configured later will have a later pairing order. The pairing order can also be preset, for example, pairs with the first peer device can have an earlier pairing order, while pairs with the second peer device can have a later pairing order. The pairing order can be determined by the scheduled pairing time, with pairs with earlier scheduled pairing times having an earlier pairing order. The pairing order can be determined based on the urgency of the surgery. For example, for medical and patient information entered within the same time range, such as 0 to 30 minutes, surgeries with higher urgency will be assigned a higher pairing order by default. When the time range is exceeded, the pairing order can be set according to the default pairing method.

[0424] In some surgical scenarios, to ensure surgical safety and privacy, a device may be paired with at most one peer device in real time at a time, while maintaining scheduled pairings with other peer devices. For example, a remote physician console may be paired with at most one patient surgical platform in real time, while maintaining scheduled pairings with other patient surgical platforms. Another example is a patient surgical platform may be paired with at most one remote physician console in real time, while maintaining scheduled pairings with other remote physician consoles.

[0425] In these surgical scenarios, the present disclosure provides a method for quickly switching pairings, see Figure 38 , the method comprising:

[0426] 801. The device obtains pairing information of multiple peer devices with which the device has established scheduled pairings.

[0427] The pairing information includes a pairing order and device network information of multiple peer devices. The multiple peer devices include a first peer device and a second peer device.

[0428] 802. When the device determines that a first peer device among multiple peer devices has a priority pairing order based on the pairing order in the pairing information, the device establishes real-time pairing with the first peer device based on the device network information of the first peer device in the pairing information.

[0429] 803. When the device detects that the switching pairing condition is met, the device cancels the real-time pairing with the first peer device and establishes a real-time pairing with the second peer device based on the device network information of the second peer device in the pairing information.

[0430] When the device terminates real-time pairing with the first peer device, the device may update the pairing information. For example, the device may mark the first peer device as previously paired, or may delete the device network information of the first peer device to prevent the device from being paired with the first peer device again in real time. The device may send the updated pairing information to the server for the server to maintain a pairing table, such as marking the pairing status of the relevant device.

[0431] In this embodiment, the device is allowed to establish a new real-time pairing with the second peer device only when the switching pairing condition is met, which can prevent the accidental interruption of the ultra-teleoperative surgery being performed by the first peer device and ensure the coherent implementation of the ultra-teleoperative surgery.

[0432] In some embodiments, satisfying the switching pairing condition includes at least two aspects: the device and the first peer device satisfy a real-time pairing release condition; and the device and the second peer device satisfy a real-time pairing establishment condition.

[0433] In some embodiments, the device and the first peer device satisfy at least one of the conditions for releasing the real-time pairing: completion of the surgery, abnormal interruption of the surgery, and agreement between the two devices to switch pairing.

[0434] Regarding the completion of the operation, it includes at least one of (1) to (3): (1) the physical accessories necessary for the operation are separated from the hand robot or the patient's body and the delay time is reached, or the hand robot is restored to the preoperative preparation state and the delay time is reached; (2) the imaging platform is shut down and the delay time is reached, or the field of view provided by the imaging device such as an endoscope in the hand robot is switched from the surgical field of view to the non-surgical field of view and the delay time is reached; (3) the doctor or patient is evacuated from the surgical environment and the delay time is reached.

[0435] Regarding (1), for various types of master-slave surgical robot systems, for example, for laparoscopic surgical robot systems or interventional surgical robot systems, obtaining the detachment of necessary physical accessories from the slave robot includes: obtaining the detachment of all medical instruments from the slave robot (its manipulator), the detachment of the sterile cover from the slave robot, and the detachment of the puncture device for guiding the insertion of medical instruments into the patient's body from the slave robot (its manipulator). Obtaining the detachment of necessary physical accessories from the patient's body includes: obtaining the withdrawal of all medical instruments from the patient's body, the withdrawal of the puncture device from the patient's body, and the detachment of auxiliary equipment such as a monitor or a ventilator from the patient's body. Obtaining the slave robot's return to the preoperative preparation state includes obtaining the return of the slave robot's robotic arm to the folded state, or the reset of the support legs of the slave robot chassis.

[0436] Regarding (2), during surgery, the provision of images is crucial, and the time it takes for the imaging platform to be shut down to reach the delay time often indicates that the surgery is complete. The time it takes for the field of view to switch from the surgical field of view to the non-surgical field of view to reach the delay time includes: the time it takes for the field of view to switch from the surgical field of view to the non-surgical field of view to reach the delay time, including the time it takes for the endoscope to be removed from the patient's body, or the time it takes for the image captured by the endoscope to not include human tissue or organs.

[0437] Regarding (3), obtaining that the doctor or patient has left the operating environment for the delay time includes: obtaining that the doctor has left the operating environment includes obtaining that the doctor has left the remote doctor console. Obtaining that the patient has left the operating environment includes obtaining that the patient has left the operating table or that the operating table carrying the patient has been removed from the operating room.

[0438] Surgical interruption due to an abnormality may include detecting an abnormal network status of the network channel, such as abnormal network connectivity or abnormal network communication quality. Furthermore, confirmation of abnormal network status may be supplemented with a judgment condition, such as the cumulative duration of the abnormal network status reaching a time threshold, or the cumulative number of abnormal network status events reaching a number threshold.

[0439] Regarding the two end devices agreeing to switch pairing, it includes either end initiating a switch pairing request to the other end, the other end accepting the switch pairing request, and the other end device can notify the end initiating the switch pairing request.

[0440] In some embodiments, the device and the second peer device satisfying the real-time pairing condition include: the second peer device and the device have established a scheduled pairing. Establishing a scheduled pairing between the two can speed up the switching of pairings.

[0441] In some embodiments, when the device is paired with any peer device in real time, it also includes establishing a network channel with the peer device. Establishing a network channel includes establishing all network channels and establishing only a target network channel based on configured control permissions. When the device is unpaired from any peer device in real time, it also includes destroying all network channels or the target network channel established with the peer device.

[0442] In some embodiments, when the device is paired with any peer device in real time, the device and the peer device may obtain configured control permissions. At least one of the device and the peer device may establish a target network channel based on the obtained configured control permissions. When the device is unpaired from any peer device in real time, the device and the peer device may release the configured control permissions.

[0443] In some embodiments, in 802, the device establishes a real-time pairing with a first peer device, and further includes establishing a first master-slave mapping relationship between the device and the first peer device. In 803, the device releases the real-time pairing with the first peer device, and further includes the device disconnecting the first master-slave mapping relationship with the peer device. When the first master-slave mapping relationship is disconnected, the master-slave control is interrupted, including the master-slave motion control or the master-slave energy control. Wherein, for safety reasons, when the master-slave motion control is interrupted, the manipulator component can be configured to maintain at least one of the current position and posture, and when the master-slave energy control is interrupted, the energy platform can be configured to reduce or shut down the energy output. In 803, the device establishes a real-time pairing with a second peer device, and further includes establishing a second master-slave mapping relationship between the device and the second peer device. Exemplarily, one of the device and the peer device includes a remote doctor console and the other includes a slave hand robot or an energy platform. The device and the peer device are generally controlled according to the master-slave mapping relationship when performing motion control or energy control. Motion control occurs between the remote doctor's console and the slave robot. The remote doctor's console can control the slave robot's motion, such as in master-slave control. The slave robot can also control the remote doctor's console's motion, such as in master-slave alignment of a laparoscopic surgical robotic system. Energy control occurs between the remote doctor's console and the energy platform. The remote doctor's console controls the energy platform's on / off functions, current regulation, voltage regulation, power regulation, or frequency regulation.

[0444] When the device is paired with the first peer device in real time, the two establish a first master-slave mapping relationship and are controlled through the first master-slave mapping relationship. However, when the device cancels the real-time pairing with the first peer device and establishes a real-time pairing with the second peer device, since the first peer device and the second peer device may be of different device types, and different device types usually correspond to different robotic arm configurations or energy output methods, etc., it may be inappropriate for the two to still be controlled through the first master-slave mapping relationship. It is necessary to re-establish a master-slave mapping relationship that is suitable for the device and the second peer device. Assuming that the reconstructed master-slave mapping relationship is the second master-slave mapping relationship, then when the device switches from real-time pairing with the first peer device to real-time pairing with the second peer device, in order to ensure the smooth implementation of remote surgery, it is necessary to switch the first master-slave mapping relationship to the second master-slave mapping relationship.

[0445] In some embodiments, the master-slave mapping relationship between the device and the peer device includes at least one of a first master-slave mapping logic and a second master-slave mapping logic. The first master-slave mapping logic includes master-slave mapping logic in which the device controls the peer device. The second master-slave mapping logic includes master-slave mapping logic in which the peer device controls the device.

[0446] In some embodiments, the master-slave mapping logic includes a master-slave motion mapping logic. Take the example of a remote doctor console and the peer device being a slave robot for illustration. Different types of slave robots usually have different robotic arm structures, and their controlled structures, such as the structures of the manipulator components used to perform surgery, are often also different. Different manipulator components may involve different connecting rod parameters. During master-slave control, different kinematic models should be determined according to the different manipulator components, and then the motion of the manipulator components should be controlled based on the corresponding kinematic models. Different types of slave robots may need to control the motion of the manipulator components according to different motion scaling ratios when executing the same posture instruction because the structures of the manipulator components may be different. Different types of slave robots often include manipulator components of different numbers or arranged in different positions. During master-slave control, one operating unit usually controls one manipulator component. When it is necessary to switch the manipulator component controlled by the operating unit, it may be necessary to switch the manipulator component according to a specific instrument switching logic, and different types of slave robots may correspond to different instrument switching logics. During master-slave control, different types of slave robots may have different instrument operation logics for controlling the movement of different medical instruments. For example, in a single-port laparoscope slave robot, two operating parts are manipulated simultaneously to generate control instructions for controlling the movement of the endoscope. In a multi-port laparoscope slave robot, one operating part is manipulated to generate control instructions for controlling the movement of the endoscope. Therefore, the master-slave motion mapping logic may include at least one of the kinematic model, motion scaling, instrument switching logic, and instrument operation logic involved in master-slave control. The kinematic model involved in master-slave control includes at least the kinematic model of the controlled structure in the slave robot, such as the manipulator assembly, and may also include the kinematic model of the robotic arm in the operating part of the remote doctor's console.

[0447] In some embodiments, the master-slave mapping logic includes a master-slave energy mapping logic. This is described using the example of a remote doctor console and an energy platform as the device. Different energy platform types may have different types of energy output forms. The remote doctor console issues energy control instructions to the energy platform, and the energy output based on the energy control instructions includes specific conversion relationships. For example, some conversion relationships include current size conversion, some conversion relationships include power size conversion, and some conversion relationships include frequency size conversion. Therefore, the master-slave energy mapping logic may include energy output conversion relationships during master-slave control, etc.

[0448] In some embodiments, such as in a laparoscopic surgical robot system, the remote physician console includes a first type remote physician console and a second type remote physician console. The first type remote physician console has an operating unit in the form of a mechanical linkage, and the second type remote physician console has an operating unit in the form of a magnetic navigation. For example, when the device refers to the remote physician console and the peer device refers to the slave robot:

[0449] If the remote doctor console is of the first type, the remote doctor console can perform master-slave (follower) control over the slave hand robot, and the slave hand robot can also control the remote doctor console for master-slave alignment. The former requires the first master-slave mapping logic, while the latter requires the second master-slave mapping logic. In other words, the master-slave mapping relationship includes the first master-slave mapping logic and the second master-slave mapping logic.

[0450] If the remote doctor console is the second type of remote doctor console, the remote doctor console can perform master-slave (follow-up) control of the slave-hand robot, but the slave-hand robot cannot control the remote doctor console for master-slave alignment. Only the former requires the first master-slave mapping logic, that is, the master-slave mapping relationship can only include the first master-slave mapping logic.

[0451] In some embodiments, the device information of the remote physician console and the patient surgical platform may include the information included in the master-slave mapping logic described above. When registering with the server, this information may be stored on the server and sent to the peer device during pairing. Furthermore, a master-slave mapping relationship is established based on the master-slave mapping logic of at least one of the two devices.

[0452] In some embodiments, in a scenario where the device is a remote doctor console and the peer device is a patient surgical platform, when the device is paired with any peer device in real time, it also includes updating the UI interface on the device's display screen. Specifically, when the device is paired with a first peer device in real time, the device's display screen displays a first UI interface; when the device switches from real-time pairing with a first peer device to real-time pairing with a second peer device, the device's display screen displays a second UI interface. The first UI interface is associated with the information of the first peer device, and the second UI interface is associated with the information of the second peer device. Different peer devices may have at least one difference in the number, type, and status of the devices, resulting in different information for different peer devices, which needs to be accurately displayed accordingly. For example, the first peer device includes a single-port laparoscope slave robot, and the second peer device includes a multi-port laparoscope slave robot. At least the change in the slave robot model and status may cause the two UI interfaces to be different.

[0453] In some embodiments, when a remote doctor console is found to have a robotic arm operating unit, when the device is paired with any peer device in real time, in order to facilitate and quickly complete preoperative preparations, one or more remote doctor consoles with operator authority can be automatically aligned as a master and a slave. Figure 39 , the master-slave alignment method includes:

[0454] 8031, the patient surgical platform sends the current posture of the medical instrument end of a manipulator component in the hand robot in the endoscope coordinate system to the remote doctor console that has operator authority over the manipulator component.

[0455] The operator authority mainly refers to the motion control authority. The transmission of the current posture can be based on the motion control network channel created by the remote doctor console and the patient surgery platform according to the operator authority.

[0456] 8032, the remote doctor console obtains the current posture of the medical instrument end in the endoscope coordinate system sent by the patient's surgical platform.

[0457] 8033. Based on the current posture of the medical device end in the endoscope coordinate system, the remote doctor console converts the current posture of the medical device end in the endoscope coordinate system into the target posture of the operating part in the display screen coordinate system in the remote doctor console.

[0458] 8034. The remote doctor console obtains target joint parameters of each joint of the robotic arm in the operating part by analyzing the target posture based on inverse kinematics.

[0459] 8035, the remote doctor console controls the movement of the corresponding joints in the robotic arm based on the target joint parameters of each joint, so that the end of the operating part reaches the target posture in the display screen coordinate system, so that the posture of the operating part is aligned with the posture of the end of the medical device, realizing master-slave alignment.

[0460] Through the above 8031~8035, the operating part and the medical device can be aligned in posture, which is beneficial for quickly completing preoperative preparations on the one hand, and is beneficial for the doctor to achieve hand-eye intuition when controlling the associated medical device by manipulating the operating part on the other hand.

[0461] <Exchange of Control Rights>

[0462] In some embodiments, in some surgical scenarios such as substitute teaching scenarios, at the same time, multiple remote doctor consoles can be paired with a patient surgical platform in real time, wherein different remote doctor consoles are configured to have different control permissions. In the substitute teaching scenario, it is assumed that there are a first remote doctor console and a second remote doctor console that are paired with the same patient surgical platform in real time, the first remote doctor console is configured to have one of operator permissions and observer permissions, and the second remote doctor console is configured to have the other of operator permissions and observer permissions. The remote doctor console with operator permissions focuses on "teaching", and the remote doctor console with observer permissions focuses on "learning"; alternatively, the remote doctor console with operator permissions focuses on "drilling", and the remote doctor console with observer permissions focuses on "guidance". In order to quickly switch between teaching and learning, or between drill and guidance, the present disclosure also provides a method for quickly switching control permissions, see. Figure 40 , the method comprising:

[0463] 901. The first remote doctor console sends a request for exchange control authority to the patient surgery platform.

[0464] The control authority exchange request is used to request an exchange of control authority with a target second remote physician console among at least one second remote physician console. The first remote physician console and the at least one second remote physician console have different control authorities. For example, the first remote physician console has a first control authority, and at least one second remote physician console has a second control authority. The plurality of second remote physician consoles may have different second control authorities.

[0465] 902. The patient surgery platform receives the exchange control authority request sent by the first remote doctor console and forwards it to the target second remote doctor console.

[0466] Since the first remote doctor console and the second remote doctor console do not communicate directly, the patient surgery platform can act as a server to transfer relevant information.

[0467] 903 , the target second remote doctor console receives the exchange control authority request and obtains a response instruction generated in response to the exchange control authority request.

[0468] The second remote doctor console may generate a notification based on the received exchange control authority request and play the notification to facilitate the doctor to respond to the exchange control authority request based on the notification.

[0469] The notification may include a notification of whether the second remote physician console accepts the exchange control permission request initiated by the first remote physician console. The notification may be played via a UI interface or an audio signal. The physician may respond to the exchange control permission request via an operable control on the UI interface or via voice recognition. The response may include acceptance or rejection.

[0470] 904 , the target second remote doctor console sends a response instruction to the patient's surgical platform.

[0471] 905. The patient surgical platform receives the response instruction. When the response instruction is an affirmative response instruction, the first control authority of the first remote doctor console is configured as the second control authority of the target second doctor console, and the second control authority of the target second remote doctor console is configured as the second control authority of the first doctor console.

[0472] The response instruction includes an affirmative response instruction or a negative response instruction. An affirmative response instruction refers to a response instruction generated by the target second remote doctor console upon receiving an acceptance request for exchange control authority. A negative response instruction refers to a response instruction generated by the target second remote doctor console upon receiving a rejection request for exchange control authority.

[0473] In addition, when the response instruction is a negative response instruction, the patient surgery platform does not reallocate the control authority of the first remote doctor console and the target second remote doctor console, and both maintain the original control authority.

[0474] In step S905, the patient operating platform typically needs to receive a positive response instruction within a preset time range before reconfiguring the control permissions of the first remote physician console and the target second remote physician console. The preset time range typically refers to the time from when the patient operating platform forwards the control permission exchange request to the target second remote physician console, for example, the preset time range is within 3 minutes.

[0475] 906 , the patient surgery platform sends the second control authority to the first remote doctor console, and sends the first control authority to the target second remote doctor console.

[0476] 907, the first remote doctor console obtains the second control authority, and the target second remote doctor console obtains the first control authority.

[0477] In some embodiments, the first remote physician console includes a first display screen, and the second remote physician console includes a second display screen. The method may include:

[0478] In response to obtaining a phased change in the surgical process, the first remote doctor console can generate a control on the display interface of the first display screen, which is used to trigger a request to exchange control permissions with the patient's surgical platform. The phased change in the surgical process may involve the need to exchange control permissions.

[0479] Then, in step 901, the first remote doctor console may, in response to detecting that the control has been triggered, send a request to exchange control permissions to the patient's surgical platform. Of course, if the first remote doctor console does not detect that the control has been triggered, it will not send a request to exchange control permissions to the patient's surgical platform and will continue to operate under the original control permissions.

[0480] In some embodiments, the phased change of the surgical procedure includes at least one of the following: a change in the target surgical object type during the surgical procedure; and a change in the target surgical action type during the surgical procedure.

[0481] Exemplarily, target surgical object types include critical and non-critical surgical object types. For example, critical surgical object types include at least one of arteries, heart, liver, spleen, kidney, stomach, lung, and gallbladder. For example, non-critical surgical object types include at least one of veins, fat, large intestine, and small intestine. Typically, different types of target surgical objects are associated with surgeons of varying experience levels. The target surgical object type can be determined by the first remote physician console identifying surgical images transmitted from the patient's surgical platform.

[0482] Exemplarily, the target surgical action types include two types: important target surgical actions and non-important target surgical actions. For example, important target surgical actions include at least one of resection and hemostasis. For example, non-important target surgical actions include at least one of suturing and drainage. Different types of target surgical actions may be associated with surgeons with different levels of experience performing the surgery. Different types of medical instruments are generally used to perform different surgical actions. The type of target surgical action can be determined by the first remote physician console identifying the type of medical instrument currently in use, as transmitted by the patient's surgical platform.

[0483] Important surgical objects or procedures typically require more experienced surgeons, while less experienced surgeons can typically perform less important surgical objects or procedures. Therefore, prompts for control authority exchange can be provided based on the stage-by-stage changes in the surgical process, allowing surgeons to determine whether control authority exchange is necessary.

[0484] In other embodiments, it is also possible to directly generate a control on the first display screen without changing according to the stages of the surgical process. The control is used for the doctor to trigger the sending of an exchange control authority request to the patient's surgical platform according to actual needs.

[0485] In some embodiments, in order to ensure that the exchange control permission request issued by the first remote doctor console is reliable, the custom protocol agreed upon between the first remote doctor console and the patient surgery platform can be used to encapsulate the request, and the request can be decapsulated on the patient surgery platform side to achieve authentication, which will not be repeated here.

[0486] In some embodiments, a prompt may be generated at the first remote doctor console or the target second remote doctor console based on the situation of the control authority exchange. When the control authority exchange is successful, a prompt is generated at both ends indicating that the authority exchange is successful. When the control authority is maintained, a prompt is generated at both ends indicating that the authority has not been exchanged.

[0487] In some embodiments, the two remote doctor consoles that can exchange control permissions are usually two remote doctor consoles of the same device type, or two remote doctor consoles with the same device functions. In some embodiments, the method may include allowing or prohibiting any remote doctor console from initiating a request for exchange control permissions based on identifying the device type or device function of the two remote doctor consoles that are paired with the patient's surgical platform in real time. For example, when the device types of the two remote doctor consoles do not match and / or the device functions do not match, the function of any remote doctor console initiating a request for exchange control permissions is prohibited; when the device types of the two remote doctor consoles match or the device functions match, the function of any remote doctor console initiating a request for exchange control permissions is allowed. Based on such results, different configuration interfaces can be provided for the two remote doctor consoles. When allowed, a configuration interface including controls that can trigger a request for exchange control permissions is provided; when prohibited, a configuration interface that does not include controls that can trigger a request for exchange control permissions is provided.

[0488] Among them, when the control authority is released, the remote doctor console with operator authority disconnects the master-slave control, including disconnecting the master-slave motion control or the master-slave energy control. When disconnecting the master-slave motion control, the manipulator component can be configured to maintain at least one of the current position and posture. When disconnecting the master-slave energy control, the energy platform can be configured to reduce or shut down the energy output to ensure safety.

[0489] In this embodiment, the first remote doctor console and the second remote doctor console can remain paired with the patient's surgical platform without the need for unpairing, and can seamlessly exchange control permissions throughout the entire surgical process, thereby improving the rapid switching between teaching and learning, or rehearsal and guidance modes, and improving surgical efficiency.

[0490] In some embodiments, a first remote doctor console can also establish real-time pairing with multiple patient surgical platforms to watch the surgical procedures of different patient surgical platforms, or to guide the surgical procedures of different patient surgical platforms. However, at the same time, the first remote doctor console can only obtain operator authority for one of the patient surgical platforms, and the first remote doctor console can simultaneously watch multiple videos sent by different patient surgical platforms on its display screen. At the same time, these patient surgical platforms can also establish real-time pairing with multiple second remote doctor consoles. Similarly, a second remote doctor console can only obtain operator authority for one patient surgical platform. When the first remote doctor console and the second remote doctor console have different control permissions for the same patient surgical platform, such as one with operator authority and the other with observer authority, the two can implement the switching of control permissions based on the records of the aforementioned embodiments, which will not be repeated here.

[0491] In the above embodiment, based on the change in the pairing relationship or the change in control authority, at least one of establishing a network channel between the two ends of the pairing, establishing a master-slave mapping relationship, detecting a successful pairing, and updating the UI interface can be performed. For the sake of brevity, please refer to the previous description and will not be repeated here.

[0492] The present disclosure also provides a computer-readable storage medium having instructions stored therein, and when the instructions are run on at least one processor, the steps in the above-mentioned various method embodiments can be implemented. The memory 503 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. The processor may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present disclosure, or a graphics processing unit GPU (Graphics Processing Unit). The one or more processors included in the control device may be processors of the same type, such as one or more CPUs, or one or more GPUs; or they may be processors of different types, such as one or more CPUs and one or more GPUs.

[0493] The present disclosure also provides a computer program product, which, when executed on a device, enables the device to implement the steps in the above-mentioned various method embodiments.

[0494] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0495] The above-described embodiments merely represent several implementation methods of the present disclosure. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person of ordinary skill in the art could make various modifications and improvements without departing from the spirit of the present disclosure, all of which fall within the scope of protection of the present disclosure. Therefore, the scope of protection of the present patent shall be determined by the appended claims.

Claims

1. A method for interconnecting devices, characterized in that: The method is applied to a server in a remote surgical robot system, the system comprising the server, and a first device and a second device capable of communicating with the server, the first device comprising a remote doctor console, the second device comprising a patient surgical platform, and the server comprising a display screen. The method comprises: Displaying the acquired first device information of the first device registered with the server in a configuration interface of the display screen; Acquire target first device information associated with a target first device determined based on the first device information, where the target first device is used to pair with the second device; Matching, based on the target first device information, second device information of a pairable second device registered with the server and capable of being paired with the target first device, and displaying the second device information on the configuration interface; Acquire target second device information associated with the target second device determined based on the second device information; Acquire device network information of at least one of the target first device and the target second device; and Sending the at least one device network information to a peer device associated with the at least one device network information, so that the target first device and the target second device can establish pairing; Obtaining permissions possessed and permissions not possessed by the remote doctor console when it is configured as an operator; Obtaining a first control instruction and a second control instruction sent by the remote physician console, where the first control instruction is a control instruction associated with a controlled object in a preset control instruction set, and the second control instruction is a control instruction associated with an uncontrolled object in the preset control instruction set, the controlled object being determined by the remote physician console from the patient surgical platform based on the authority attribution of the possessed authority, and the uncontrolled object being determined by the remote physician console from the patient surgical platform based on the authority attribution of the unpossessed authority; sending the first control instruction and the second control instruction to the patient surgical platform; When it is detected that the controlled object executes the first control instruction and the state of the uncontrolled object does not change, it is determined that the possessing permission and the not possessing permission are configured successfully; When the possessed permission and the non-possessed permission are configured successfully, it is determined that the pairing between the remote doctor console and the patient surgery platform is successful to achieve direct communication.

2. The method according to claim 1, characterized in that The first device information is displayed on the configuration interface via a list control, and different pieces of the first device information are associated with different list options of the list control; The acquiring target first device information associated with the target first device determined based on the first device information includes: In response to detecting that a target list option in the list options is selected, the first device information of the first device associated with the target list option is determined as the target first device information of the target first device for pairing.

3. The method according to claim 2, characterized in that The step of matching the second device information of a pairable second device registered with the server and capable of being paired with the target first device based on the target first device information, and displaying the second device information on the configuration interface includes: In response to detecting that a target list option in the list options is selected, based on the target first device information associated with the target list option, the second device information of the pairable second device that can be paired with the target first device is matched from the second devices registered with the server, and displayed in the configuration interface through another list control, and different second device information is associated with different list options of the list control.

4. The method according to claim 1, wherein The step of acquiring target second device information associated with the target second device determined based on the second device information includes: In response to detecting that a target list option in another list option is selected, the second device information of the second device associated with the target list option is determined to be the target second device information of the target second device desired to be paired.

5. The method according to claim 1, characterized in that The first device information is displayed on the configuration interface via a first drag control, different first device information is associated with different first drag controls, and the configuration interface includes a window control; and obtaining target first device information associated with the target first device determined based on the first device information includes: In response to detecting that a target first drag control in the first drag control moves into the window of the window control, first device information of the first device associated with the target first drag control is determined as target first device information of the target first device for pairing.

6. The method according to claim 5, characterized in that The step of matching the second device information of a pairable second device registered with the server and capable of being paired with the target first device based on the target first device information, and displaying the second device information on the configuration interface includes: In response to detecting that the target first drag control in the first drag control is moved into the window of the window control, based on the target first device information associated with the target first drag control, the second device information of the pairable second device that can be paired with the target first device is matched from the second device registered with the server, and is displayed in the configuration interface outside the window through the second drag control, and different second device information is associated with different second drag controls.

7. The method according to claim 6, characterized in that The step of acquiring target second device information associated with the target second device determined based on the second device information includes: In response to detecting that a target second drag control in the second drag control moves into the window, determining that the second device information associated with the target second drag control is the target second device information of the target second device to be paired.

8. The method according to claim 5, characterized in that The window control includes a first area and a second area, the first area includes a first sub-area and a second sub-area, the first sub-area and the second sub-area are used to accommodate the first drag control, and the second area is used to accommodate the second drag control, the first sub-area and the second sub-area are associated with different control permissions or different pairing orders, and when the first drag control is associated with the remote doctor console and the second drag control is associated with the patient surgical platform, the method further includes: In response to the second drag control moving to the second area, one first drag control moving to the first sub-area, and another first drag control moving to the second sub-area, the remote doctor console associated with different first drag controls is configured to have different control permissions or pairing orders for the patient surgical platform associated with the second drag control.

9. The method according to claim 1, characterized in that The step of acquiring device network information of at least one of the target first device and the target second device includes: In response to triggering of an event, device network information of at least one of the target first device and the target second device is obtained, and the event includes a timing from determining the target second device information reaching a delay time, or a control in the configuration interface is triggered.

10. The method according to claim 1, characterized in that When the target first device includes one and the target second device includes at least one, the pairable second device includes the second device whose custom pairing protocol is compatible with the custom pairing protocol supported by the target first device; or When the target first device includes at least one and the target second device includes one, the pairable second device includes the second device that supports a custom pairing protocol and is compatible with the custom pairing protocol supported by each of the target first devices.

11. The method according to claim 1, wherein The method further comprises: receiving a registration request sent by a corresponding device, and authenticating the registration request based on a judgment rule agreed upon between the server and the corresponding device, wherein the corresponding device includes the first device or the second device, and the judgment rule includes at least one of a custom protocol or an encryption algorithm agreed upon between the server and the corresponding device; When the authentication is successful, the registration of the corresponding device is accepted and the registration information of the corresponding device included in the registration request is stored.

12. A remote surgical robot system, characterized in that: The system includes the server, and a first device and a second device that can communicate with the server, the first device includes one of a remote doctor console and a patient surgery platform that can be manipulated by the remote doctor console, and the second device includes the other of the remote doctor console and the patient surgery platform. The server is configured to implement the method described in any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on at least one processor, the method according to any one of claims 1 to 11 is implemented.

Images