Remote surgical robot system and method for device interconnection therein

Abstract

The present disclosure relates to a remote surgical robot system and a method for device interconnection therein. The method comprises: receiving second device information, which is sent by a server, of a pairable second device; acquiring target second device information, which is determined on the basis of the second device information, of a target second device that is expected to be paired, and sending a first pairing request to the server; receiving device network information, which is sent by the server when detecting that the first pairing request matches a second pairing request sent by the target second device, of the target second device; and establishing pairing with the target second device on the basis of the device network information, thereby implementing direct communication with the target second device. When the server detects that a pairing request initiated by a remote surgeon console matches a pairing request initiated by a patient surgery platform, device network information of a peer device is sent to at least one of the two for establishing the pairing, thereby avoiding incorrect pairing, ensuring the reliability of establishing the pairing, and improving the safety of remote surgery.

Description

Remote surgery robot system and method for device interconnection thereof

[0001] This application claims priority to the Chinese patent application No.CN202411337998.4,CN20241133803.0,CN202411338072.7,CN202411338089.2,CN202411338129.3,CN202411338155.6,CN202411338180.4,CN202411338203.1 filed on September 25,2024 with the China Patent Office and entitled "Remote surgery robot system and method for device interconnection thereof",the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present disclosure relates to the technical field of surgery robots, and in particular, to a remote surgery robot system and a method for device interconnection thereof. BACKGROUND

[0003] In recent years, surgery robot technology has developed rapidly, resulting in surgery robots for different application scenarios, such as orthopedic surgery robots, interventional surgery robots, and endoscopic surgery robots. For doctors, these robots have the advantages of easy operation and high operation precision. For patients, these robots have the advantages of small trauma, light pain, and fast recovery, and are widely accepted by doctors and patients.

[0004] A surgery robot includes a doctor console and a patient surgery platform. The doctor console includes a display screen and an operation part, and the patient surgery platform includes one or more manipulator assemblies. Each manipulator assembly includes a manipulator and a medical instrument detachably assembled to the manipulator. In use, a doctor generates a control instruction by manipulating the operation part under the surgery field displayed by the display screen and provided by an imaging device in the medical instrument, and the manipulator drives the medical instrument to perform a surgery action based on the control instruction.

[0005] With the maturity of high-speed communication network technologies such as 5G and the Internet, which have low latency and high bandwidth, remote surgery robot technology combining the advantages of surgery robot technology and high-speed communication network technology has emerged. Remote surgery robot technology allows doctors to be far away from patients, for example, allowing doctors to perform super-remote surgery on patients in different countries or different provinces and cities, thereby maximizing the breakthrough of space constraints on surgery implementation and effectively alleviating the problem of uneven distribution of medical resources.

[0006] The remote surgery robot system includes a remote doctor console and a patient surgery platform, which are usually deployed in different countries, different provinces or cities, or different districts of the same city. Before implementing the super remote surgery, it is necessary to establish the communication between the remote doctor console and the patient surgery platform, especially when at least one of the remote doctor console and the patient surgery platform is deployed with a plurality of devices. How to conveniently and reliably establish the communication between the target remote doctor console and the target patient surgery platform becomes a technical problem to be solved. SUMMARY

[0007] Therefore, it is necessary to provide a remote surgery robot system and a method for device interconnection, which can conveniently and reliably establish the pairing between the remote doctor console and the patient surgery platform.

[0008] In one aspect, the present disclosure provides a method for device interconnection, which is applied to a first device in a remote surgery robot system, the system including a server, and the first device and a second device which can communicate with the server, the first device including one of a remote doctor console and a patient surgery platform which can be manipulated by the remote doctor console, and the second device including the other of the remote doctor console and the patient surgery platform, the method including:

[0009] receiving second device information of a pairable second device sent by the server, the pairable second device including a second device registered to the server and agreeing with the first device on a matching custom protocol;

[0010] obtaining target second device information of a target pairable second device determined based on the second device information, and sending a first pairing request to the server, the first pairing request including first device information of the first device and the target second device information, and the first pairing request being used to request to establish pairing with the target second device;

[0011] receiving device network information of the target second device sent by the server when detecting that the first pairing request and a second pairing request sent by the target second device are matched, the second pairing request including target second device information of the target second device and target first device information of a target pairable first device determined by the target second device based on received first device information of a pairable first device sent by the server, the second pairing request being used to request to establish pairing with the target first device, and the pairable first device including a first device registered to the server and agreeing with the target second device on a matching custom protocol; and

[0012] establish a pairing with the target second device based on the device network information to enable direct communication with the target second device.

[0013] The method further includes, after the step of establishing the pairing with the target second device based on the device network information, establishing a master-slave mapping relationship between the first device and the target second device, the master-slave mapping relationship including at least one of kinematics models, motion scaling ratios, instrument switching logic, and instrument operation logic involved when the first device and the target second device perform master-slave control.

[0014] The detecting that the first pairing request and the second pairing request sent by the target second device match includes detecting that the first device information included in the first pairing request matches the target first device information included in the second pairing request.

[0015] Before the receiving of the second device information of the second device that can be paired sent by the server, the method includes sending a first request to the server, the first request being used to request the server to send the second device information of the second device that can be paired, the first request including a first screening condition, so that the server matches the second device information of the second device that can be paired satisfying the first screening condition from the second devices registered to the server, wherein the first screening condition is associated with the first device and the second device, and the satisfaction of the first screening condition includes at least one of the following cases: a pairing custom protocol supported by the second device matches a pairing custom protocol supported by the first device; a device function supported by the second device matches a device function supported by the first device; a surgery type supported by the second device matches a surgery type supported by the first device; a device type of the second device matches a device type of the first device; and a planned surgery time of the second device does not conflict with a pairing time of the first device.

[0016] The method further includes: before receiving the second device information of the second device that can be paired sent by the server, sending a first request to the server, the first request being used to request the server to send the second device information of the second device that can be paired, the first request including a second screening condition, so that the server matches the second device information of the second device that can be paired from the second devices registered to the server according to the second screening condition, wherein the second screening condition is associated with the second device, and the second screening condition is satisfied at least one of the following conditions: an online state of the second device satisfies an expected online state; a network state of the second device satisfies an expected network state; a pairing state of the second device satisfies an expected pairing state; a device positioning of the second device satisfies an expected device positioning; a control authority allocation state of the second device satisfies an expected control authority; and a power-on self-test state of the second device satisfies an expected power-on self-test state, and the power-on self-test state is associated with patient surgery information.

[0017] The device network information includes information associated with a target network channel, the target network channel being a network channel for transmitting corresponding network information, which is expected to be established by the first device and the target second device, the target network channel being determined based on a control authority configured for the first device or based on input medical and patient information, and the pairing with the target second device based on the device network information includes establishing a target network channel between the first device and the target second device based on the information associated with the target network channel.

[0018] The method further includes: based on the detected communication state of the target network channel, determining whether the pairing with the target second device is successful, wherein when the communication state is normal, it is determined that the pairing with the target second device is successful, and when the communication state is abnormal, it is determined that the pairing with the target second device fails.

[0019] The first device is configured with an operator authority for the target second device, and the method further includes: based on whether the detected operator authority is successfully configured, determining whether the pairing with the target second device is successful, wherein when the operator authority is successfully configured, it is determined that the pairing with the target second device is successful, and when the operator authority is not successfully configured, it is determined that the pairing with the target second device fails.

[0020] The operation permission is successfully configured includes that the possession permission in the operation permission is successfully configured and the non-possession permission in the operation permission is successfully configured.

[0021] The first device is the remote physician console, the remote physician console includes an operation part with a mechanical arm, the second device is the patient surgery platform, the patient surgery platform includes a slave robot, the slave robot includes a plurality of manipulator assemblies, one of the manipulator assemblies includes a medical instrument, another of the manipulator assemblies includes a endoscope, after the step of establishing the pairing with the target second device based on the device network information, the method further includes: acquiring a current pose of a tip of the medical instrument in an endoscope coordinate system of the endoscope; converting the current pose into a target pose of the operation part in a display screen coordinate system of a display screen in the remote physician console; and controlling movement of the mechanical arm based on the target pose, so that the pose of the operation part is aligned with the pose of the tip of the medical instrument.

[0022] In another aspect, the disclosure also provides a device interconnection method, which is applied to a server in a remote surgery robot system, the system including the server, and a first device and a second device that can communicate with the server, the first device including one of a remote physician console and a patient surgery platform that can be manipulated by the remote physician console, the second device including the other of the remote physician console and the patient surgery platform, the method including:

[0023] receiving a first pairing request sent by the first device, the first pairing request including first device information of the first device, and target second device information associated with a target second device determined by the first device based on received second device information of a pairable second device sent by the server, the first pairing request being used to request to establish pairing with the target second device, the pairable second device including a second device registered to the server and having a matched custom protocol agreed with the first device;

[0024] receiving a second pairing request sent by the target second device, the second pairing request including second device information of the target second device, and target first device information associated with a target first device determined by the target second device based on received first device information of a pairable first device sent by the server, the second pairing request being used to request to establish pairing with the target first device, the pairable first device including the first device registered to the server and having a same communication protocol with the server as a communication protocol used by the second device to communicate with the server;

[0025] Upon detecting that the first pairing request matches the second pairing request, device network information of a peer device is sent to at least one of the first device and the target second device for the first device and the target second device to establish pairing, and thereby direct communication between the first device and the target second device.

[0026] In another aspect, the disclosure also provides a tele-surgery robotic system, which includes a server, and a first device and a second device in communication with the server, the first device including one of a remote physician console and a patient surgery platform manipulatable by the remote physician console, the second device including the other of the remote physician console and the patient surgery platform, the first device being configured to implement the method according to any one of the above embodiments.

[0027] In another aspect, the disclosure also provides a tele-surgery robotic system, which includes a server, and a first device and a second device in communication with the server, the first device including one of a remote physician console and a patient surgery platform manipulatable by the remote physician console, the second device including the other of the remote physician console and the patient surgery platform, the server being configured to implement the method according to any one of the above embodiments.

[0028] In another aspect, the disclosure also provides a computer readable storage medium having instructions stored therein, which when executed on at least one processor implement the method according to any one of the above embodiments.

[0029] The tele-surgery robotic system and the method of interconnecting devices thereof according to the disclosure have the following advantages:

[0030] When the pairing request initiated by the remote physician console matches the pairing request initiated by the patient surgery platform as detected by the server, device network information of a peer device is sent to at least one of the two to establish pairing, which can avoid incorrect pairing, ensure reliability of pairing, and improve safety of tele-surgery. BRIEF DESCRIPTION OF DRAWINGS

[0031] Fig. 1 is a structural schematic diagram of an embodiment of a master-slave surgery robot suitable for a tele-surgery robotic system according to the disclosure;

[0032] Fig. 2 is a structural schematic diagram of an embodiment of a surgical instrument suitable for the master-slave surgery robot shown in Fig. 1;

[0033] Fig. 3 is a structural schematic diagram of another embodiment of a slave robot suitable for the master-slave surgery robot of the tele-surgery robotic system according to the disclosure;

[0034] FIG. 4 is a structural schematic diagram of another embodiment of the master-slave surgical robot of the remote surgical robot system of the present disclosure;

[0035] FIG. 5 is a network topology diagram of an embodiment of the remote surgical robot system of the present disclosure;

[0036] FIG. 6 is a schematic diagram of a pairing scenario of the remote surgical robot system of the present disclosure;

[0037] FIG. 7 is a flowchart of an embodiment of a device registration method of the present disclosure;

[0038] FIG. 8 is a structural schematic diagram of an embodiment of a custom protocol for communication between devices of the present disclosure;

[0039] FIG. 9 is a flowchart of an embodiment of a device interconnection method of the present disclosure;

[0040] FIG. 10 is a flowchart of another embodiment of a device interconnection method of the present disclosure;

[0041] FIG. 11 is a flowchart of another embodiment of a device interconnection method of the present disclosure;

[0042] FIG. 12 is a flowchart of another embodiment of a device interconnection method of the present disclosure;

[0043] FIGS. 13-15 are schematic diagrams of an interactive interface of an embodiment of a device interconnection method of the present disclosure;

[0044] FIGS. 16-19 are schematic diagrams of an interactive interface of another embodiment of a device interconnection method of the present disclosure;

[0045] FIGS. 20-22 are schematic diagrams of an interactive interface of another embodiment of a device interconnection method of the present disclosure;

[0046] FIG. 23 is a flowchart of another embodiment of a device interconnection method of the present disclosure;

[0047] FIG. 24 is a flowchart of another embodiment of a device interconnection method of the present disclosure;

[0048] FIGS. 25-28 are schematic diagrams of an interactive interface of an embodiment of a device interconnection method of the present disclosure;

[0049] FIGS. 29-31 are schematic diagrams of an interactive interface of another embodiment of a device interconnection method of the present disclosure;

[0050] FIGS. 32-34 are schematic diagrams of an interactive interface of another embodiment of a device interconnection method of the present disclosure;

[0051] FIG. 35 is a flowchart of another embodiment of a device interconnection method of the present disclosure;

[0052] FIG. 36 is a flowchart of another embodiment of a device interconnection method of the present disclosure;

[0053] FIG. 37 is a flow chart of another embodiment of a method of device interconnection of the present disclosure;

[0054] FIG. 38 is a flow chart of another embodiment of a method of device interconnection of the present disclosure;

[0055] FIG. 39 is a flow chart of another embodiment of a method of device interconnection of the present disclosure;

[0056] FIG. 40 is a flow chart of another embodiment of a method of device interconnection of the present disclosure. DETAILED DESCRIPTION

[0057] For the purpose of promoting an understanding of the disclosure, the present disclosure will be described in greater detail below with reference to the drawings. Preferred embodiments of the present disclosure are shown in the drawings. However, the present disclosure can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and fully convey the scope of the disclosure to those skilled in the art.

[0058] It should be noted that when an element is referred to as being "on" another element, it can be directly on the other element or intervening elements can also be present. When an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements can also be present. The term "vertical", "horizontal", "left", "right", and similar terms as used herein are for the purpose of illustration only and do not indicate a unique orientation. The terms "proximal" and "distal" as used herein are terms of orientation in the medical device arts, in which "proximal" means closer to the operator during a procedure and "distal" means farther from the operator during a procedure. The terms "first", "second", and the like as used herein refer to one of two or more elements having common characteristics.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used in the disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure, the term "and / or" includes any and all combinations of one or more of the associated listed items. As used in this disclosure, the term "plurality" includes two or more.

[0060] The tele-surgery robotic system of the present disclosure comprises a master-slave surgery robot suitable for performing super-tele-surgery. The master-slave surgery robot comprises a remote physician console and a patient surgery platform. The patient surgery platform can comprise different types of slave robot arms, including but not limited to single-port laparoscopic slave robot arms, multi-port laparoscopic slave robot arms, bronchoscopic interventional slave robot arms, vascular interventional slave robot arms, and orthopedic slave robot arms, which can have different structural characteristics and can be the same or different in the applicability of surgery types.

[0061] Exemplarily, in one master-slave surgery robot shown in FIG. 1, a remote physician console 100 and a patient surgery platform 200 comprising a single-port laparoscopic slave robot arm are included. The remote physician console 100 is configured to send control commands to the patient surgery platform 200 according to the operation of a physician to control the patient surgery platform 200, and is also configured to display images acquired by the patient surgery platform 200. The patient surgery platform 200 is configured to respond to the control commands sent by the remote physician console 100 and perform corresponding operations, and is also configured to acquire images in the body.

[0062] The patient surgery platform 200 comprises a mechanical arm 210 and a manipulator assembly disposed on the mechanical arm 210. The manipulator assembly comprises a manipulator 220 disposed on the mechanical arm 210 and a surgical instrument 230 disposed on the manipulator 220 as shown in FIG. 2. The patient surgery platform 200 further comprises a puncture device 240 sleeved on a long shaft 231 of the surgical instrument 230. When responding to the control commands of the remote physician console 100, the mechanical arm 210 is configured to adjust the pose of the surgical instrument 230, the manipulator 220 is configured to drive the surgical instrument 230 to perform corresponding operations, and the end effector 232 of the surgical instrument 230 is configured to extend into the body and perform surgical operations and / or acquire images in the body through the end effector located at the distal end.

[0063] FIG. 3 schematically shows another patient surgery platform comprising a multi-port laparoscopic slave robot arm. The patient surgery platform 200' comprises a mechanical arm and a plurality of manipulator assemblies 230' disposed on the mechanical arm, the mechanical arm comprising a main arm 210' and a plurality of adjustment arms 220' disposed on the main arm 210' and orienting a platform 215', and the different manipulator assemblies 230' are disposed on different adjustment arms 220'. The main arm 210' can adjust the pose of the adjustment arms 220' and the manipulator assemblies 230', and the adjustment arms 220' can adjust the pose of the manipulator assemblies 230'. The manipulator assembly 230' comprises a manipulator 240' and a medical instrument 250' detachably mounted on the manipulator 240'.

[0064] The manipulator assembly 230' includes a parallelogram mechanism, which utilizes the parallelogram principle to define a remote center (RC) around which the manipulator 240' can perform rotational movement. A plurality of medical instruments 250' can be inserted into a patient's body through different puncture devices 500' respectively. The remote physician console 100 shown in FIG. 1 can also be used to manipulate the movement of the manipulator assembly 230' in the patient surgery platform 200' shown in FIG. 3.

[0065] For example, in an interventional surgery robot 300 shown in FIG. 4, which is also a natural orifice surgery robot, the remote physician console includes a handle 310' and an image cart 330 connected to each other, and / or the patient surgery platform 320 also includes an image cart 330. The patient surgery platform 320 is connected to a catheter instrument 340, a sensor system 350, a control system 360 for controlling the catheter instrument 340, the sensor system 350, and the image cart 330, and the like. When a physician performs various procedures on a patient next to the patient surgery platform 320, the physician can trigger a control instruction by operating the handle 310', and send the control instruction to the patient surgery platform 320 for driving, so as to control the catheter instrument 340 to advance, retract, and bend, and the like.

[0066] The patient surgery platform 320 can be moved next to a surgery bed for connecting the catheter instrument 340, and controlling the catheter instrument 340 to ascend or descend in a vertical direction, or to translate in a horizontal direction, or to move in a non-vertical and non-horizontal direction under a control instruction, so as to provide a better preoperative preparation angle for the operation of the catheter instrument 340. The control instruction can be triggered by the physician operating the patient surgery platform 320, or by the physician directly clicking or pressing a button provided on the patient surgery platform 320. Of course, in other embodiments, the control instruction can also be a voice control or a force feedback mechanism triggered instruction.

[0067] As shown in FIG. 4, further, the patient surgery platform 320 can include a base 321, a sliding seat 322 that can move up and down along the base 321, and two mechanical arms 323 fixedly connected with the sliding seat 322. The mechanical arm 323 can include a plurality of arm segments coupled at joints, which provide the mechanical arm 323 with a plurality of degrees of freedom, for example, seven degrees of freedom corresponding to seven arm segments. The end of the mechanical arm 323 is provided with a manipulator (not shown in the figure), which is used to engage the catheter instrument 340 and control the end of the catheter instrument 340 to bend and turn correspondingly under the driving action of the manipulator. Among them, the two mechanical arms 323 can be structures that are completely or partially the same, one mechanical arm 323 is used to engage the inner catheter instrument 341, and the other mechanical arm 323 is used to engage the outer catheter instrument 342. When installed, the outer catheter instrument 342 can be installed first, and after the outer catheter instrument 342 is installed, the catheter of the inner catheter instrument 341 is inserted into the catheter of the outer catheter instrument 342.

[0068] The sensor system 350 has one or more subsystems for receiving information about the catheter instrument 340. The subsystems can include a position sensor system, a shape sensor system for determining the position, orientation, velocity, speed, pose, and / or shape of the end of the catheter instrument 340 and / or along one or more segments of the catheter that can constitute the catheter instrument 340, and / or a visualization system for capturing images from the end of the catheter instrument 340.

[0069] The image car 330 can be provided with a display system 331 and a flushing system (not shown in the figure) and the like. The display system 331 is used to display images or representations of the surgical site and the catheter instrument 340 generated by the subsystems of the sensor system 350. Real-time images of the surgical site and the catheter instrument 340 captured by the visualization system can also be displayed. Image data from imaging techniques such as computed tomography (CT), magnetic resonance imaging (MRI), optical coherence tomography (OCT), and ultrasound, etc. can also be used to present images of the preoperative or intraoperative recorded surgical site.

[0070] Among them, the preoperative or intraoperative image data can be presented as two-dimensional, three-dimensional or four-dimensional (such as time-based or speed-based information) images and / or as images from models created according to preoperative or intraoperative image data sets, and virtual navigation images can also be displayed. In the virtual navigation image, the actual position of the catheter instrument 340 is registered with the preoperative image to present the virtual image of the catheter instrument 340 within the surgical site to the operator from the outside.

[0071] The control system 360 includes at least one memory and at least one processor. It can be appreciated that the control system 360 can be integrated in the patient surgery platform 320 or the imaging cart 330, or can be independently arranged. The control system 360 can support wireless communication protocols such as IEEE 802.11, IrDA, Bluetooth, HomeRF, DECT, and wireless telemetry, etc. The control system 360 can transmit one or more signals instructing the catheter instrument 340 to move, which are generated by the manipulator moving the catheter instrument 340. The catheter instrument 340 can extend to the surgical position in the body via the opening of the natural cavity of the patient or the surgical incision.

[0072] Further, the control system 360 can include a mechanical control system (not shown in the figure) for controlling the movement of the catheter instrument 340, and thus can be integrated in the patient surgery platform 320. The control system 360 can also include an image processing system (not shown in the figure) for virtual navigation path planning, and thus can be integrated in the imaging cart 330. Of course, the various subsystems of the control system 360 are not limited to the specific cases listed above, and can be reasonably arranged according to actual conditions.

[0073] The image processing system can use the above imaging techniques to image the surgical site based on the images of the surgical site recorded before or during the surgery. Software that can be used in combination with manual input can also be used to convert the recorded images into two-dimensional or three-dimensional composite images of part or the entire anatomical organ or segment. During the virtual navigation procedure, the sensor system 350 can be used to calculate the position of the catheter instrument 340 relative to the patient's anatomical structure, which can be used to generate external tracking images and internal virtual images of the patient's anatomical structure, to achieve registration of the actual position of the catheter instrument 340 with the preoperative images, so that the operator can be presented with a virtual image of the catheter instrument 340 inside the surgical site from the outside.

[0074] The internal catheter instrument 341 and the external catheter instrument 342 have substantially the same structure, each having an elongated flexible internal catheter 41 and an external catheter 42, wherein the diameter of the external catheter 42 is slightly larger than that of the internal catheter 41, so that the internal catheter 41 can pass through the external catheter 42 and provide certain support for the internal catheter 41, so that the internal catheter 41 can reach the target position in the patient's body to facilitate tissue or cell sampling operations at the target position.

[0075] Certain movements of the handle 310 can cause corresponding movements of the catheter instrument 340. For example, when a physician moves the direction dial of the handle 310 up or down, the movement of the direction dial of the handle 310 can be mapped to a corresponding pitch movement of the tip of the catheter instrument 340; when the physician moves the direction dial of the handle 310 left or right, the movement of the direction dial of the handle 310 can be mapped to a corresponding yaw movement of the tip of the catheter instrument 340. In the present embodiment, the handle 310 can control the tip of the catheter instrument 340 to move in a 360° spatial range.

[0076] The teleoperation robotic system of the present disclosure also includes a server. As shown in FIG. 5, the remote physician consoles Al-A3 and the patient surgery platforms Bl-B3 of the same or different master-slave surgery robots are connected to the server S respectively to communicate with the server S. The server S can be used to manage the information sent by the remote physician consoles Al-A3 and the patient surgery platforms Bl-B3, and can be used to relay the transmission of information between the remote physician consoles Al-A3 and the patient surgery platforms Bl-B3. The server S can also be used to implement the device interconnection (i.e., pairing) between the remote physician consoles Al-A3 and the patient surgery platforms Bl-B3. Pairing includes the connection between two devices to enable communication.

[0077] In super-teleoperation, the remote physician console and the patient surgery platform are deployed at different locations respectively. For example, they can be deployed at different countries, different provinces or cities, or different districts of the same city.

[0078] The server can also be deployed at a location different from any of the remote physician console and the patient surgery platform, for example, the server, the remote physician console and the patient surgery platform can be deployed at different cities respectively. The server can also be deployed at a location same as one of the remote physician console and the patient surgery platform, for example, the server can be deployed independently of the remote physician console or the patient surgery platform located at the same location, and for another example, the server can be integrated with the remote physician console or the patient surgery platform located at the same location. It can be understood that the server can also be deployed in the cloud.

[0079] In the teleoperation robotic system of the present disclosure, at least one of the remote physician console, the patient surgery platform and the server is deployed respectively.

[0080] The remote physician console can include at least one. The patient surgery platform can also include at least one. The server can also include at least one.

[0081] The server includes one server, each remote physician console and each patient surgical platform is connected to the one server. The server includes multiple servers, the servers are connected through a network topology, different remote physician consoles and different patient surgical platforms can be connected to any same or different server. The network topology between the multiple servers includes at least one of a star topology, a ring topology, a bus topology, a tree topology, a mesh topology, a virtual local area network, and a wireless topology.

[0082] The remote physician console and the server, the patient surgical platform and the server, and the servers can be connected through the same type or different type of high-speed communication network. For example, these types of high-speed communication networks can include at least one of an Internet broadband, an Internet private network, a 5G network, and a 5G private network.

[0083] For example, the remote physician consoles A1-A3 and the patient surgical platforms B1-B3 can be paired one-to-one, one-to-many, and many-to-one under the action of the server S. The paired remote physician consoles and patient surgical platforms can communicate end-to-end without the help of the server's relay function. For example, from the perspective of the remote physician console, the pairing can be understood as shown in FIG. 6:

[0084] One-to-one pairing refers to pairing one remote physician console with one patient surgical platform. For example, for remote physician consoles A2 and A3, they are respectively paired with only one patient surgical platform, such as A2 paired with B2 or A3 paired with B1, so for A2 and A3, it is one-to-one pairing.

[0085] One-to-many pairing refers to pairing one remote physician console with multiple patient surgical platforms. For example, for remote physician console A1, it is simultaneously paired with patient surgical platforms B2 and B3, so for A1, it is one-to-many pairing.

[0086] Many-to-one pairing refers to pairing multiple remote physician consoles with one patient surgical platform. For example, for remote physician consoles A1 and A2, they are simultaneously paired with one patient surgical platform B2, so for A1 and A2, it is many-to-one pairing.

[0087] <Registration of equipment>

[0088] The types of remote physician consoles and patient surgery platforms are various, and there are many fake and inferior ones. The pairing of inappropriate remote physician consoles and patient surgery platforms can easily lead to surgical risks. In order to reduce the surgical risks that may be caused by the inappropriate pairing between the remote physician console and the patient surgery platform as much as possible, the remote physician console or the patient surgery platform that is not allowed to be paired (i.e. not allowed to be used) can be filtered by adopting the device registration manner. That is, only the remote physician console and the patient surgery platform that have completed the registration on the server are allowed to be paired.

[0089] The registration method of the remote physician console and the patient surgery platform to the server is the same. For the sake of simplifying the description, any remote physician console and any patient surgery platform can be described as a device respectively. In some embodiments, referring to FIG. 7, the method of registering the device to the server comprises:

[0090] 101. After the device establishes a connection with the server, the device sends a registration request to the server.

[0091] The process of establishing a connection between the device and the server comprises: the device sends a connection establishment request to the server; the server receives the connection establishment request sent by the device, attempts to establish a connection with the device, and returns a connection state to the device. Wherein, when the returned connection state is connection success, the device and the server establish a long connection; otherwise, the device and the server do not establish a long connection. When the returned connection state is connection failure, the connection process can be re-executed.

[0092] 102. The server receives the registration request sent by the device, and authenticates the registration request based on the judgment rule agreed between the server and the device.

[0093] 103. In response to the server determining that the authentication is successful, the server accepts the registration of the device, and returns a registration success state to the device.

[0094] The server can maintain the registration information of the registered devices by means of a registration list and the like, and the registration information comprises the device information of these devices.

[0095] 104. In response to the server determining that the authentication fails, the server rejects the registration of the device, and returns a registration failure state to the device.

[0096] In some embodiments, the agreed judgment rule in 102 includes a first judgment rule, and the first judgment rule includes a self-defined protocol agreed between the device and the server. The self-defined protocol includes a protocol header and a protocol body. The protocol header is a message header, and the protocol body is a message data part. For different self-defined protocols, the format specified by the protocol header is usually the same, but the format specified by the protocol body is usually different. The difference of the protocol body can be reflected in at least one of the number of fields, the type of fields, the arrangement order of fields, and the data exchange format. FIG. 8 illustrates a self-defined protocol based on the TCP protocol.

[0097] The registration request is a data message encapsulated according to the format specified by the self-defined protocol. After receiving the registration request, the server decapsulates the registration request according to the format specified by the self-defined protocol. If the decapsulation is successful, it can be determined that the authentication is successful; if the decapsulation fails, it can be determined that the authentication fails.

[0098] In the initial state of the device, for example, before the device is registered to the server, different devices only store the device information of the local end and the device information of the server. The registration request of the device includes the device information of the server and the device information of the local end. The device information of the server includes at least one of the IP address and the port number of the server. The device information of the local end includes device basic information and device network information. The device basic information includes at least one of the device unique identification (ID or UDID), the device name (name), the device type (type), the device manufacturer name (manufacture), the device function (function), the suitable operation type, and the device position. The device network information includes at least one of the IP address and the port number of the device.

[0099] The registration request of the device is encapsulated into a data message according to the format specified by the self-defined protocol. Taking the TCP protocol as the self-defined protocol, the port number of the server and the port number of the local end of the device are encapsulated in the protocol header, and other device information of the local end of the device is encapsulated in the protocol body. The protocol body includes a plurality of fields, and the device information of the local end of the device is stored in the corresponding field. For example, the fields include at least one of the fields of the device unique identification (ID or UDID), the device name (name), the device type (type), the device manufacturer name (manufacture), the device function (function), and the IP address. The fields of the protocol body can also include a time stamp (time) field for storing the time when the registration request of the device is sent to the server.

[0100] In some embodiments, different devices can communicate with each other through the same or different custom protocols. For example, the remote physician console and the server can communicate with each other through a first custom protocol agreed upon; the patient surgery platform and the server can communicate with each other through a second custom protocol agreed upon; and the remote physician console and the patient surgery platform can communicate with each other through a third custom protocol agreed upon. The three custom protocols can be the same or different. In some embodiments, one remote physician console can be configured to support communication with multiple patient surgery platforms through the same or different custom protocols respectively, and one patient surgery platform can be configured to support communication with multiple remote physician consoles through the same or different custom protocols respectively.

[0101] In some embodiments, for example, when the remote physician console or the patient surgery platform communicates with the server through a custom protocol agreed upon, the protocol body of the custom protocol can include at least one field for storing the custom protocol that the remote physician console or the patient surgery platform supports pairing with. In 103, when the server accepts the registration of a device, the custom protocol that the corresponding device is capable of supporting pairing with can also be stored in the server.

[0102] For example, the device functions of the fields in the protocol body are described. For the remote physician console, the device functions include the operable functions; for the patient surgical platform, the device functions include the operable functions. Different remote physician consoles or different patient surgical platforms can have the same or different device functions. The remote physician console and the patient surgical platform expected to be used in pairing need to match at least the device core functions in the device functions to be allowed to be used in pairing. For example, the device core functions include at least one of the motion control function and the energy control function, both of which are related to the device hardware conditions, and the corresponding device core functions are supported only when the corresponding device hardware conditions are met. For example, for the patient surgical platform including the slave robot and the energy platform, the patient surgical platform supports the motion control function based on the slave robot and supports the energy control function based on the energy platform; for the remote physician console including the hand operation part and the foot pedal part, the remote physician console supports the motion control function based on the hand operation part and supports the energy control function based on the foot pedal part; for the patient surgical platform, if the remote physician console cannot support the motion control function or the energy control function due to the limitation of the device hardware conditions, it can not be allowed to be paired with the patient surgical platform, or at least it is not allowed to be configured to have full operator control authority over the patient surgical platform, and at this time it can have observer authority. In some embodiments, the motion control function or the energy control function can be further classified, the device core functions correspond to more specific core functions, and the remote physician console and the patient surgical platform expected to be used in pairing need to match at least the more specific core functions to be allowed to be used in pairing, of course, this is also closely related to the device hardware conditions of the device, which will not be described here. Among them, the device functions including the device core functions under the device function field can be recorded by associated coding, for example, recorded by numbers. Among them, the device core functions and the device non-core functions in the device functions can be added with different flag records to facilitate the matching of the device core functions.

[0103] In 103, the server accepts the registration of the device, and the server stores the registration information of the device, which includes the data stored in at least part of the fields in the custom protocol body, such as one or more of the device information, the custom protocol supported by the device for pairing, etc., for subsequent use in pairing. After 103, after the server accepts the registration of the device, a login password can be assigned to the device, which is used for simple authentication of the device when it logs in to the server later, so that the server does not need to authenticate the device based on the agreed judgment rules such as the custom protocol or the encryption algorithm.

[0104] In some embodiments, the agreed judgment rule can include a second judgment rule, and the second judgment rule includes an encryption algorithm agreed between the device and the server. The registration request can be a data packet encapsulated according to a format defined by the custom protocol, or a data packet encapsulated according to a format defined by a general protocol. In some embodiments, the encryption algorithm can encrypt at least part of the data in the protocol body in the data packet to generate a first check value, and the first check value is stored in an authentication field of the protocol body. The data in the protocol body can not include the data stored in the authentication field. After the server receives the registration request, the server can obtain all the data of the data packet by decapsulating the registration request according to the format defined by the custom protocol or the general protocol. The server can generate a second check value by encrypting the data in the protocol body based on the agreed encryption algorithm. The server detects whether the first check value and the second check value are the same. If they are the same, it indicates that the data is complete and legal, and the authentication is successful. If they are different, it indicates that the data is missing or illegal, and the authentication fails.

[0105] Alternatively, when the encryption algorithm can be reversely solved, the server can also not generate the second check value, but directly decrypt the first check value stored in the authentication field obtained by decapsulating the registration request based on the reverse solution of the encryption algorithm to obtain the verification data. The server compares the verification data obtained by decryption with the corresponding at least part of the data in the protocol body. If they are the same, it indicates that the data is complete and legal, and the authentication is successful. If they are different, it indicates that the data is missing or illegal, and the authentication fails.

[0106] For example, the encryption algorithm can be a base64 encryption algorithm or a hash algorithm. The data used for encryption in the protocol body can be the data of all fields in the protocol body, or the data of the fields selected according to a certain rule. For example, the rule can be the fields selected at a mutual interval, or the first few continuous fields, or the last few continuous fields, etc. In one example, one or more of the device information associated with the device in the protocol body can be selected for encryption. For example, the unique identifier, the device name, and the timestamp in the protocol body can be selected as the data of the fields to be encrypted. In one example, the data of the selected fields can be concatenated into a string and then subjected to the encryption processing.

[0107] In some embodiments, the registration request can be subjected to double authentication in combination with the first judgment rule and the second judgment rule, which can prevent unauthorized devices from being allowed to access the remote surgery robot system described in the present disclosure, thereby ensuring the safety and reliability of the remote surgery.

[0108] In the dual authentication scenario, the registration request is usually a data packet encapsulated according to the format specified in the custom protocol. The server decapsulates the registration request based on the format specified in the custom protocol, performs the first authentication, and if the decapsulation is successful, determines that the first authentication is successful and enters the second authentication; if the decapsulation fails, it is determined that the first authentication fails and the second authentication is not entered. The server will reject the registration of the device. In the second authentication, the server generates a second check value by encrypting the data in the protocol body based on the agreed encryption algorithm. The server detects whether the first check value stored in the authentication field in the data packet corresponding to the registration request and the second check value are the same. If they are the same, it is determined that the second authentication is successful, and when the two authentications are successful, the server will accept the registration of the device; if they are different, it is determined that the second authentication fails, and the server will reject the registration of the device.

[0109] Alternatively, in the second authentication, the server can also not generate the second check value as described above, but directly decrypt the first check value stored in the authentication field obtained by decapsulating the registration request based on the inverse solution of the encryption algorithm to obtain the verification data. The server compares the decrypted verification data with at least part of the corresponding data in the protocol body. If they are the same, it is determined that the authentication is successful; if they are different, it is determined that the authentication fails.

[0110] In some embodiments, the patient surgery platform can include multiple types, and the types of data exchanged between the remote physician console and the patient surgery platform can be different when the same remote physician console manipulates different patient surgery platforms to perform the same or different types of surgery. The remote physician console can also include multiple types, and the types of data exchanged between the remote physician console and the patient surgery platform can also be different when different remote physician consoles manipulate the same patient surgery platform to perform the same or different types of surgery. Among them, the types of data exchanged between the remote physician console and the patient surgery platform can be different in the field types in the protocol body of the custom protocol. It can be understood that due to the different field types in the protocol body, the custom protocol including the protocol body is also different.

[0111] In some embodiments, the server can serve as an open platform and be provided for use by multiple remote physician consoles and multiple patient surgery platforms. The server and different devices can agree on different judgment rules for authentication during registration, and devices that can be successfully authenticated can be allowed to register and subsequently be paired for use in super-remote surgery. For example, when the judgment rule includes a custom protocol, different judgment rules correspond to different custom protocols. For another example, when the judgment rule includes an encryption algorithm, different judgment rules correspond to different encryption algorithms, wherein encrypting data corresponding to different field types using the same named encryption algorithm can be understood as different encryption algorithms.

[0112] For example, the server authenticates the registration request sent by the device using the custom protocol. Since the server does not know whether the server and the device have agreed on a custom protocol when the server receives the registration request sent by any device, and does not know which custom protocol has been agreed on, the server usually needs to call all known custom protocols respectively to try to unpack the registration request sent by the current device. If the registration request can be unpacked based on a custom protocol, it is determined that the authentication is successful; if the registration request cannot be unpacked based on any custom protocol, it is determined that the authentication fails.

[0113] For example, the server receives the registration request and tries to unpack the registration request using all known custom protocols in sequence. In some embodiments, it is assumed that the device includes a first device and a second device, the first device and the server agree on a first custom protocol as a judgment rule, the second device and the server agree on a second custom protocol as a judgment rule, and all known custom protocols include the first custom protocol and the second custom protocol. When the server receives the registration request, the server first tries to unpack the registration request received currently using the first custom protocol, and then tries to unpack the registration request received currently using the second custom protocol if the unpacking fails.

[0114] For example, the server receives the first registration request sent by the first device, and first unpacks the first registration request based on the first custom protocol. At this time, the unpacking is successful, and it is determined that the authentication is successful. For another example, the server receives the second registration request sent by the second device, and first unpacks the second registration request based on the first custom protocol. At this time, the unpacking fails, and the server then unpacks the second registration request based on the second custom protocol. At this time, the unpacking is successful, and it is determined that the authentication is successful. For another example, the server can also receive the third registration request sent by the third device. However, the third device can not have agreed on a custom protocol with the server. In this case, the server receives the third registration request sent by the third device, first unpacks the third registration request based on the first custom protocol, and then unpacks the third registration request based on the second custom protocol. At this time, the unpacking still fails, and it is determined that the authentication fails.

[0115] In some embodiments, the core devices of the patient surgery platform include slave robots, an image platform, and an energy platform. The slave robots include manipulators and medical instruments mounted on the manipulators, and a remote physician console controls the slave robots to control the medical instruments by controlling the motion of the manipulators. The image platform is used to process multimedia data, such as processing video or audio, where the video can be from an imaging device in the medical instruments, such as an endoscope, and the audio can be from the remote physician console or sound input from the slave robots, including voice intercom. The energy platform is used to control energy instruments in the medical instruments, such as electrotome, ultrasonic knife, microwave ablation knife, radiofrequency ablation knife, anastomat, plasma knife, etc. The image platform and the energy platform can be separately arranged or integrated on a movable trolley. For the convenience of description, the image platform and the energy platform can be collectively referred to as an image energy processing platform. In some embodiments, the patient surgery platform can further include auxiliary devices, such as a monitor, a breathing machine, a CT machine, an EM electromagnetic positioning system, a pneumoperitoneum machine, a surgery bed, or a shadowless lamp, etc.

[0116] Generally, the type of the tele-surgery robotic system can depend on the type of the slave robot in the master-slave surgery robot, and different slave robots usually have different structural characteristics. Various types of slave robots include, but are not limited to, single-hole laparoscopic slave robots, multi-hole laparoscopic slave robots, bronchial interventional slave robots, vascular interventional slave robots, and orthopedic slave robots.

[0117] In some embodiments, the type of the remote physician console can depend on the type of the operation part, which is used to control the action of the medical instruments in the slave robot, including motion control, clamping control, cutting control, coagulation control, suturing control, etc. The operation part includes a hand operation part, and the type of the hand operation part includes, but is not limited to, a gamepad type operation part, a mechanical linkage type operation part, a magnetic navigation type operation part, a trackball type operation part, etc.

[0118] In some embodiments, the remote physician console and the patient surgery platform that are not registered to the server or have different communication protocols are generally prohibited from being paired for use. The pairing requires the same communication protocol, which generally refers to the same custom protocol, mainly including the same format specified by the custom protocol.

[0119] In some embodiments, the remote physician console and the patient surgery platform that are registered to the server and have the same communication protocol are generally allowed to be paired for use. Without being limited to the similarity or difference of the type of the remote physician console or the patient surgery platform, when the basic conditions that the remote physician console and the patient surgery platform are registered to the server and the remote physician console and the patient surgery platform have the same communication protocol are met, the remote physician console and the patient surgery platform can be paired.

[0120] In some embodiments, a custom protocol can be agreed between a type of remote physician console and a type of patient surgery platform, such that the type of remote physician console is dedicated for pairing use with the type of patient surgery platform, i.e., the remote physician console is dedicated for manipulating a specific type of patient surgery platform.

[0121] In some embodiments, a custom protocol can be agreed between a type of remote physician console and a type of patient surgery platform, such that the type of remote physician console is dedicated for pairing use with the type of patient surgery platform, i.e., the remote physician console is dedicated for manipulating a specific type of patient surgery platform.

[0122] For example, a manufacturer A manufactures a universal remote physician console Al, and manufactures a patient surgery platform A2 with a first type of patient surgery platform, e.g., a patient surgery platform with a single-port laparoscopic teleoperational robot, and a patient surgery platform A3 with a second type of patient surgery platform, e.g., a patient surgery platform with a multi-port laparoscopic teleoperational robot, a same custom protocol can be agreed between the remote physician console Al and the patient surgery platforms A2, A3, or different custom protocols can be agreed respectively, and the remote physician console Al can be configured for manipulating one or more of the patient surgery platforms A2 and A3.

[0123] For example, a remote physician console Bl is manufactured by manufacturer B, and a patient surgery platform B2 having a first type and a patient surgery platform B3 having a second type are manufactured by manufacturer B, and a same custom protocol, such as a first custom protocol, can be agreed between the remote physician console Bl and the patient surgery platforms B2, B3. A remote physician console Cl is manufactured by manufacturer C, and a patient surgery platform C2 having a first type and a patient surgery platform C3 having a second type are manufactured by manufacturer C, and a same custom protocol, such as a second custom protocol, different from the first custom protocol, can be agreed between the remote physician console Cl and the patient surgery platforms C2, C3. The remote physician consoles Bl and Cl are, for example, both remote physician consoles having mechanical linkage type operation portions, the patient surgery platforms B2 and C2 are, for example, both single-port laparoscopic slave robots, and the patient surgery platforms B3 and C3 are, for example, both multi-port laparoscopic slave robots. The remote physician console Bl manufactured by manufacturer B can be configured to be exclusively used for manipulating one or more of the patient surgery platforms B2 and B3 manufactured by manufacturer B, and cannot be configured to be used for manipulating the patient surgery platforms C2 and C3 manufactured by manufacturer C. Meanwhile, the remote physician console Cl manufactured by manufacturer C can be configured to be exclusively used for manipulating one or more of the patient surgery platforms C2 and C3 manufactured by manufacturer C, and cannot be configured to be used for manipulating the patient surgery platforms B2 and B3 manufactured by manufacturer B.

[0124] For example, telemedicine consoles manufactured by different manufacturers can be configured to operate one or more types of patient surgical platforms manufactured by different manufacturers. Let's continue with the example of manufacturer B manufacturing a telemedicine console B1, and manufacturing a patient surgical platform B2 of a first type and a patient surgical platform B3 of a second type; manufacturer C manufacturing a telemedicine console C1, and manufacturing a patient surgical platform C2 of a first type and a patient surgical platform C3 of a second type; telemedicine consoles B1 and C1 are, for example, telemedicine consoles with mechanical linkage-type operating parts; patient surgical platforms B2 and C2 are, for example, single-port laparoscopic hand-held robots; and patient surgical platforms B3 and C3 are, for example, multi-port laparoscopic hand-held robots. This illustrates how a first custom protocol can be agreed upon between the telemedicine console B1 and the patient surgical platforms B2, B3, C2, and C3, and how this protocol can be used remotely... A second custom protocol is agreed upon between the doctor's console C1 and the patient surgical platforms B2, B3, C2, and C3. This first custom protocol may be the same as or different from the second custom protocol. The remote doctor's console B1 manufactured by manufacturer B can be configured to operate one or more of the patient surgical platforms B2 and B3 manufactured by it, and can also be configured to operate one or more of the patient surgical platforms C2 and C3 manufactured by manufacturer C. The remote doctor's console C1 manufactured by manufacturer C can be configured to operate one or more of the patient surgical platforms C2 and C3 manufactured by it, and can also be configured to operate one or more of the patient surgical platforms B2 and B3 manufactured by manufacturer B.

[0125] In some embodiments, a custom protocol may be agreed upon between a type of patient surgical platform and a type of remote doctor console. Alternatively, multiple custom protocols may be agreed upon between a type of patient surgical platform and multiple types of remote doctor consoles, so that the type of patient surgical platform can also be configured to be manipulated by one or more remote doctor consoles of that type.

[0126] For example, a manufacturer A manufactures a first type of patient surgery platform Al, and the manufacturer A also manufactures a first type of remote physician console A2 and a second type of remote physician console A3. The first type of remote physician console A2 can be, for example, a remote physician console having an operation part in the form of electromagnetic navigation, and the second type of remote physician console A3 can be, for example, a remote physician console having an operation part in the form of mechanical linkage. A same custom protocol can be agreed between the patient surgery platform Al and the remote physician consoles A2 and A3, or different custom protocols can be agreed respectively. The patient surgery platform Al can be configured to be manipulatable by one or more of the remote physician consoles A2 and A3.

[0127] For example, a manufacturer B manufactures a first type of patient surgery platform Bl, and the manufacturer B also manufactures a first type of remote physician console B2 and a second type of remote physician console B3. A same custom protocol, such as a first custom protocol, can be agreed between the patient surgery platform Bl and the remote physician consoles B2 and B3. A manufacturer C manufactures a first type of patient surgery platform Cl, and the manufacturer B also manufactures a first type of remote physician console C2 and a second type of remote physician console C3. A same custom protocol, such as a second custom protocol, can be agreed between the patient surgery platform Cl and the remote physician consoles C2 and C3, which is different from the first custom protocol. The patient surgery platforms Bl and Cl can be, for example, each a single-port laparoscopic tele-surgical robot. The remote physician consoles B2 and C2 can be, for example, each a remote physician console having an operation part in the form of electromagnetic navigation. The remote physician consoles B3 and C3 can be, for example, each a remote physician console having an operation part in the form of mechanical linkage. The patient surgery platform Bl can be configured to be manipulatable by one or more of the remote physician consoles B2 and B3. Meanwhile, the patient surgery platform Cl can be configured to be manipulatable by one or more of the remote physician consoles C2 and C3.

[0128] For example, a patient surgery platform manufactured by manufacturer B can be configured to be manipulated by one or more types of remote physician consoles manufactured by manufacturer B, and a patient surgery platform manufactured by manufacturer C can be configured to be manipulated by one or more types of remote physician consoles manufactured by manufacturer C. For example, manufacturer B manufactures a first type of patient surgery platform B1, and manufacturer B also manufactures a first type of remote physician console B2 and a second type of remote physician console B3; manufacturer C manufactures a first type of patient surgery platform C1, and manufacturer B also manufactures a first type of remote physician console C2 and a second type of remote physician console C3; patient surgery platform B1 and C1 can each be, for example, a single-port laparoscopic tele-surgical robot, remote physician console B2 and C2 can each be, for example, a remote physician console having an electromagnetic navigation form of operating portion, and remote physician console B3 and C3 can each be, for example, a remote physician console having a mechanical linkage form of operating portion. For example, a first custom protocol can be agreed between patient surgery platform B1 and remote physician consoles B2, B3, C2, C3, and a second custom protocol can be agreed between patient surgery platform C1 and remote physician consoles B2, B3, C2, C3, and the first custom protocol can be the same as or different from the second custom protocol. Patient surgery platform B1 manufactured by manufacturer B can be configured to be manipulated by one or more of remote physician consoles B2 and B3 manufactured by manufacturer B, and can also be configured to be manipulated by one or more of remote physician consoles C2 and C3 manufactured by manufacturer C. Patient surgery platform C1 manufactured by manufacturer C can be configured to be manipulated by one or more of remote physician consoles C2 and C3 manufactured by manufacturer C, and can also be configured to be manipulated by one or more of remote physician consoles B2 and B3 manufactured by manufacturer B.

[0129] It can be understood that whether a remote physician console and a patient surgery platform can be paired for use mainly depends on whether the two have been registered to the server and whether the communication protocols of the two are the same. Even if a remote physician console and a patient surgery platform are manufactured by different manufacturers, as long as the two are registered to the server and the communication protocols of the two are the same, the two can be paired for use, and at least the observer permission can be allowed to be used. The disclosure will be described later with more examples of conditions for allowing pairing.

[0130] <Device maintenance>

[0131] When a remote physician console and a patient surgery platform are successfully registered to the server, the server will store the device information of these devices. However, as the number of remote physician consoles and patient surgery platforms successfully registered to the server increases, if the server does not regularly maintain these remote physician consoles and patient surgery platforms, the existence of a large number of inactive remote physician consoles or patient surgery platforms will waste the storage space of the server and reduce the efficiency of pairing between active remote physician consoles and patient surgery platforms.

[0132] In some embodiments, the server can clear the registration information of the inactive remote physician console and patient surgery platform registered in the server according to a period, for example, the registration information of the inactive devices can be removed from the device list maintained by the server. When clearing the inactive devices, it can be detected whether the device to be cleared is currently in a pairing state, and if not, the clearing program is started; if it is currently in a pairing state, the device to be cleared can be converted to an active device, or the clearing program can be started after the pairing state of the device to be cleared is released, so as to prevent the device to be cleared from being paired for use when the clearing program is started, which may adversely interrupt the super-remote surgery and affect the implementation of the surgery.

[0133] Inactivity includes login inactivity and pairing inactivity. Login inactivity includes login frequency less than a preset login frequency or login duration less than a preset login duration within a server-set inactivity period. Pairing inactivity includes successful pairing frequency less than a preset pairing frequency or use duration after successful pairing less than a preset pairing use duration within the inactivity period. The server can clear the inactive devices, for example, once a week; the inactivity period is the time from successful registration of the device to the server, for example, 1 year; the preset login frequency is the number of times the device logs in to the server recorded by the server, for example, 1 time; the preset login duration is the cumulative time of the device logging in to the server, for example, 1 hour; the preset pairing frequency is the number of times the device establishes pairing with other devices recorded by the server, for example, 1 time; and the preset pairing use duration is the cumulative time of the device establishing pairing with other devices to releasing pairing recorded by the server, for example, 1 hour.

[0134] The server clears the registration information of the inactive remote physician console and patient surgery platform, effectively saving the resources of the server, and can improve the efficiency of pairing between the remote physician console and the patient surgery platform, in addition, it also avoids the illegal use of these inactive remote physician consoles and patient surgery platforms, affecting the safety of super-remote surgery.

[0135] <Device interconnection>

[0136] In some embodiments, the registered remote physician console and patient surgery platform can be interconnected, that is, paired. This disclosure discusses three pairing modes, and the pairing initiator is different in different pairing modes. The first mode is a double-end call mode, which means that the remote physician console and the patient surgery platform both initiate pairing. The second mode is a single-end call mode, which means that one end of the remote physician console and the patient surgery platform initiates pairing. The third mode is an administrator direct connection mode, which means that the server initiates pairing.

[0137] In the first mode and the second mode, the pairing initiator needs to log in to the server to initiate pairing. Among them, the pairing initiator can quickly log in to the server by using the login password assigned by the server when registration is completed. The pairing initiator includes at least one of the remote doctor console and the patient surgery platform. In the third mode, the pairing initiator is the server, and the user needs to log in to the server with an administrator account and password to obtain an administrator permission, which can be used to configure the judgment rule agreed with each device for the server, or can be used to directly pair the devices.

[0138] For simplicity of description, one of the remote doctor console and the patient surgery platform can be referred to as a first device, and the other can be referred to as a second device. The pairing methods in the three modes are introduced below.

[0139] <First mode - double-end mutual call>

[0140] In some embodiments, referring to FIG. 9, the method includes:

[0141] 201, the first device sends a first request to the server.

[0142] The first request is used to request the server to send second device information of a pairable second device that can be paired with the first device.

[0143] In an embodiment, the communication protocol, such as a custom protocol, agreed between all first devices and all second devices registered to the server is the same. Any first device and any second device can communicate after pairing. In this case, the first request is simply used to request the server to send the second device information of the pairable second device.

[0144] In an embodiment, the communication protocol, such as a custom protocol, agreed between at least some first devices and second devices registered to the server is different. Between the first devices and the second devices with different communication protocols, they cannot communicate after pairing. In this case, in order to improve compatibility and pairing efficiency, the server can match the custom protocol supported by the first device for pairing with the custom protocol supported by the second device for pairing based on the relevant information carried by the first request, such as the first device, and filter out the second device related to the custom protocol supported by the first device for pairing from the second device as the pairable second device, and provide the second device information of the pairable second device.

[0145] Among them, the pairable second device can be a second device that meets several conditions among the second devices. For example, the pairable second device includes an online second device, i.e., a second device logged in to the server. For example, the pairable second device includes a second device in an unpaired state.

[0146] The first request is encapsulated into a data packet in a format specified by a self-defined protocol agreed between the first device and the server and sent to the server.

[0147] 202. The server receives the first request and sends second device information of a pairable second device to the first device based on the first request.

[0148] After receiving the first request, the server decapsulates the first request based on the self-defined protocol agreed with the first device, and obtains a self-defined protocol supported by the first device for pairing. The self-defined protocol supported by the first device for pairing is used for communication with the second device and is different from the self-defined protocol agreed with the server.

[0149] The server matches a second device having the same self-defined protocol supported for pairing from the registered second devices as the pairable second device based on the obtained self-defined protocol supported by the first device for pairing, and obtains second device information associated with the pairable second device.

[0150] In an embodiment, the second device information includes at least one of a plurality of device information of the corresponding second device, and the at least one device information can uniquely represent the corresponding second device. Considering security and simplicity, the second device information can only include a unique identifier of the corresponding second device, and no device network information of the device is provided. The unique identifier can be named in combination with device location information, for example, "XX hospital XX patient operation platform", which can provide more useful information without affecting communication security.

[0151] 203. The first device receives the second device information of the pairable second device sent by the server.

[0152] After receiving the second device information, the first device can display the second device information in at least one of a plurality of forms such as a list on a display screen of the first device for the doctor to select for pairing.

[0153] 204. The first device obtains target second device information of a target second device determined based on the second device information, and sends a first pairing request to the server.

[0154] The target second device is a second device in the pairable second device that the doctor expects to pair with the first device. The first pairing request is used to request to establish pairing with the target second device, and the first pairing request includes device information of the first device and the target second device information of the target second device. In an embodiment, the device information of the first device and the target second device information can only include a unique identifier of the corresponding device.

[0155] The first pairing request is encapsulated into a data packet according to a format defined by a self-defined protocol agreed between the first device and the server, and is sent to the server.

[0156] 205. The server receives the first pairing request sent by the first device.

[0157] After receiving the first pairing request, the server can return a signaling to the first device indicating that the first pairing request is received.

[0158] 206. The target second device sends a second request to the server.

[0159] The second request is used to request the server to send first device information of the first device that can be paired with the target second device.

[0160] The first device that can be paired can include at least part of the first devices registered to the server. Also, in order to improve the pairing efficiency, the first device that can be paired can include the first devices registered to the server and having the same communication protocol as the second device. To achieve this purpose, the second request can include a self-defined protocol supported by the second device for pairing. The second request is encapsulated into a data packet according to a format defined by a self-defined protocol agreed between the target second device and the server, and is sent to the server.

[0161] 207. The server receives the second request and sends first device information of the first device that can be paired to the target second device based on the second request.

[0162] After receiving the second request, the server decapsulates the second request based on a self-defined protocol agreed with the target second device for pairing, and obtains the self-defined protocol supported by the second device for pairing.

[0163] Based on the obtained self-defined protocol supported by the second device for pairing, the server matches the first devices registered to the server to obtain the first devices having the same self-defined protocol supported for pairing as the first device that can be paired, and obtains first device information associated with the first device that can be paired.

[0164] The first device information includes at least one of a plurality of device information of the corresponding first device, for example, can only include a unique identifier of the corresponding first device, without providing device network information of the device.

[0165] 208. The target second device receives the first device information of the first device that can be paired sent by the server.

[0166] The description of 208 can refer to the description of 203, which will not be repeated here.

[0167] 209, the target second device acquires the target first device information of the target first device of the expected pairing determined based on the first device information, and sends a second pairing request to the server.

[0168] The target first device is the first device in the pairable first devices that the doctor expects to pair with the target second device. The second pairing request is used to request to establish pairing with the target first device, and the second pairing request includes the target second device information of the target second device and the target first device information of the target first device. In an embodiment, the first device information includes at least one of the various device information of the corresponding first device, and the at least one device information can uniquely represent the corresponding first device. For example, the first device information can only include the unique identifier of the corresponding device.

[0169] The second pairing request is also encapsulated into a data packet in a format specified by the custom protocol agreed by the target second device and the server and sent to the server.

[0170] 210, the server receives the second pairing request sent by the target second device.

[0171] After receiving the second pairing request, the server can return a signaling to the target second device indicating that the second pairing request is received.

[0172] Since the first device and the target second device requesting pairing are registered to the server, and the target second device and the target first device requesting pairing are registered to the server, the server can return a received pairing request to the first device or the target second device, indicating that the pairing request is allowed. The server can return a paired request ok signaling to the device initiating the pairing request.

[0173] 211, the server determines whether the two peer devices expected to be paired match based on the first pairing request and the second pairing request.

[0174] The two peer devices refer to the first device and the target second device, which are peer devices to each other. The server detects the matching of the first pairing request and the second pairing request.

[0175] In the 211, the server can determine whether the two peer devices expected to be paired match based on the information included in the first pairing request and the information included in the second pairing request. For example, the server can determine whether the two peer devices expected to be paired match based on detecting whether the target first device information matches the first device information of the first device. If the target first device information matches the first device information of the first device, it is determined that the two peer devices expected to be paired match; otherwise, it is determined that the two peer devices expected to be paired do not match.

[0176] 204 can comprise determining at least one target second device information associated with a target second device, for example, comprising a plurality of times, the first device can send a plurality of first pairing requests to the server, each first pairing request is used to request to establish pairing with a target second device. 209 can comprise determining at least one target first device information associated with a target first device, for example, comprising a plurality of times, the target second device can send a plurality of second pairing requests to the server, each second pairing request is used to request to establish pairing with a target first device.

[0177] In 211, if the server determines that the two counterpart devices of the expected pairing match, go to 212; otherwise, go to 214.

[0178] 212, the server sends the device network information of the counterpart device to at least one of the first device and the target second device.

[0179] The device network information can include at least one of an IP address, a port number, and a custom protocol supported by the device for pairing.

[0180] In 212, the server can send the device network information of the target second device to the first device. Alternatively, the server can send the device network information of the first device to the target second device. Alternatively, the server can send the device network information of the target second device to the first device, and send the device network information of the first device to the target second device.

[0181] 213, at least one of the first device and the target second device receives the device network information of the counterpart device sent by the server, and establishes pairing with the counterpart device based on the device network information.

[0182] Among them, the first device receives the device network information of the target second device, and establishes pairing with the target second device based on the device network information. Alternatively, the target second device receives the device network information of the first device, and establishes pairing with the first device based on the device network information. Alternatively, both are performed simultaneously.

[0183] Among them, after the pairing is completed, the first device and the target second device can directly communicate based on any custom protocol supported by both for pairing.

[0184] 214, the server sends a pairing failure notification to the first device and the target second device.

[0185] In 204 or 209 above, the corresponding device can send the corresponding pairing request to the server in response to the triggering of an event, for example, including that the control for sending the pairing request is triggered, or the corresponding target device information is determined to reach a preset delay time, which can be 0, or other time not equal to 0.

[0186] In some embodiments, the pairing implemented by 213 is real-time pairing, after which the first device and the target second device can communicate in real time, and the communication between the two does not need to be relayed through the server, but is directly end-to-end communication. In some embodiments, the pairing implemented by 213 is pre-appointment pairing, after which the first device and the target second device enter and implement real-time pairing when a pre-appointment condition is met. Exemplary pre-appointment conditions include arrival of a pre-appointment pairing time, a physical state of the remote doctor console or the patient surgery platform meeting pairing requirements, and the like.

[0187] In some embodiments, after 213, the first device and the target second device establish pairing, and the server returns a pairing success state to the server, and the server receives the pairing success state and sends a pairing success prompt to the first device and the target second device. In some embodiments, after 211, if the server determines that the opposite device expected for pairing does not match, the server can send a pairing failure prompt to the first device and the target second device. The pairing success or pairing failure prompt can be presented in the form of a UI interface or sound.

[0188] In some embodiments, the two devices can send pairing requests to the server at the same time or at different times. When any device sends a pairing request to the server, the server can start a timer to time the time taken by the opposite device expected by the device to send a pairing request to the server. If the opposite device also sends a pairing request to the server within a set waiting time range, and the server determines that the pairing request sent by the device matches the pairing request sent by the opposite device, it is determined that the device and the opposite device match, and is considered to be a successful pairing. If the pairing request does not match or times out, it is determined that the device and the opposite device do not match, and is considered to be a failed pairing. For example, the set time can be 60s, 120s, 180s, etc. Setting an appropriate waiting time range can improve user experience and avoid unnecessary time waste during pairing.

[0189] <Second mode - single-end call>

[0190] In some embodiments, referring to FIG. 10, the method comprises:

[0191] 301. The first device sends a first request to the server.

[0192] 302. The server receives the first request and sends second device information of a second device that can be paired to the first device based on the first request.

[0193] 303. The first device receives the second device information of the second device that can be paired sent by the server.

[0194] 304, the first device acquires target second device information of the target second device determined based on the second device information, and sends a first pairing request to the server.

[0195] 305, the server receives the first pairing request sent by the first device.

[0196] The server can return signaling that the first pairing request is received to the first device after receiving the first pairing request.

[0197] 301~305 in the second mode are the same or similar in principle to 201~205 in the first mode, which will not be repeated here.

[0198] 306, the server sends the first pairing request to the target second device associated with the target second device information.

[0199] 307, the target second device receives the first pairing request sent by the server, and acquires a response instruction generated in response to the first pairing request.

[0200] The target second device can generate and play a notification based on the first pairing request to facilitate the doctor to respond to the pairing request, that is, to respond.

[0201] The content of the notification is whether the target second device accepts the pairing initiated by the first device. In some embodiments, the notification can be played in the form of a UI interface or in the form of sound.

[0202] In some embodiments, the user can make a decision on the notification that the target second device accepts or rejects the pairing of the first device. For example, when played in the form of a UI interface, the user can display "accept the pairing of the first device" and "reject the pairing of the first device" in the display screen of the second target device. When the user operates the "accept the pairing of the first device" operation control, it means to accept the pairing of the first device, or when the user operates the "reject the pairing of the first device" operation control, it means to reject the pairing of the first device. For example, when played in the form of sound, "whether to accept the pairing of the first device" can be played in the audio device of the second target device. The user can identify the "accept the pairing of the first device" input by the user through the voice recognition device set by the second target device, and generate a response instruction to accept the pairing of the first device, or identify the "reject the pairing of the first device" input by the user, and generate a response instruction to reject the pairing of the first device.

[0203] In some embodiments, the second target device can also make the decision of accepting or rejecting the pairing of the first device, i.e. by itself. For example, when the target second device receives the first pairing request sent by the server, the target second device can start a timer to count time. When the response waiting time range is reached and the target second device does not obtain the response of the user to the notification, the target second device generates a response instruction of rejecting the pairing of the first device by default, and sends the response instruction to the server. After receiving the response instruction, the server sends a notification of pairing failure to the first device. For another example, the target second device can be configured to store a white list associated with one or more first devices, and the target second device allows the first devices associated with the white list to pair with it without the user responding to the notification. For example, in the white list scenario, if the first device is in the white list of the target second device, the user can actively respond to the notification within the response waiting time range, and the response includes the target second device accepting or rejecting the pairing of the first device. When the preset condition is met, such as when the response waiting time range is reached and the target second device does not obtain the response of the user to the notification, the target second device automatically generates a response instruction of accepting the pairing of the first device, and sends the response instruction to the server. After receiving the response instruction, the server sends a notification of pairing success to the first device.

[0204] 308, the target second device sends the response instruction to the server.

[0205] 309, the server sends the device network information of the opposite device to at least one of the first device and the target second device in response to receiving the positive response instruction.

[0206] If the response instruction indicates that the target second device accepts the pairing of the first device, i.e. the response instruction is a positive response instruction.

[0207] 310, at least one of the first device and the target second device receives the device network information of the opposite device sent by the server, and establishes the pairing with the opposite device based on the device network information.

[0208] For example, the server sends the device network information of the target second device to the first device, the first device receives the device network information, and establishes the pairing with the target second device based on the device network information.

[0209] 311, the server sends a prompt of pairing success to the first device and the target second device.

[0210] 312, the first device receives the prompt of pairing success sent by the server, and plays the prompt of pairing success; the target second device receives the prompt of pairing success sent by the server, and plays the prompt of pairing success.

[0211] 313, the server sends a pairing failure prompt to the first device in response to receiving the negative response instruction.

[0212] If the response instruction indicates that the target second device rejects the pairing of the first device, i.e., the response instruction is a negative response instruction.

[0213] In addition, the server can also send a pairing failure prompt to the target second device at the same time.

[0214] 314, the first device receives the pairing failure prompt sent by the server and plays the pairing failure prompt.

[0215] In some embodiments, the first device is a remote doctor console, and the target second device is a patient surgery platform, which is suitable for use when the doctor initiates matching with the patient. In some embodiments, the first device is a patient surgery platform, and the target second device is a remote doctor console, which is suitable for use when the patient initiates matching with the doctor. The two application scenarios can maximize the use of high-quality medical resources.

[0216] In the above embodiments, the request sent by the local device to the server for requesting a pairable opposite device, such as the first request sent by the first device to the server, or the second request sent by the second device to the server, can include or not include a filtering condition.

[0217] When these requests do not include any filtering condition, in order to facilitate the pairing of the local device with the target opposite device among the pairable opposite devices, the server can obtain various state information of the opposite devices and send them to the local device for prompting, such as displaying. These state information can include at least one of the following: custom protocol information supported by the opposite device for pairing, online state information, device type information, device function information, applicable surgery type information, network state information, device positioning information, power-on self-test state information, pairing state information, pre-booking pairing time information, and control authority allocation state information. The doctor can select the target opposite device for desired pairing according to these state information, however, too much state information is not conducive to quickly and accurately performing the desired pairing. Therefore, in order to further improve the pairing efficiency, the request can be configured to include a filtering condition, so that the server can accurately provide the pairable opposite devices based on the filtering condition, so that the doctor can select the target opposite device for desired pairing.

[0218] In some embodiments, the filtering condition can be configured based on the above-mentioned state information of the local device or the opposite device that can be obtained. Some filtering conditions are associated with the state information of the local device and the opposite device, and some filtering conditions are only associated with the state information of the opposite device. The filtering condition can include at least one of the following.

[0219] For example, the screening condition includes a custom protocol supported by the local device for pairing. The server can obtain the screening condition from the device information of the local device, and match, based on the screening condition, a counterpart device from the counterpart devices registered to the server, which matches the custom protocol supported by the local device for pairing, as a pairable counterpart device. By setting the screening condition, a pairable counterpart device that is actually pairable and used can be obtained.

[0220] For example, the screening condition includes a device type of the local device. The registration information of the local device or the counterpart device stored by the server can include the device type of the device. For the counterpart device being a patient surgery platform, the device type can be embodied in the type of the slave robot, which includes a single-hole laparoscopic slave robot, a multi-hole laparoscopic slave robot, a bronchial intervention slave robot, a vascular intervention slave robot, or an orthopedic slave robot, etc. For the counterpart device being a remote doctor console, the device type can be embodied in the type of the operation part, which includes a gamepad-form operation part, a mechanical linkage-form operation part, a magnetic navigation-form operation part, or a trackball-form operation part, etc. Different types of remote doctor consoles can be suitable for different types of patient surgery platforms. The server can obtain the screening condition from the device information of the local device, and match, based on the screening condition, a counterpart device from the counterpart devices registered to the server, which matches the device type of the local device, as a pairable counterpart device.

[0221] For example, the screening condition includes a device function supported by the local device. The registration information of the local device or the counterpart device stored by the server can include a device function supported by the device. The device function includes a device core function. Only when the remote doctor console and the patient surgery platform match in the device core function, pairing is allowed. Exemplarily, the device core function includes having a motion control function or having an energy control function. This is especially needed to be considered when the control authority to be configured is an (full) operator authority. If the control authority to be configured is only an observer authority, as long as the two counterpart devices expected to be paired both include an observation function, pairing is allowed regardless of whether the device core function matches. The server can obtain the screening condition from the device information of the local device, and match, based on the screening condition, a counterpart device from the counterpart devices registered to the server, which matches the device function supported by the local device, as a pairable counterpart device. By setting the screening condition, a pairable counterpart device that is actually pairable and used can be obtained.

[0222] For example, the screening condition includes a surgery type supported by the local device, and the registration information of the local device or the remote device stored by the server can include the suitable surgery type of the device, different slave robots can be suitable for the same or different surgery types, for example, for thoracic or abdominal surgery, a (single-hole or multi-hole) endoscopic slave robot can be suitable; for bronchial or vascular or urinary tube surgery, a (bronchial or vascular) interventional slave robot can be suitable; for orthopedic surgery, an orthopedic slave robot can be suitable. Different remote physician consoles can be suitable for the same or different surgery types. The surgery type can also include more specific surgical procedures, such as urinary surgery, lung surgery, liver surgery, kidney surgery, knee replacement surgery, etc. For urinary surgery, a slave robot can be matched to a (vascular) interventional slave robot; for lung surgery, a slave robot can be matched to a (bronchial) interventional slave robot; for liver surgery or kidney surgery, a slave robot can be matched to a (single-hole or multi-hole) endoscopic slave robot; for knee replacement surgery, a slave robot can be matched to an orthopedic slave robot. Among them, the server can obtain the screening condition from the device information of the local device, and match the remote devices that match the surgery type supported by the local device from the remote devices registered to the server as the pairable remote devices based on the screening condition. By setting the screening condition, the pairable remote devices that can be truly paired and used can be obtained.

[0223] For example, the screening condition includes a pre-pairing time of the local device. The pre-pairing time of the local device can be input by the physician and obtained by the server. The remote device can be input by the physician with a surgery planning time, and the surgery planning time includes an occupied time period and an idle time period. The server can match the pairable remote devices whose pre-pairing time does not conflict with the idle time period from the surgery planning time of the remote devices obtained by the server based on the pre-pairing time of the local device. Among them, the remote device in the unpaired state and without configuring the surgery planning time can be considered as the idle time period of the surgery planning time.

[0224] For example, the screening condition includes an expected online state of the remote device. When the server obtains that the remote device is currently logged in the server, the server can mark the remote device as online; when the server obtains that the remote device is currently not logged in the server, the server can mark the remote device as offline. The physician can configure the expected online state as the screening condition, so that the server matches the pairable remote devices that meet the online state, which is beneficial to quickly obtain the remote devices that can be paired in real time.

[0225] For example, the screening condition includes a desired network state of the peer device. The server can obtain the network state between the peer device and the server, and the network state can include a network type and a network communication state, and the network communication state includes normal and abnormal. The doctor can configure the desired network state as the screening condition, so that the server matches the peer device that meets the network state.

[0226] For example, the screening condition includes a desired pairing state of the peer device. The server can obtain and maintain the pairing state of the local device or the peer device, and the pairing state includes paired and unpaired. To ensure privacy and security, the peer device that can be paired is usually from the peer device in the unpaired state. In the remote teaching scenario, multiple local devices can be paired with one peer device, and the peer device in the paired state may appear. In this scenario, the peer device that can be paired is also allowed to come from the peer device in the paired state. The doctor can configure the desired pairing state as the screening condition, so that the server matches the appropriate peer device that can be paired.

[0227] For example, the screening condition includes a desired device location of the peer device. The registration information of the local device or the peer device stored by the server can include the device location of the device, which can be obtained by the positioning module configured for the device, such as a GPS positioning module, a Beidou positioning module, a Starlink positioning module, etc. It can also be obtained by IP address resolution, or it can be manually written. Various positioning methods can be used for mutual verification. If the verification is successful, it is used for screening. If the verification fails, the user can be prompted to manually modify the device location for screening. The device location can include multiple levels, such as continent level, country level, province level, city level, district level, street level, or building level. The doctor can configure the desired device location as the screening condition, so that the server matches the peer device that matches the device location.

[0228] For example, the screening condition includes a desired control authority allocation state of the peer device. The server can obtain the control authority allocation state configured by the peer device, and the control authority allocation state includes all control authorities allocated state, part of the control authority allocated state, and control authority unallocated state. For the all control authority allocated state, it is more suitable for pairing in the teaching scenario. For the part of the control authority allocated state, it is more suitable for pairing in the collaborative surgery scenario. For the control authority unallocated state, it is suitable for pairing in any scenario. The doctor can configure the desired control authority as the screening condition, so that the server matches the appropriate peer device that can be paired.

[0229] For example, the screening condition includes the power-on self-test state expected by the terminal device. When the server communicates with the local terminal device or the opposite terminal device, the power-on self-test state of the device can be obtained, which can include various levels, for example, it can include that the software and hardware of the device are all normal or the core functions of the software and hardware of the device are all normal. Generally, the matched opposite terminal device is expected to be normal in software and hardware. However, when the opposite terminal device expected to be paired does not meet the requirement of being normal in software and hardware, in order to ensure that the ultra-long distance surgery can be implemented, as long as the core functions of the software and hardware are normal, the opposite terminal device can also be paired for use. The power-on self-test state can be associated with the patient condition information. For example, when the opposite terminal device is a patient surgery platform, the device software and hardware functions include core devices such as mobile robots and energy platforms, and auxiliary devices such as monitors or breathing machines. When the patient condition is not serious and the auxiliary device does not need to be applied, even if the auxiliary device software and hardware functions are abnormal, as long as the core device software and hardware functions are normal, it can also be paired for use, which can meet the minimum pairing requirement and improve the pairing success rate. Wherein, whether the auxiliary device needs to be applied can be obtained from the patient condition information input by the local terminal device or the opposite terminal device, and the patient condition information can include the definition of the core device and the auxiliary device to be used. The doctor can configure the expected power-on self-test state for the server to match the appropriate pairable opposite terminal device.

[0230] Wherein, each of the above screening conditions can be used alone or in combination, and the opposite terminal device that meets one or more of the above screening conditions can be used as a pairable opposite terminal device.

[0231] <Third mode - administrator direct connection>

[0232] In some embodiments, the server can obtain all first devices and second devices that complete registration, and perform expected pairing based on the obtained first devices and second devices, which can realize batch configuration of devices in multiple places and improve pairing efficiency. However, in actual situations, different communication protocols can be agreed between these first devices and second devices, and since the communication protocol is invisible, the pairing led by the doctor is often trial and error, resulting in low pairing efficiency.

[0233] In some embodiments, the server can configure different tags for different custom protocols supported by the corresponding device for pairing, and when the device information of the corresponding device is displayed on the display screen of the device or the server, the tag can be displayed together to facilitate the doctor to identify the devices that can be paired with each other. Wherein, the custom protocol supported by each device for pairing can include that different tags can be configured for the custom protocol of the device respectively. For example, the first device and the second device with the same tag can be paired with each other, and the first device and the second device with different tags cannot be paired with each other. Wherein, different tags can be represented by different numerical codes, characters, words, colors or symbols, etc.

[0234] In some embodiments, the physician can utilize the server itself for administrator direct connection; or, the physician can utilize any device that can log into the server with administrator permission, including a remote physician console, a patient surgery platform, a desktop computer, or a portable mobile terminal, including a cell phone, a tablet, or a laptop, etc. In some embodiments, these devices that can be used for administrator direct connection can be registered to the server, and the registration of these devices can be authenticated by a custom protocol or an encryption algorithm as previously described to improve security. In some embodiments, the mobile terminal device can be bound to the remote physician console or the patient surgery platform, and participate in the pairing configuration of the above-mentioned multiple modes as the remote physician console or the patient surgery platform.

[0235] In some embodiments, to facilitate administrator direct connection using the server, the present disclosure provides a method, as shown in FIG. 11, which comprises:

[0236] 401, obtaining first device information of a first device registered to the server.

[0237] 402, obtaining target first device information associated with a target first device determined based on the first device information.

[0238] The target first device is used for pairing with a second device.

[0239] 403, based on the target first device information, matching second device information of a pairable second device that can be paired with the target first device from a second device registered to the server.

[0240] As previously described, the pairable second device that can be paired with the target first device is usually compatible with the custom protocol supported by the target first device for pairing.

[0241] 404, obtaining target second device information associated with a target second device determined based on the second device information.

[0242] 405, obtaining device network information of at least one of the target first device and the target second device.

[0243] 406, sending the device network information to a peer device associated with the network information.

[0244] 407, the peer device receives the network information, and establishes pairing with a local device associated with the network information based on the network information, to realize direct communication.

[0245] For example, in 406, the device network information includes the device network information of the target first device, the device network information is sent to the target second device. Then in 407, the target second device establishes the pairing with the target first device based on the device network information of the target first device.

[0246] For example, in 406, the device network information includes the device network information of the target second device, the device network information is sent to the target first device. Then in 407, the target first device establishes the pairing with the target second device based on the device network information of the target second device.

[0247] For example, in 406, the device network information includes the device network information of the target first device and the device network information of the target second device, the device network information of the target first device is sent to the target second device, and the device network information of the target second device is sent to the target first device. Then in 407, the target second device establishes the pairing with the target first device based on the device network information of the target first device, and the target first device establishes the pairing with the target second device based on the device network information of the target second device.

[0248] Through the above 401-406, the pairing can be quickly performed, and the pairing efficiency can be improved.

[0249] After 406, after the target first device and the target second device establish the pairing, at least one of the target first device and the target second device returns a pairing success state to the server, and the server receives the pairing success state to update the pairing state maintained by the server. The server can send a pairing success prompt to the target first device and the target second device.

[0250] In some embodiments, the target first device can include one, the target second device can include at least one, and the pairable second device can include a second device supporting a custom protocol for pairing compatible with the custom protocol for pairing supported by the target first device.

[0251] For example, assuming that the custom protocol for pairing supported by the target first device is A, and the custom protocol for pairing supported by the second device includes A, the second device can be regarded as a pairable second device.

[0252] In some embodiments, the target first device can include at least one, the target second device can include one, and the pairable second device can include a second device supporting a custom protocol for pairing compatible with the custom protocol for pairing supported by each target first device.

[0253] For example, assuming that the custom protocol for pairing supported by one target first device is A, and the custom protocol for pairing supported by another target first device is B, and the custom protocol for pairing supported by the second device is compatible with A and B, the second device can be regarded as a pairable second device.

[0254] For example, the second device can be a pairable second device when the custom protocol for pairing supported in the second device is compatible with at least one of the custom protocols for pairing supported in each of the target first devices.

[0255] For example, assuming that a target first device supports a custom protocol for pairing A and B, another target first device supports a custom protocol for pairing C and D, and the second device supports a custom protocol for pairing that is compatible with at least one of A and B and at least one of C and D, the second device can be a pairable second device. For example, the second device can be a pairable second device when the custom protocol for pairing includes A and C, or A and D, or B and C, or B and D. Of course, the second device can also be a pairable second device when the custom protocol for pairing includes A, B and C, or A, C and D, or B, C and D, or A, B, C and D.

[0256] For example, assuming that a target first device supports a custom protocol for pairing A and B, another target first device supports a custom protocol for pairing B and C, and the second device supports a custom protocol for pairing that is compatible with at least one of A and B and at least one of B and C, the second device can be a pairable second device. For example, the second device can be a pairable second device when the custom protocol for pairing includes B, or A and B, or B and C, or A and C. Of course, the second device can also be a pairable second device when the custom protocol for pairing includes A, B and C.

[0257] In some embodiments, the screening conditions can be configured to obtain suitable devices when determining the first device for pairing and the second device for pairing with the first device. The screening conditions are described above and will not be repeated here.

[0258] The pairing methods of the first mode to the third mode described above can quickly realize one-to-one, one-to-many, or many-to-one pairing between any remote doctor console and patient surgery platform. The methods can simplify user operation, save labor cost and time cost, and reduce the complexity of remote surgery, improve the universality of remote surgery, and help to establish a super-remote surgery medical network.

[0259] In some embodiments, in the above pairing modes, the server can save pairing information of the first device and the second device for establishing pairing, and assign a unique pairing identifier to each of the first device and the second device. The first device and the second device can attempt to establish pairing and communicate according to the pairing identifier, and are prohibited from communicating with each other before the pairing is successful. The pairing identifier can be included in the signaling (i.e., data) of each subsequent communication for communication authentication, and a receiving device has the right to refuse communication services for devices with mismatched pairing identifiers.

[0260] In some embodiments, the server can obtain the pairing status of all devices in real time, and can maintain a pairing table according to the pairing status, which records the paired, pairing or unpaired status of the devices. The server can set the status of the device that has completed real-time pairing as paired. The server can set the status of the device that is scheduled to be paired or is establishing real-time pairing as pairing. The server can also delete the pairing information of the device that fails to pair or is unpaired, and set the status of the device that fails to pair or is unpaired as unpaired. Wherein, with the update of the pairing table, the second device information of the second device that can be paired, or the first device information of the first device and the second device information of the second device that can be paired with each other can be updated and sent to the related devices, for example, the former is updated and sent to the first device, and the latter is updated and sent to the server, to facilitate pairing.

[0261] In some embodiments, the first device, the second device or the server can include a display screen and an input device, and the visual pairing of the first device and the second device can be quickly realized by using the display screen and the input device. The display screen and the input device can be independent components, and the input device can include a mouse, a keyboard, etc., wherein, on the side of the remote doctor console, the operating part thereof can also be used as a mouse in the pairing stage. The display screen and the input device can also be integrated components, for example, a touch screen.

[0262] <UI interaction of double-end mutual call>

[0263] In some embodiments, the present disclosure provides a method for pairing, referring to FIG. 12, the method comprises:

[0264] 501, the first device displays a first configuration interface on the first display screen thereof, and in response to obtaining the triggering of a first control in the first configuration interface, sends a first request to the server.

[0265] 502, the server receives the first request sent by the first device, and sends the second device information of the second device that can be paired to the first device based on the first request.

[0266] 503, the first device receives the second device information of the second device that can be paired sent by the server, and displays the second device information in the first configuration interface.

[0267] 504, the first device obtains the target second device information of the target second device that is expected to be paired based on the second device information, and sends a first pairing request to the server.

[0268] The first device can automatically trigger the first pairing request to the server immediately after determining the target second device information. The first device can trigger the first pairing request to the server in response to an event after determining the target second device information. The event includes that a timer of the first device reaches a delay time since determining the target second device information, or a control in the first configuration interface is triggered, the control being a control for controlling sending of the pairing request. The first device triggers the first pairing request to the server after obtaining that the timer reaches the delay time or obtaining that the control is triggered.

[0269] 505. The server receives the first pairing request sent by the first device.

[0270] The server can return a signaling that the first pairing request is received to the first device after receiving the first pairing request.

[0271] 506. The target second device displays a second configuration interface on a second display screen thereof, and sends a second request to the server in response to obtaining a trigger of a first control in the second configuration interface.

[0272] 507. The server receives the second request sent by the target second device, and sends first device information of a first device that can be paired to the target second device based on the second request.

[0273] 508. The target second device receives the first device information of the first device that can be paired sent by the server, and displays the first device information in the second configuration interface.

[0274] 509. The target second device obtains target first device information determined based on the first device information, and sends a second pairing request to the server.

[0275] The target second device can automatically trigger the second pairing request to the server immediately after determining the target first device information. The target second device can trigger the second pairing request to the server in response to an event after determining the target first device information. The event includes that a timer of the target second device reaches a delay time since determining the target first device information, or a control in the second configuration interface is triggered, the control being a control for controlling sending of the pairing request. The target second device triggers the second pairing request to the server after obtaining that the timer reaches the delay time or obtaining that the control is triggered.

[0276] 510. The server receives the second pairing request sent by the target second device.

[0277] The server can return a signaling that the second pairing request is received to the target second device after receiving the second pairing request.

[0278] 511, the server determines whether the two peer devices expected to be paired match based on the first pairing request and the second pairing request.

[0279] If the server determines that the two peer devices expected to be paired match, go to 512; otherwise, go to 515.

[0280] 512, the server sends the device network information of the peer device to at least one of the first device and the target second device.

[0281] 513, at least one of the first device and the target second device receives the device network information of the peer device sent by the server, and establishes pairing with the peer device based on the device network information.

[0282] 514, after the first device and the target second device establish pairing, a message indicating that the pairing is successfully established is displayed on the first configuration interface and the second configuration interface respectively.

[0283] 515, the server sends a notification indicating that the pairing is failed to the first device and the target second device.

[0284] 516, the first device and the target second device receive the notification indicating that the pairing is failed sent by the server, and display a message indicating that the pairing is failed on the first configuration interface and the second configuration interface respectively.

[0285] For example, the message indicating that the pairing is successfully established or the message indicating that the pairing is failed can be displayed by popping up a dialog box on the configuration interface, or can be displayed by a fixed or floating status bar in the configuration interface.

[0286] In some embodiments, referring to FIG. 13 to FIG. 15, in 501 or 506, the first control 1 can be a button, and when the button 1 is clicked, it indicates that the first control 1 is triggered, and the corresponding device sends a corresponding request to the server to request the device information of the pairable counterpart device. In 503 or 508, the corresponding device information sent by the server can be displayed on the corresponding configuration interface through the second control 2, which can be a list control. The list control 2 includes at least one list option, and one list option is associated with the device information of one device. In 504 or 509, when the corresponding device detects that one or more list options are selected, it determines that the device information associated with the one or more list options is the target device information, i.e., determines the target counterpart device that the corresponding device expects to pair. For example, referring to the configuration interface of a certain remote doctor console, such as a remote doctor console of hospital A, as shown in FIG. 13, when the first control 1 is triggered, the remote doctor console of hospital A sends a request to request a pairable counterpart device. As shown in FIG. 14, the server matches the patient surgery platforms of hospitals A to E and displays them on the configuration interface as a list control 2. As shown in FIG. 15, when the list option associated with the patient surgery platform of hospital B is selected and confirmed, the patient surgery platform of hospital B is the counterpart device that the remote doctor console of hospital A expects to pair with, and sends a pairing request. Similarly, if the remote doctor console of hospital A is determined to be the counterpart device that the patient surgery platform of hospital B expects to pair with in the corresponding configuration interface of the patient surgery platform of hospital B, and sends another pairing request, the two can be matched to establish a pair; otherwise, the two are not matched and cannot establish a pair.

[0287] In this embodiment, when the corresponding device obtains that the pairing fails, it can automatically restore the list option to the initial state, such as the unselected state, to facilitate subsequent pairing of the corresponding device with other counterpart devices. The corresponding device can also release the pairing with the counterpart device when it obtains that the list option is restored to the initial state, such as the unselected state.

[0288] In some embodiments, with reference to FIGS. 16-19, in 501 or 506, the first control 1' can be a window control, and the corresponding device can generate a second control 2' in the respective configuration interface according to the device information of the corresponding device, the second control 2' can be a drag control, which has an initial position in the respective configuration interface, when the second control 2' is moved into the window of the window control 1', it indicates that the first control 1' is triggered, and the corresponding device sends a corresponding request to the server to request the device information of the pairable opposite device. In 503 or 508, the corresponding device information can be displayed in the respective configuration interface through a third control 3', which can be a drag control, which has an initial position in the respective configuration interface, the number of the third control 3' is the same as the number of the pairable opposite device, and each third control 3' corresponds to an opposite device. In 504 or 509, when the corresponding device detects that the third control 3' is moved into the window of the window control 1', it determines that the device information associated with the third control 3' is the target device information, i.e., determines the target opposite device that the corresponding device expects to pair. For example, with reference to the configuration interface of a certain remote physician console, such as a remote physician console of hospital A, when the second control 2' representing the remote physician console of hospital A is moved into the window of the window control 1', the remote physician console of hospital A sends a request to request a pairable opposite device; as shown in FIGS. 16 and 17, at this time, the server matches the patient surgery platforms of hospitals A-E and displays them in the configuration interface as drag controls 3'; as shown in FIG. 19, when the drag control 3' associated with the patient surgery platform of hospital B is moved into the window of the window control 1' and is confirmed, the patient surgery platform of hospital B is the opposite device that the remote physician console of hospital A expects to pair, and a pairing request is sent. Similarly, if the remote physician console of hospital A is determined to be the opposite device that the patient surgery platform of hospital B expects to pair in the configuration interface of the patient surgery platform of hospital B and sends another pairing request, the two devices can be matched to establish a pairing; otherwise, the two devices are not matched and cannot establish a pairing.

[0289] In this embodiment, referring to FIGS. 20-22, the window of the window control 1' can include a first area 11' and a second area 12', the first area 11' including a first sub-area 11a' and a second sub-area 11b' for accommodating the second controls 2' respectively, and the second area 12' for accommodating the third control 3', wherein these areas can also be implemented by windows. In the case that one or more remote physician consoles can be paired with one patient surgery platform, the second controls 2' are associated with remote physician consoles, and the third control 3' is associated with a patient surgery platform. The first sub-area 11a' and the second sub-area 11b' can be associated with different control authorities, or can also be associated with different pairing sequences. For example, in response to the third control 3' moving to the second area 12', the patient surgery platform associated with the third control 3' sends a first request to the server, and the patient surgery platform can match the pairable remote physician consoles that can be paired with the patient surgery platform and display in the configuration interface of the patient surgery platform, and the pairable remote physician consoles can be displayed in the form of drag controls outside the window.

[0290] For example, the first sub-area 11a' can be associated with the first control authority of the remote physician console accommodated therein to the patient surgery platform accommodated in the second area 12', and the second sub-area 11b' can be associated with the second control authority of the remote physician console accommodated therein to the patient surgery platform accommodated in the second area 12', and the first control authority is different from the second control authority. For example, the first control authority is a full operator authority, and the second control authority is an observer authority. When the second control 2' associated with the first remote physician console is moved to the first sub-area 11a', the first remote physician console is configured to have full operator authority to the patient surgery platform accommodated in the second area 12'; when the second control 2' associated with the second remote physician console is moved to the second sub-area 11b', the second remote physician console is configured to have observer authority to the patient surgery platform accommodated in the second area 12'. As shown in FIG. 22, for the same patient surgery platform, such as the patient surgery platform of hospital A, the remote physician console of hospital A has different control authorities or pairing sequences relative to the remote physician console of hospital B. In this example, the second sub-area 11b' can include one or more, which is suitable for the rapid pairing of a remote physician console with a full operator authority and a plurality of observer authorities with a patient surgery platform. In this way, the rapid allocation of control authorities can be achieved when pairing.

[0291] For example, the first sub-region can associate the first remote physician console accommodated therein with a first pairing order with the patient surgery platform accommodated in the second region, the second sub-region can associate the second remote physician console accommodated therein with a second pairing order with the patient surgery platform accommodated in the second region, and the first pairing order and the second pairing order are different. For example, the first pairing order is prior to the second pairing order, when the second control associated with the first remote physician console is moved to the first sub-region, the first remote physician console is configured to have the first pairing order with the patient surgery platform accommodated in the second region; when the second control associated with the second remote physician console is moved to the second sub-region, the second remote physician console is configured to have the second pairing order with the patient surgery platform accommodated in the second region. In this example, the first region can include more sub-regions, and different sub-regions have different pairing orders. In this way, the pairing order can be quickly set during pairing. In this example, the first region can also be used to accommodate different patient surgery platforms, and the second region is used to accommodate remote physician consoles, so that the pairing order between different patient surgery platforms and the same remote physician console can also be quickly set. For example, in response to the drag control associated with the patient surgery platform being moved to the second region, the patient surgery platform can send a first request to the server, and the patient surgery platform can match the pairable remote physician console that can be paired with the patient surgery platform and display it in the configuration interface of the patient surgery platform. The pairable remote physician console can be displayed in the form of a drag control outside the window.

[0292] In this embodiment, whether the second control or the third control is moved into the first control can be determined by detecting whether the pixel coordinates of the boundary of the second control or the third control fall within the pixel coordinates of the boundary of the first control. Whether the second control or the third control is moved into the corresponding region can also be determined by detecting whether the pixel coordinates of the boundary of the second control or the third control fall within the pixel coordinates of the boundary of the corresponding region.

[0293] In this embodiment, the drag control can have the same or different size before and after being moved. For example, the drag control can have a larger or smaller size relative to the size before being moved when the drag control is moved into the first control.

[0294] In this embodiment, when the corresponding device obtains that the pairing fails, at least the third control of the second control and the third control is removed from the first control and reset to the position before being moved, to facilitate subsequent pairing of the corresponding device with other peer devices. The corresponding device can also release the pairing with the peer device when the second control and the third control are removed from the first control. When the second control or the third control is manually removed, the second control or the third control can be automatically restored to the position before being moved once the second control or the third control is removed from the first control.

[0295] In some embodiments, in 501 or 506, the first control can be a button, and when the button is clicked, it indicates that the first control is triggered, and the corresponding device sends a corresponding request to the server to request the device information of the counterpart devices that can be paired. The corresponding device can generate the second controls in the respective configuration interface according to its device information. In 503 or 508, the corresponding device information sent by the server can be displayed in the respective configuration interface through the third controls, and the number of the third controls is the same as the number of the counterpart devices that can be paired, each third control corresponds to a counterpart device, and the second controls and the third controls have fixed positions in the configuration interface. In 504 or 509, when the display screen of the corresponding device is a touch screen, the corresponding device determines the device information associated with the third control corresponding to the start or end of the contact and continuous movement as the target device information, i.e., determines the target counterpart device that the corresponding device expects to pair, when it detects that the finger on the touch screen moves from one of the second controls and the third controls and keeps the contact and continuously moves to the other of the second controls and the third controls. When the display screen of the corresponding device is a touch screen or a non-touch screen, the corresponding device determines the device information associated with the third control corresponding to the start or end of the continuous movement as the target device information, i.e., determines the target counterpart device that the corresponding device expects to pair, when it detects that the cursor moves from one of the second controls and the third controls and continuously moves to the other of the second controls and the third controls on the display screen. In 504 or 509, when the display screen of the corresponding device is a touch screen, the corresponding device can also determine the device information associated with the third controls included in the closed trajectory as the target device information, i.e., determines the target counterpart device that the corresponding device expects to pair, when it detects that the finger keeps the contact and continuously moves around the third controls to form a closed trajectory such as a circular, square, triangular, etc. closed trajectory on the touch screen. When the display screen of the corresponding device is a touch screen or a non-touch screen, the corresponding device can also determine the device information associated with the third controls included in the closed trajectory as the target device information, i.e., determines the target counterpart device that the corresponding device expects to pair, when it detects that the cursor continuously moves around the third controls to form a closed trajectory on the display screen. In 504 or 509, the corresponding device can also determine the counterpart device as the target counterpart device when it detects that the number of clicks or the time of staying of the finger or the cursor on the space corresponding to the counterpart device reaches a threshold value.

[0296] In this embodiment, the movement trajectory of the finger or the cursor on the corresponding display screen can be displayed in the corresponding configuration interface to distinguish the pairing relationship.

[0297] In some embodiments, each configuration interface can include a control for configuring the aforementioned filtering condition to accurately match the counterpart device that can be paired. The control can include a text box, a list, or a check box, etc. For example, the text box can input the filtering condition. The list can select the filtering condition. The check box can check the filtering condition. Before performing 501 or 506, the doctor can configure the filtering condition in advance. In 501 or 506, the corresponding device can encapsulate the configured filtering condition into the corresponding request and send it to the server, so that the server can filter out the counterpart device that meets the condition based on the corresponding request.

[0298] <UI interaction of single-end call>

[0299] In some embodiments, the present disclosure provides a method for pairing, referring to FIG. 23, the method comprises:

[0300] 531, the first device displays a first configuration interface on its first display screen, and in response to obtaining a trigger to a first control in the first configuration interface, sends a first request to the server.

[0301] 532, the server receives the first request sent by the first device, and sends second device information of the second device that can be paired to the first device based on the first request.

[0302] 533, the first device receives the second device information of the second device that can be paired sent by the server, and displays the second device information in the first configuration interface.

[0303] 534, the first device obtains target second device information of the second target device that is expected to be paired based on the second device information, and sends a first pairing request to the server.

[0304] 535, the server receives the first pairing request sent by the first device.

[0305] The server can return a signaling that the first pairing request is received to the first device after receiving the first pairing request.

[0306] 536, the server sends the first pairing request to the target second device associated with the target second device information.

[0307] 537, the target second device receives the first pairing request sent by the server, and displays a second configuration interface in the second display screen of the target second device, and generates a first control and a second control in the second configuration interface based on the first pairing request.

[0308] The first control may be, for example, a pop-up dialog box for notifying the doctor whether to accept the pairing of the first device. The second control may be, for example, a button, including a first button and a second button, the first button being triggered to accept the pairing of the first device, and the second button being triggered to reject the pairing of the first device.

[0309] 538, the target second device acquires the triggering of the second control, and sends the response instruction generated by the triggering of the second control to the server.

[0310] If it is detected that the first button is triggered, it is determined that the target second device accepts the pairing of the first device, and 539 is entered; if it is detected that the second button is triggered, it is determined that the target second device rejects the pairing of the first device, and 543 is entered.

[0311] 539, the server sends the device network information of the opposite device to at least one of the first device and the target second device.

[0312] 540, at least one of the first device and the target second device receives the device network information of the opposite device sent by the server, and establishes the pairing with the opposite device based on the device network information.

[0313] 541, the server sends a pairing success message to the first device and the target second device.

[0314] 542, the first device receives the pairing success message sent by the server, and displays the pairing success in the first configuration interface; the target second device receives the pairing success message sent by the server, and displays the pairing success in the second configuration interface.

[0315] 543, the server sends a pairing failure message to the first device.

[0316] 544, the first device receives the pairing failure message sent by the server, and displays the pairing failure in the first configuration interface.

[0317] The pairing success or the pairing failure can be displayed by popping up a dialog box in the corresponding configuration interface.

[0318] The method of 531~535 is the same as the method of 501~505 of the foregoing embodiment, and will not be repeated here. The method of pairing failure and unpairing can also be the same as the principle described in the foregoing embodiment, and will not be repeated here.

[0319] <UI interaction of administrator direct connection>

[0320] In some embodiments, the present disclosure provides a method for pairing, referring to FIG. 24, the method comprises:

[0321] 601, display the acquired first device information of the first device registered to the server in a configuration interface of the display screen.

[0322] 602, acquire target first device information associated with the target first device determined based on the first device information.

[0323] 603, based on the target first device information, match second device information of a second device registered to the server and capable of pairing with the target first device, and display in the configuration interface.

[0324] 604, acquire target second device information associated with the target second device determined based on the second device information.

[0325] 605, acquire device network information of at least one of the target first device and the target second device.

[0326] 606, send the device network information to a peer device associated with the network information.

[0327] 607, the peer device receives the network information and establishes pairing with the local device associated with the network information based on the network information to realize direct communication.

[0328] In some embodiments, referring to FIGS. 25-28, the first device information can be displayed in the configuration interface through a list control 1a. Different first device information is associated with different list options of the list control 1a. When it is detected that a target list option in the list options is selected, the first device information associated with the target list option can be determined as target first device information of a target first device expected to be paired.

[0329] At the same time or slightly delayed, when it is detected that a target list option in the list options 1a is selected, second device information of a second device capable of pairing with the target first device can be matched from the second devices registered to the server based on the target first device information associated with the target list option. The matched second device information of the second device capable of pairing can be displayed in the configuration interface through another list control 1b for the doctor to configure. Different second device information is associated with different list options of the other list control 1b. When it is detected that a target list option in the other list options 1b is selected, the second device information associated with the target list option can be determined as target second device information of a target second device expected to be paired.

[0330] In some embodiments, after the target first device and the target second device are determined, the device network information of at least one of the target first device and the target second device can be acquired in response to a triggering of an event. The event includes that a time counting from the determination of the target second device reaches a delay time, or a control in the configuration interface is triggered, the control being a control for triggering the acquisition of the device network information of at least one of the target first device and the target second device, for example, the confirmation button 1c in FIG. 28. In this case, the server acquires that the time counting reaches the delay time, or acquires that the control is triggered, and acquires the device network information of at least one of the target first device and the target second device.

[0331] In some embodiments, after the target first device and the target second device establish the pairing, at least one of the target first device and the target second device can return a pairing success status to the server. The server receives the pairing success status, displays a message of successful establishment of the pairing in the configuration interface, and sends a message of successful establishment of the pairing to the target first device and the target second device. The configuration interface in the target first device and the target second device can display the message of successful establishment of the pairing. For example, the message can be displayed by a pop-up dialog box.

[0332] In some embodiments, referring to FIGS. 29-31, the first device information can be displayed in the configuration interface by the first drag control 1a'. Different first device information is associated with different first drag controls 1a'. The configuration interface can include a window control 1c' having a window. After detecting that the target first drag control in the first drag control 1a' is moved into the window of the window control 1c', it can be determined that the first device information of the first device associated with the target first drag control is the target first device information of the target first device for pairing.

[0333] Meanwhile or slightly lagged, when detecting that the target first drag control 1a' in the first drag control 1a' moves into the window of the window control 1c', the second device information of the second device that can be paired with the target first device can be matched from the second devices registered to the server based on the target first device information associated with the target first drag control. Wherein the matched second device information of the second device that can be paired can be displayed in the configuration interface outside the window through the second drag control 1b', and different second device information is associated with different second drag controls 1b'. Wherein when detecting that the target second drag control in the second drag control 1b' moves into the window 1c', it can be determined that the second device information of the second device associated with the target second drag control is the target second device information of the target second device that is expected to be paired. For example, as shown in FIG. 29, the first device that can be paired includes the remote doctor console of hospital A-D; as shown in FIG. 30, when the first drag control 1a' associated with the remote doctor console of hospital A is moved to the window 1c', the server automatically matches the second device information of the second device that can be paired with the custom protocol supported by the remote doctor console of hospital A, for example, including the patient surgery platform of hospital A-E; as shown in FIG. 31, when the second drag control 1b' associated with the patient surgery platform of hospital B is moved to the window 1c', it is determined that the patient surgery platform of hospital B is the device that is expected to be paired with the remote doctor console of hospital A.

[0334] In the above embodiments, referring to FIGS. 32-34, the window control 1c' can include a first area 11c' and a second area 12c'. The first area 11c' can include a first sub-area 111c' and a second sub-area 112c' for accommodating the first drag control 1a', and the second area 12c' for accommodating the second drag control 1b', the first sub-area 111c' and the second sub-area 112c' being configured to associate different control authorities or different pairing sequences. In the case where the first drag control 1a' is associated with a remote doctor console and the second drag control 1b' is associated with a patient surgery platform, when the second drag control 1b' is moved to the second area 12c', a first drag control 1a' is moved to the first sub-area 111c', and another first drag control 1a' is moved to the second sub-area 112c', the remote doctor console associated with different first drag controls 1a' can be configured to have different control authorities or pairing sequences over the patient surgery platform associated with the second drag control 1b'. For example, as shown in FIG. 32, the first device available for pairing includes the remote doctor console of Hospital A-D; as shown in FIG. 33, when the first drag controls 1a' associated with the remote doctor console of Hospital A and the remote doctor console of Hospital B are moved to the first sub-area 111c' and the second sub-area 112c', respectively, the server automatically matches the second device information of the second device available for pairing that is compatible with the custom protocol supported by the remote doctor console of Hospital A and the remote doctor console of Hospital B, such as the patient surgery platform of Hospital A-C; as shown in FIG. 34, when the second drag control 1b' associated with the patient surgery platform of Hospital B is moved to the second area 12c', it is determined that the patient surgery platform of Hospital B is the device expected to be paired with the remote doctor console of Hospital A and the remote doctor console of Hospital B. If the first sub-area 111c' and the second sub-area 112c' are associated with different control authorities or different pairing sequences, the remote doctor console of Hospital A and the remote doctor console of Hospital B are configured to have different control authorities or different pairing sequences over the patient surgery platform of Hospital B.

[0335] <Pairing success detection>

[0336] In some embodiments, when the remote doctor console and the patient surgery platform establish real-time pairing, pairing success detection can be performed on both to ensure that the remote surgery can proceed smoothly. In this regard, at least one network channel (i.e., communication link) constructed by the remote doctor console and the patient surgery platform when they are paired in real time can be used for related detection to determine whether the two are truly successfully paired.

[0337] During a super-long distance surgery, there can be multiple types of network information exchanged between the remote physician console and the patient surgery platform. For example, the types of network information include first network information exchanged between the remote physician console and core devices in the patient surgery platform, and second network information exchanged between the remote physician console and auxiliary devices in the patient surgery platform.

[0338] The first network information can include different types of network information, such as at least one of motion control information, energy control information, multimedia interaction information, and human-machine interface interaction information. The motion control information can be typically sent from the remote physician console and executed by the slave robot, such as master-slave following control; the motion control information can also be sent from the slave robot and executed by the remote physician console, such as when the remote physician console includes an operation part in the form of a mechanical linkage, the slave robot sends motion control information for the pose of its medical instrument, and the operation part of the remote physician console executes control for the pose of the medical instrument. The energy control information is typically sent from the remote physician console and executed by the energy platform, such as controlling the on-off of a monopolar circuit, a bipolar circuit, an ultrasonic transducer, a radio frequency circuit, or a microwave circuit, etc. of the energy platform. The multimedia interaction information includes video interaction information and audio interaction information, the video interaction information is typically collected by an imaging device such as an endoscope, processed by the image platform, and sent to the display screen of the remote physician console for display; the audio interaction information is typically collected by audio intercom devices respectively configured at the remote physician console and the slave robot, processed by the image platform, and sent to the opposite device for playing. The human-machine interaction information is typically operated by the remote physician console on the touch screen of the display screen, and the interface changed after the operation is displayed on the display screen of the slave robot, such as image annotation.

[0339] The second network information can also include different types of network information, the remote physician console and different auxiliary devices typically have different network information interactions, and the remote physician console and the same auxiliary device have at least one type of network information interaction. For example, when the auxiliary device includes a monitor, the interaction information includes at least one of heart rate, pulse, and blood pressure information; for example, when the auxiliary device includes a surgical bed, the remote physician console can be configured to control the position or pose of the surgical bed, and the interaction information includes motion control information of the surgical bed sent from the remote physician console.

[0340] In some embodiments, when the pairing between the remote physician console and the patient surgery platform is established, a network channel is created between the remote physician console and the patient surgery platform for communication of network information between the two, based on the device network information sent by the server. In a teleoperation surgery, different network channels can be created to meet the transmission requirements of different types of network information between the remote physician console and the patient surgery platform. By creating different network channels for the transmission of different types of network information, a plurality of advantages can be achieved, such as facilitating bandwidth optimization, facilitating priority management, facilitating data isolation to improve security, facilitating fault isolation to improve the fault tolerance of the network, and facilitating compliance of specific data transmission.

[0341] In some embodiments, different network channels can be created to distinguish the transmission of different types of network information by configuring the port numbers of the remote physician console and the patient surgery platform. For example, a motion control network channel can be created for the transmission of motion control information, an energy control network channel can be created for the transmission of energy control information, a multimedia interaction network channel can be created for the transmission of multimedia interaction information, and a human-computer interface interaction network channel can be created for the transmission of human-computer interface interaction information.

[0342] In some embodiments, the remote physician console or the patient surgery platform can create different network channels for the transmission of network information of corresponding data types between the remote physician console and the patient surgery platform according to different configured control permissions, wherein different control permissions can be associated with the transmission of different data types. In some embodiments, different network channels can be created for the transmission of network information between corresponding device types in the remote physician console and the patient surgery platform according to different physician-patient information input in the remote physician console or the patient surgery platform, wherein different physician-patient information can be associated with the transmission of network information between different device types. For example, the physician-patient information can include the type of surgery or the surgical equipment to be used, and different physician-patient information can require the use of different core devices or auxiliary devices. Therefore, different network channels can be created based on the number or type of devices to be connected.

[0343] In some embodiments, the present disclosure also provides a method for detecting whether the pairing is successful, as shown in FIG. 35, which comprises:

[0344] 711, the server obtains information of a target network channel expected to be established between the first device and the second device, and sends device network information of the opposite device associated with the target network channel to at least one of the first device and the second device based on the obtained information of the target network channel.

[0345] The first device and the second device have a pairing relationship. Different network channels usually have different port numbers. The server sends a port number associated with a target network channel to at least one of the first device and the second device.

[0346] 712. At least one of the first device and the second device receives the device network information associated with the target network channel of the opposite device sent by the server, and creates the target network channel with the opposite device based on the device network information.

[0347] 713. At least one of the first device and the second device evaluates the communication state of the target network channel.

[0348] In some embodiments, the communication state of the target network channel can be evaluated by detecting network connectivity. For example, a heartbeat mechanism can be used to detect the network connectivity of the target network channel. Using the heartbeat mechanism requires creating a sending end and a receiving end at both ends of the target network channel. One of the first device and the second device can act as the sending end, and the other can act as the receiving end. When the sending end and the receiving end are used as a pair, both can use TCP protocol or UDP protocol. When using the heartbeat mechanism for detection, the sending end sends a message to the receiving end through the target network channel. If the receiving end returns a signaling indicating that the message has been received within a predetermined time, it indicates that the network connection between the sending end and the receiving end is normal, i.e., the network connectivity of the target network channel between the two is normal. If the receiving end does not return the signaling indicating that the message has been received within the predetermined time, it indicates that the network connection between the sending end and the receiving end is disconnected, i.e., the network connectivity of the target network channel between the two is abnormal.

[0349] In some embodiments, the target network channel includes one or more. When the target network channel includes multiple, different sending ends and receiving ends can be created for different target network channels. In other words, when the target network channel includes a first target network channel and a second target network channel, for the first target network channel, the first device can act as the sending end and the second device can act as the receiving end; for the second target network channel, the second device can act as the sending end and the first device can act as the receiving end.

[0350] In some embodiments, the communication status of the target network channel can also be evaluated by detecting network communication quality, for example, by detecting at least one of network delay, jitter, packet loss and out-of-order of the packet transmitted in the target network channel to evaluate the network communication quality. For example, the network communication quality can be evaluated by evaluating the network delay. Also, a sending end and a receiving end are created at both ends of the target network channel, the sending end sends a preset type of instruction to the receiving end through the target network channel, the preset type of the instruction can be related to the type of the network information to be transmitted by the target network, the instruction is encapsulated into a packet according to a custom protocol agreed between the devices at both ends of the target network channel, and the packet includes a sending timestamp. The receiving end returns a signaling to the sending end through the target network channel after receiving the packet, and the signaling includes a receiving timestamp of the packet received by the receiving end. The sending end calculates the difference between the receiving timestamp and the sending timestamp, and compares the difference with a preset delay threshold corresponding to the target network channel. For example, when the difference is less than the preset delay threshold, it is determined that the network communication quality is good; when the difference exceeds a first range of the preset delay threshold, it is determined that the network communication quality is fair; and when the difference exceeds a second range of the preset delay threshold, it is determined that the network communication quality is poor. The first range is less than the second range. In some embodiments, when the network communication quality is a first quality, for example, good or fair, it can be determined that the network communication quality of the target network channel is normal; and when the network communication quality is a second quality, for example, poor, it can be determined that the network communication quality of the target network channel is abnormal.

[0351] 714, when the communication status of the target network channel obtained by the evaluation is normal, at least one of the first device and the second device determines that the pairing of the first device and the second device is successful.

[0352] 715, when the communication status of the target network channel obtained by the evaluation is abnormal, at least one of the first device and the second device determines that the pairing of the first device and the second device fails.

[0353] The target network channel can include multiple target network channels. Whether the pairing between the first device and the second device is successful generally needs to consider the communication status of multiple target network channels.

[0354] For example, network connectivity is a key indicator for evaluating the communication status. If the network connectivity of all target network channels is normal, it can be determined that the pairing of the first device and the second device is successful; if the network connectivity of one or more target network channels is abnormal, it can be determined that the pairing of the first device and the second device fails. For example, network communication quality is a key indicator for evaluating the communication status. If the network communication quality of all target network channels is normal, it can be determined that the pairing of the first device and the second device is successful; if the network communication quality of one or more target network channels is abnormal, it can be determined that the pairing of the first device and the second device fails.

[0355] In some embodiments, for example, when the first device or the second device is configured to have the operator's authority, the pairing success of the first device and the second device can be evaluated in combination with the priority of the network information transmitted by the target network channel and the communication state of the target network channel. Some target network channels are used to transmit network information with high priority, such as motion control information, energy control information, and video information. Some target network channels are used to transmit network information with low priority, such as audio information and human-computer interaction information. In order to improve the pairing success rate, the system can have a high degree of redundancy. For example, as long as the communication state of all target network channels used to transmit network information with high priority is normal, and the communication state of all target network channels used to transmit network information with low priority is abnormal, it can be determined that the first device and the second device are paired successfully. For example, as long as the communication state of any target network channel used to transmit network information with high priority is abnormal, it can be determined that the first device and the second device are paired unsuccessfully.

[0356] After 714 or 715, a pairing success or pairing failure prompt can be performed in the first device and the second device.

[0357] In 711, at least one of the first device and the second device can obtain the configured control authority or the doctor inputted patient information, and can determine the information of the target network channel expected to be established between the first device and the second device based on the control authority or the patient information, and send the determined information of the target network channel to the server. The server can also obtain the configured control authority or the doctor inputted patient information, and can determine the information of the target network channel expected to be established between the first device and the second device based on the control authority or the patient information.

[0358] In some embodiments, 711 can be replaced by that the server sends the device network information of the opposite device to at least one of the first device and the second device. The device network information includes all possible network channels that can be created.

[0359] 712 can be replaced by that at least one of the first device and the second device receives the device network information of the opposite device, and can determine the device network information associated with the target network channel from the device network information based on the configured control authority or the patient information, and further create the target network channel with the opposite device based on the device network information of the target network channel for communication.

[0360] 712 Alternatively, at least one of the first device and the second device receives device network information of the counterpart device, and creates all network channels with the counterpart device for communication based on the device network information. 713 Alternatively, at least one of the first device and the second device determines a target network channel from the all network channels based on the configured control authority or the doctor-patient information, and further evaluates the communication status of the target network channel. In this embodiment, the first device and the second device create all network channels, but only the communication status of the necessary network channel, i.e., the target network channel, is detected.

[0361] In some embodiments, upon determining that the first device and the second device are successfully paired, the corresponding device, such as the first device or the second device, can allow or prohibit the use of the corresponding control authority according to the different network communication qualities of the plurality of target network channels; or can allow or disable the implementation of the corresponding surgical procedure according to the different network communication qualities of the plurality of target network channels. For example, when the network communication quality of the plurality of target network channels used for transmitting motion control information, energy control information, and video information is better, the relatively dangerous surgical procedure, such as surgery on arteries and organs such as lungs, livers, kidneys, and stomachs, can be allowed to be performed; for example, when the network communication quality of any one of the plurality of target network channels used for transmitting motion control information, energy control information, and video information is general, only the relatively safe surgical procedure, such as surgery on skin and fat, can be allowed to be performed.

[0362] In some embodiments, the control authority of the remote doctor console over the patient surgery platform includes, but is not limited to, the control authority over at least one of the slave robot, the image platform, and the energy platform.

[0363] For example, the control authority includes operator authority and observer authority, and the operator authority has a higher level than the observer authority. The operator authority covers the observer authority, i.e., the observer authority is part of the operator authority. Therefore, the network channels required to be enabled by the operator authority include the network channels required to be enabled by the observer authority, and the number of the network channels required to be enabled by the operator authority is more than the number of the network channels required to be enabled by the observer authority.

[0364] In some embodiments, the observer permission is applicable to an observer identity, and is mainly applied to a teaching scenario. A trainee can learn by observing a doctor's operation implementation process, and the doctor can also observe the trainee's operation implementation process to guide. The observer permission is mainly reflected in the image platform. When configuring the observer permission, a network channel between the remote doctor console and the image platform usually needs to be created. The image platform can process multimedia interactive information, and can also process human-computer interface interactive information. The multimedia interactive information includes, for example, video information and audio information, and the human-computer interface interactive information includes, for example, annotation information or annotation information of a surgical image collected by a doctor on an endoscope and displayed on a display screen of the remote doctor console. The video information can be sourced from an endoscope or an external camera electrically connected to the image platform and mounted on a slave robot. The external camera is used to monitor the working state of at least part of the equipment of the patient operation platform. After the video information and the human-computer interface interactive information are processed by the image platform, they are transmitted to the remote doctor console through the network channel between the image platform and the remote doctor console and displayed on the display screen of the remote doctor console. The audio information can be sourced from an audio device electrically connected to the image platform and mounted on the slave robot. The audio device includes an input device such as a microphone and an output device such as a loudspeaker. The remote doctor console also includes an audio device. The input device of the remote doctor console and the slave robot can collect sound input such as voice talkback of both ends of the user, and after processing such as noise reduction by the image platform, the sound input is transmitted to the output device of the opposite end device through the network channel between the remote doctor console and the image platform and played. Different network channels can be created between the remote doctor console and the image platform to transmit different types of multimedia interactive information according to the types of the multimedia interactive information. For example, under the observer permission, when different network channels are created, three network channels can be created respectively, one for transmitting video information, one for transmitting human-computer interface interactive information, and one for transmitting audio information.

[0365] In some embodiments, the operator authority is applicable to an operator identity, and the operator includes a physician who performs a surgery. The operator authority can be embodied in various devices in a patient surgery platform, including an imaging platform, a slave robot, and an energy platform. The physician performing the surgery can include motion control of medical instruments (including energy instruments) mounted on the slave robot under a surgical field provided by the imaging platform, and energy control of the energy instruments mounted on the slave robot through control of the energy platform. During the surgery, the physician can annotate or comment on the surgery images, or communicate with a physician or a patient on the side of the slave robot. Therefore, under the operator authority, the remote physician console can create multiple network channels with the imaging platform, the slave robot, the energy platform, and auxiliary devices for transmission of different types of network information. In some embodiments, the remote physician console and any device can create multiple network channels according to different types of signals transmitted therebetween or different physical communication links. In some embodiments, for example, for a laparoscopic slave robot or an interventional slave robot, the slave robot includes multiple manipulators, each of which is used to mount a medical instrument, and the motion control of the medical instruments by the remote physician console is mainly motion control of the manipulators, and the motion control of different manipulators by the remote physician console is independent of each other, and their physical communication links are different, so that multiple motion control network channels associated with different manipulators can be created when the motion control network channel between the remote physician console and the slave robot is created. In some embodiments, the energy platform can include processing of different types of energy control signals, and multiple energy control network channels associated with different types of energy control signals can be created when the energy control network channel between the remote physician console and the energy platform is created.

[0366] In some embodiments, different control authorities can be configured more finely. When a certain specific authority is allowed, any device can open or create a network channel corresponding to the authority. When a certain specific authority is prohibited, any device can close or destroy a network channel corresponding to the authority. Disabling a certain specific authority can help to shield interference on the one hand, and to release bandwidth resources when the network bandwidth is insufficient on the other hand. For example, when the visual authority with high network bandwidth occupancy is disabled, the released bandwidth resources can help to ensure transmission of other information.

[0367] In some embodiments, for the observer authority, the visual authority, the audible and speakable authority, and the annotatable authority can be included for a first network channel for transmission of video information, a second network channel for transmission of audio information, and a third network channel for transmission of human-computer interface interaction information, respectively. For example, the observer authority can be configured to include the visual authority and the audible and speakable authority, and the first network channel and the second network channel are opened or created, and the third network channel is closed or destroyed.

[0368] In some embodiments, for operator permissions, partial operator permissions and full operator permissions can be included, both of which can override observer permissions. A remote physician console can obtain partial operator permissions to a patient surgical platform, which is suitable for use when multiple remote physician consoles are needed to cooperatively manipulate a patient surgical platform to perform surgery. A remote physician console can also obtain full operator permissions to a patient surgical platform, which is suitable for use when a single remote physician console is needed to independently manipulate a patient surgical platform to perform surgery.

[0369] In some embodiments, partial operator permissions include, for example, permissions to control a subset of manipulators or a subset of types of medical instruments from a slave robot. For example, a slave robot can include a first manipulator, a second manipulator, a third manipulator, and a fourth manipulator, one of which can be equipped with a medical instrument, and the medical instruments equipped by different manipulators can be the same or different, partial operator permissions can be configured such that a remote physician console can only control the first and second manipulators, and the third and fourth manipulators not configured can be configured to be controlled by one or more other remote physician consoles, wherein the configuration of partial operator permissions can be implemented based on the unique identification of the manipulators, such as the number of the manipulators. For another example, the medical instruments can include first, second, third, and fourth types of medical instruments, the types of medical instruments are recorded in the storage device thereof, and are identified by a read-write device provided in the manipulator when the medical instrument is installed to the manipulator of the slave robot, and can be sent to the remote physician console through a motion control network channel between the slave robot and the remote physician console. The remote physician console can determine whether it can control the medical instrument according to the type of the medical instrument obtained, wherein partial operator permissions can be configured such that a remote physician console can only control the first and second types of medical instruments, and the third and fourth types of medical instruments not configured can be configured to be controlled by one or more other remote physician consoles. For example, the first type of medical instrument is surgical forceps, the second type of medical instrument is an endoscope, the third type of medical instrument is an electrotome, and the fourth type of medical instrument is an anastomat.

[0370] In some embodiments, full operator permissions include, for example, permissions to control all manipulators or all types of medical instruments from a slave robot.

[0371] In some embodiments, after 715, when prompting the pairing failure, the target network channel with abnormal communication status can be prompted to facilitate the doctor to detect or maintain the target network channel, or to facilitate the doctor to reconfigure the control authority to avoid the pairing failure caused by the target network channel with the corresponding abnormal communication status. For the latter, for example, in the scenario of a teaching doctor, when the doctor learns from the prompt of the pairing failure that the network channel corresponding to the visual permission of the observer authority is normal, and the network channel corresponding to the audible and speaking permission is abnormal, the doctor can reconfigure the observer authority to only include the visual permission to ensure the pairing success, and even if the network channel corresponding to the audible and speaking permission is closed or destroyed, the implementation of the teaching function will not be greatly hindered.

[0372] In some embodiments, when creating the target network channel with the peer device, different target network channels with different communication protocol standards can be created according to the different control authorities obtained. For example, when the control authority is the observer authority, the target network channel with the UDP protocol standard can be created, such as the multimedia network channel and the human-computer interface interaction network channel using the UDP protocol standard, which has faster signaling transmission speed. For another example, when the control authority is the operator authority, for the observer authority covered by the operator authority, the target network channel with the UDP protocol standard can be created, and for other authorities, the target network channel with the TCP protocol standard can be created, such as the motion control network channel and the energy control network channel using the TCP protocol standard, which has more reliable signaling transmission and less packet loss and out-of-order, and can ensure the reliable implementation of the remote surgery. Of course, the target network channels can also have unified communication protocol standards.

[0373] In some embodiments, the target network channel is created according to the doctor-patient information, which can include the type of surgery expected to be implemented by the doctor or the patient. For example, when the type of surgery is the laparoscopic surgery, the target network channel between the remote doctor console and the corresponding device of the laparoscopic slave robot can be created; when the type of surgery is the bronchial / interventional surgery, the target network channel between the remote doctor console and the corresponding device of the bronchial / interventional slave robot can be created. The doctor-patient information can include the surgical equipment required for the surgery. For example, when the doctor-patient information includes the surgical equipment required for the slave robot, the monitor, and the breathing machine, the target network channel between the remote doctor console and the slave robot, the monitor, and the breathing machine can be created; when the doctor-patient information includes the surgical equipment required for the slave robot, the monitor, and the breathing machine, the target network channel between the remote doctor console and the slave robot can be created.

[0374] In some embodiments, the configuration of control authorities can also be included in the entire pairing phase. Whether the control authorities are successfully configured can affect whether the teleoperation can be reliably implemented, and whether the control authorities are successfully configured is an important reference to measure whether the pairing is successful. Among the control authorities, the successful configuration of the operator authority is particularly important to ensure the reliable implementation of the teleoperation. In most cases, a teleoperation is mainly the control of the core equipment in the patient surgery platform, including both the motion control of the slave robot by the remote physician console and the energy control of the energy platform by the remote physician console. Therefore, the operator authority usually involves the motion control authority and the energy control authority. In some cases, however, the remote physician console that is paired with the patient surgery platform in real time can include one or more, and when multiple remote physician consoles are included, one or more remote physician consoles can be configured to have only observer authority, and one or more remote physician consoles can be configured to have operator authority. Among them, one remote physician console can be configured to have full operator authority, that is, both the motion control authority of the slave robot and the energy control authority of the energy platform are obtained by the remote physician console. One remote physician console can be configured to have partial operator authority, for example, including one of the motion control authority of the slave robot and the energy control authority of the energy platform is obtained by the remote physician console; or, including partial motion control authority of the slave robot is obtained by the remote physician console; or, including partial energy control authority of the energy platform is obtained by the remote physician console, wherein the energy platform can include multiple energy output links, and different energy output links can be controlled by different remote physician consoles, that is, the multiple energy output links can include multiple energy control authorities.

[0375] In a master-slave surgery robot system, the operator authority is usually transparent to the operation end and the execution end, that is, the operation end and the execution end usually know the operator authority they have, for example, the operation end knows which operations he can control the execution end to perform, and the execution end knows which operations he can perform controlled by the operation end. Among them, one of the operation end and the execution end is a remote physician console, and the other can be a patient surgery platform. For example, in master-slave control, the operation end is a remote physician console, and the execution end is a slave robot or an energy platform of a patient surgery platform; and in master-slave alignment, the operation end is a slave robot, and the execution end is a remote physician console.

[0376] In some embodiments, the successful configuration of the operator authority includes at least two aspects. The first aspect includes that the possession (i.e. permission) of the authority is configured successfully, indicating that the object desired to be operated can be operated. The second aspect includes that the non-possession (i.e. prohibition) of the authority is configured successfully, indicating that the object desired not to be operated cannot be operated. Whether the operator authority is configured successfully includes the detection of at least one of the two aspects. Based on this, for the remote physician console and the patient surgery platform that have established real-time pairing, the present disclosure provides a method for detecting whether the operator authority is configured successfully, the method comprising:

[0377] 721, at least one of the remote physician console and the patient surgery platform acquires the possession authority in the configured operator authority.

[0378] The possession authority mainly includes authority attribution, which includes association with the controlled object. The operator can only control the controlled object in the execution end, but cannot control the non-controlled object. From the perspective of the remote physician console, the controlled object can include a specific manipulator component in the slave robot or a manipulator component associated with a specific type of medical instrument desired to be controlled; the controlled object can include a specific energy output link in the energy platform desired to be controlled by the remote physician console. From the perspective of the patient surgery platform, the controlled object can include a specific manipulator component in the slave robot or a manipulator component associated with a specific type of medical instrument desired to be controlled by the remote physician console; the controlled object can include a specific energy output link in the energy platform desired to be controlled by the remote physician console. Thus, whether from the perspective of the remote physician console or from the perspective of the patient surgery platform, the authority attribution of the two ends is corresponding. Wherein, “specific” includes all or part.

[0379] Different authority attribution can be closely associated with the authority type of the operator authority. For example, when the controlled object includes a manipulator component, the authority type is usually associated with motion control authority; for another example, when the controlled object includes a specific energy output link, the authority type is usually associated with energy control authority. That is, when configuring the operator authority, the authority attribution is usually configured, and the associated authority type can be configured by default according to the authority attribution.

[0380] In addition, different types of control instructions are usually required for control based on the different authority types of the controlled object. For example, when the authority type is motion control authority, motion control instructions are required to control the controlled object; when the authority type is energy control authority, energy control instructions are required to control the controlled object.

[0381] 722, the remote physician console determines the controlled object from the patient surgery platform based on the authority attribution of the possession authority, and determines the target control instruction associated with the controlled object from the preset control instruction set.

[0382] Even if two slave robots in two patient surgery platforms are of the same type, due to differences in machining and mechanical assembly, the two slave robots can be slightly different in kinematics, and thus a preset control instruction set can be designed for each patient surgery platform. The preset control instruction set includes motion control instructions simulating the operation of the operation part of the remote physician console and energy control instructions simulating the operation of the foot pedal assembly of the remote physician console, and both of them can include at least one control instruction.

[0383] For example, in a laparoscopic slave robot, different motion control instructions can be designed for different manipulator assemblies in the slave robot. The motion control instructions are mainly used to control the manipulators in the manipulator assemblies to control the medical instruments assembled on the manipulators, and the motion control instructions are usually a pose instruction, which includes at least one of a position instruction and an attitude instruction.

[0384] In some embodiments, the slave robot includes a main arm and a plurality of manipulators connected to the distal end of the main arm. When the slave robot is powered on and initialized, the main arm is unfolded to drive each manipulator to a first initial pose fixed relative to a base coordinate system. When the medical instrument is assembled to the manipulator, the initialization operation of the manipulator on the medical instrument also enables the medical instrument to be kept in a second initial pose fixed relative to the base coordinate system, and the second initial pose is different from the first initial pose. The medical instrument includes an endoscope and a surgical instrument. Generally, the motion of the endoscope is performed relative to the base coordinate system, and the motion of the surgical instrument is performed relative to the endoscope coordinate system. However, when detecting whether the motion control authority is configured successfully, the control mode can be simplified, for example, the remote physician console can be defined to control the motion of all types of medical instruments relative to the base coordinate system.

[0385] The kinematic models of the manipulators in the same manipulator assembly of the same slave robot are basically the same, and the kinematic models of the medical instruments can be the same or different.

[0386] Assuming that the kinematic models of different medical instruments suitable for the slave robot are substantially the same, the target poses of the medical instruments installed on the manipulators can be pre-designed in the case that each medical instrument is at a second initial pose relative to the base coordinate system, in combination with the current joint parameters of each joint of the manipulator and the target joint increments of at least part of the joints, and based on the kinematic model of the manipulator and the kinematic model of the medical instrument. The target joint increments are pre-designed and can be a small value, for example, when the joint parameters are all joint angles, the target joint increments can be designed as 0.1°, 0.5°, 1°, 1.5°, 2°, or other values between any two of the foregoing, etc., to reduce the amplitude of the subsequent motion for verifying whether the motion control authority is successfully configured. Of course, the target pose of each manipulator assembly can also be a directly specified pose relative to the base coordinate system, and the target pose is different from the corresponding second initial pose, for example, the target pose is an upward, downward, leftward, rightward, forward or backward pose relative to the second initial pose. In this case, the target pose of each manipulator assembly will not change because the manipulator is installed with different medical instruments, that is, each manipulator assembly is associated with a relatively fixed target pose. The target pose designed here is the target motion control instruction for detecting whether the motion control authority is successfully configured. Further, in step 722, the steps of determining the controlled object from the patient surgery platform and determining the target control instruction associated with the controlled object from the preset control instruction set include: determining the controlled manipulator assembly from the slave robot, and determining the target motion control instruction associated with the controlled manipulator assembly from the preset control instruction set based on the controlled manipulator assembly.

[0387] Assuming that the kinematic models of different medical instruments suitable for the slave robot are substantially different, the target poses of different medical instruments installed on different manipulators can be pre-designed based on the similar method described above in the case that the manipulator and the medical instrument are at the corresponding initial pose state. When the same medical instrument is installed on different manipulators, the target poses of each manipulator assembly are different; when different medical instruments are installed on the same manipulator, the target pose of the manipulator assembly is also different. That is, the target pose of the manipulator assembly is associated with the medical instrument itself and the manipulator on which the medical instrument is installed. Further, in step 722, the steps of determining the controlled object from the patient surgery platform and determining the target control instruction associated with the controlled object from the preset control instruction set include: determining the controlled manipulator assembly from the slave robot, identifying the medical instrument in the controlled manipulator assembly, and determining the target motion control instruction associated with the controlled manipulator assembly from the preset control instruction set based on the controlled manipulator assembly and the medical instrument installed thereon.

[0388] In some embodiments, the energy control instruction is relatively simple. When the energy output link of the energy platform includes multiple energy output links, the energy control instruction includes multiple energy control instructions corresponding to the multiple energy output links. These energy control instructions can include switch control instructions, current regulation instructions, voltage regulation instructions, power regulation instructions, or frequency regulation instructions, etc. Further, in 722, the step of determining the controlled object from the patient surgery platform and determining the target control instruction associated with the controlled object from the preset control instruction set includes: determining the controlled energy output link from the energy platform, and determining the target energy control instruction associated with the controlled energy output link from the preset control instruction set.

[0389] 723, the remote physician console sends the target control instruction to the patient surgery platform.

[0390] The communication between the remote physician console and the patient surgery platform is realized through the network channel established between the two, and the communication includes the transmission of the target control instruction and the transmission of the actual execution result of executing the target control instruction.

[0391] 724, the patient surgery platform receives the target control instruction, and the controlled object of the patient surgery platform executes the target control instruction.

[0392] In some embodiments, the physician on the controlled object side can observe the execution of the target control instruction by the controlled object and notify the remote physician console or the patient surgery platform of the observation result, which includes execution success or execution failure. The execution success can indicate that the possession permission configuration in the operator permission is successful, and the execution failure can indicate that the possession permission configuration in the operator permission fails.

[0393] In some embodiments, referring to FIG. 36, the method can further include:

[0394] 725, after the controlled object executes the target control instruction, the patient surgery platform returns the actual execution result to the remote physician console.

[0395] Corresponding to the target control instruction being a motion control instruction, the actual execution result can include the actual pose of the corresponding manipulator component relative to the base coordinate system. Corresponding to the target control instruction being an energy control instruction, the actual execution result can include the actual energy output state of the corresponding energy transmission link, and the actual energy output state includes an actual switch state, an actual current output value, an actual voltage output value, an actual power output value, or an actual frequency output value.

[0396] 726, the remote physician console receives the actual execution result and detects whether the actual execution result matches the expected execution result.

[0397] The expected execution result can be an execution result determined based on the target control instruction in an ideal state. Corresponding to the target control instruction being a motion control instruction, the expected execution result can include an expected pose of the corresponding manipulator component relative to the base coordinate system. Corresponding to the target control instruction being an energy control instruction, the expected execution result can include an expected energy output state of the corresponding energy transmission link, the expected energy output state being an expected switch state, an expected current output value, an expected voltage output value, an expected power output value, or an expected frequency output value.

[0398] 727, upon detecting that the actual execution result matches the expected execution result, determining that the possession right configuration is successful.

[0399] The actual execution result matching the expected execution result includes at least one layer of meaning. This layer of meaning includes that the controlled object will act in response to the target control instruction. This layer of meaning includes that the controlled object moves in response to the motion control instruction, or the controlled object outputs energy in response to the energy control instruction.

[0400] The actual execution result matching the expected execution result can also include another layer of meaning. This layer of meaning includes that the controlled object accurately acts in response to the target control instruction. This layer of meaning includes that, corresponding to the target control instruction being a motion control instruction, the deviation between the actual pose and the expected pose is within a deviation threshold in the same coordinate system; or, corresponding to the target control instruction being an energy control instruction, the actual energy output state and the expected energy output state should be consistent, for example, being an open state, when the actual energy output state is an actual switch state and the expected energy output state is an expected switch state; or, the deviation between the actual energy output state and the expected energy output state is within a deviation threshold, when the actual energy output state is an actual current output value, an actual voltage output value, an actual power output value, or an actual frequency output value, and the expected energy output state is an expected current output value, an expected voltage output value, an expected power output value, or an expected frequency output value.

[0401] 728, upon detecting that the actual execution result does not match the expected execution result, determining that the possession right configuration fails.

[0402] Through 725-728, it can be more intelligently determined whether the possession right in the operator right is configured successfully.

[0403] In some embodiments, referring to FIG. 37, it can also be further detected whether the non-possession right in the operator right is configured successfully, and the method can include:

[0404] 731, the remote doctor console and the patient surgery platform acquire the non-possession right in the configured operator right.

[0405] The operator's permissions include owned permissions and unowned permissions, the unowned permissions can be acquired independently or according to the owned permissions in the operator's permissions. As described above, the permission attribution of the owned permissions is associated with the controlled objects in the patient surgery platform, and the permission attribution of the unowned permissions is associated with the non-controlled objects in the patient surgery platform. It is assumed that different non-controlled objects need to send control instructions corresponding to the non-controlled objects if the non-controlled objects can be controlled, for example, the non-controlled objects are manipulator assemblies, and motion control instructions need to be sent, or the non-controlled objects are energy output links, and energy control instructions need to be sent.

[0406] 732, the remote physician console determines the non-controlled objects from the patient surgery platform based on the permission attribution of the unowned permissions, and determines the target control instructions associated with the non-controlled objects from the preset control instruction set.

[0407] The design of the preset control instruction set of the non-controlled objects can refer to the design of the preset control instruction set of the controlled objects, which will not be repeated here.

[0408] For the slave robot, the kinematic models of different medical instruments are basically the same, in 732, the step of determining the non-controlled objects from the patient surgery platform and determining the target control instructions associated with the non-controlled objects from the preset control instruction set includes: determining the non-controlled manipulator assemblies from the slave robot, and determining the target motion control instructions associated with the non-controlled manipulator assemblies from the preset control instruction set based on the non-controlled manipulator assemblies.

[0409] For the slave robot, the kinematic models of different medical instruments are basically different, in 732, the step of determining the non-controlled objects from the patient surgery platform and determining the target control instructions associated with the non-controlled objects from the preset control instruction set includes: determining the non-controlled manipulator assemblies from the slave robot, identifying the medical instruments in the non-controlled manipulator assemblies, and determining the target motion control instructions associated with the non-controlled manipulator assemblies from the preset control instruction set based on the non-controlled manipulator assemblies and the medical instruments assembled therewith.

[0410] In 732, the step of determining the non-controlled objects from the patient surgery platform and determining the target control instructions associated with the non-controlled objects from the preset control instruction set includes: determining the non-controlled energy output links from the energy platform, and determining the target energy control instructions associated with the non-controlled energy output links from the preset control instruction set.

[0411] 733, the remote physician console sends the target control instructions to the patient surgery platform.

[0412] 734, the patient surgery platform receives the target control instructions, and the non-controlled objects of the patient surgery platform execute the target control instructions.

[0413] In the case that the non-possessory permission configuration is successful, the non- controlled object usually does not receive the target control instruction, or even if it can receive the target control instruction, it does not execute the target control instruction. The reasons that the non- controlled object does not receive the target control instruction include that a network channel between the remote physician console and the non-controlled object is not created according to the configuration of the control permission, or even if the network channel between the remote physician console and the non-controlled object is created, the target control instruction is not transmitted between the two according to the configuration of the control permission. Accordingly, in 734, the patient surgery platform can not receive the target control instruction, or the non-controlled object of the patient surgery platform can not execute the target control instruction. That is, in any case, the state of the non-controlled object does not change when the non-possessory permission configuration is successful, including that the motion state does not change or the energy output state does not change. When the non-possessory permission configuration fails, the state of the non-controlled object changes, including that the motion state changes or the energy output state changes. In the abnormal state, the reasons that cause the non-possessory permission configuration to fail can include that the transmission of the target control instruction in the network channel is disturbed, or the network channel between the remote physician console and the patient surgery platform is incorrectly created.

[0414] Whether the state of the non-controlled object changes can be determined by the observation of the physician on the side of the non-controlled object. The physician can inform the remote physician console or the patient surgery platform of the observation result, so as to artificially determine whether the non-possessory permission configuration is successful.

[0415] In some embodiments, referring to FIG. 17, the method can further include:

[0416] 735, after the non-controlled object executes the target control instruction, the patient surgery platform returns an actual execution result to the remote physician console.

[0417] Whether the non-controlled object executes the target control instruction or not, the patient surgery platform can return an actual execution result to the remote physician console. The actual execution result includes whether the state of the non-controlled object changes.

[0418] 736, the remote physician console receives the actual execution result and detects whether the actual execution result matches an expected execution result.

[0419] When the non-possessory permission, the expected execution result associated with the target control instruction usually includes that the state of the non-controlled object does not change. The state of the non-controlled object not changing includes that the motion state does not change or the energy output state does not change. The motion state not changing means that the pose of the non-controlled manipulator assembly does not change; the energy output state not changing means that the switch state of the non-controlled energy platform does not change, the current output value does not change, the voltage output value does not change, the power output value does not change, or the frequency output value does not change.

[0420] 737, when detecting that the actual execution result matches the expected execution result, it is determined that the non-possessed permission configuration is successful.

[0421] For example, when the execution result corresponds to no change in the state of the uncontrolled object, and the expected execution result corresponds to no change in the state of the uncontrolled object, it is determined that the non-possessed permission configuration is successful.

[0422] 738, when detecting that the actual execution result does not match the expected execution result, it is determined that the non-possessed permission configuration fails.

[0423] For example, when the execution result corresponds to a change in the state of the uncontrolled object, and the expected execution result corresponds to no change in the state of the uncontrolled object, it is determined that the non-possessed permission configuration fails.

[0424] Through 735-738, it can be determined whether the non-possessed permission in the operator's permission is configured successfully.

[0425] In some embodiments, when one or more remote physician consoles are paired with the patient surgery platform in real time, for any remote physician console, when both the possessed permission and the non-possessed permission of the remote physician console are configured successfully, it can be confirmed that the remote physician console is actually paired successfully with the patient surgery platform; when the possessed permission or the non-possessed permission of the remote physician console fails to be configured, it can be confirmed that the remote physician console is actually paired unsuccessfully with the patient surgery platform. Such pairing success detection can fully guarantee the smooth implementation of the super-remote surgery.

[0426] <Switching of pairing>

[0427] In some embodiments, a device (local device) can be paired with multiple remote devices. For example, one remote medical console can be paired with multiple patient surgery platforms. For another example, one patient surgery platform can be paired with multiple remote medical consoles. The medical professional can configure the one device to establish pairing with multiple remote devices simultaneously, and the established pairings can be pre-arranged pairings. The device can obtain pairing information of the multiple remote devices from a server, and the pairing information can include pairing orders of the remote devices and device network information. The pairing orders can also be obtained by the device from itself since the device can record the pairing orders of the remote devices. The pairing orders can be default, for example, the pairing order of the earlier configured pairing is earlier, and the pairing order of the later configured pairing is later. The pairing orders can be preset, for example, the first remote device can be preset to have an earlier pairing order, and the second remote device can be preset to have a later pairing order. The pairing orders can be determined by pre-arranged pairing time, and the pairing order of the earlier pre-arranged pairing time is earlier. The pairing orders can be determined according to the emergency degree of surgery, for example, in the medical professional-patient information input in the same time range (e.g., 0-30 minutes), the higher the emergency degree of surgery, the earlier the default configured pairing order. When the time range is exceeded, the pairing order can be set according to the default pairing mode.

[0428] In some surgery scenarios, to ensure the safety and privacy of surgery, at the same time, the one device is paired with at most one remote device in real time and is pre-arranged with other remote devices. For example, one remote medical console is paired with at most one patient surgery platform in real time and is pre-arranged with other patient surgery platforms. For another example, one patient surgery platform is paired with at most one remote medical console in real time and is pre-arranged with other remote medical consoles.

[0429] In these surgery scenarios, the present disclosure provides a method for quickly switching pairing, as shown in FIG. 38, which includes the following steps:

[0430] 801. The device obtains pairing information of multiple remote devices with which the device has established pre-arranged pairings.

[0431] The pairing information includes pairing orders of the multiple remote devices and device network information. The multiple remote devices include a first remote device and a second remote device.

[0432] 802. When the device determines that the first remote device in the multiple remote devices has a priority pairing order based on the pairing orders in the pairing information, the device establishes real-time pairing with the first remote device based on the device network information of the first remote device in the pairing information.

[0433] 803, when detecting that the switching pairing condition is satisfied, the device cancels the real-time pairing with the first counterpart device, and establishes the real-time pairing with the second counterpart device based on the device network information of the second counterpart device in the pairing information.

[0434] When the device cancels the real-time pairing with the first counterpart device, the device can update the pairing information, for example, can mark that the first counterpart device has been paired, or can delete the device network information of the first counterpart device to avoid real-time pairing with the first counterpart device again. The device can send the updated pairing information to the server for the server to maintain the pairing table, such as marking the pairing state of the related device.

[0435] In this embodiment, the switching pairing condition is satisfied to allow the device to establish a new real-time pairing with the second counterpart device, which can prevent accidental interruption of the ongoing super-remote surgery of the first counterpart device and ensure the continuous implementation of the super-remote surgery.

[0436] In some embodiments, the switching pairing condition is satisfied at least in two aspects: the device and the first counterpart device satisfy the condition of canceling real-time pairing; and the device and the second counterpart device satisfy the condition of establishing real-time pairing.

[0437] In some embodiments, the device and the first counterpart device satisfying the condition of canceling real-time pairing includes at least one of the following: completion of surgery, abnormal interruption of surgery, and agreement of both devices to switch pairing.

[0438] Regarding the completion of surgery, at least one of (1) to (3) is included: (1) the necessary physical accessories for surgery are detached from the slave robot or from the patient's body and the delay time is reached, or the slave robot is recovered to the preoperative preparation state and the delay time is reached; (2) the image platform is closed and the delay time is reached, or the field of view provided by the imaging device such as an endoscope in the slave robot is switched from a surgical field of view to a non-surgical field of view and the delay time is reached; (3) the doctor or patient leaves the surgical environment and the delay time is reached.

[0439] Regarding (1), for each type of master-slave surgical robotic system, for example, for a laparoscopic surgical robotic system or an interventional surgical robotic system, the acquiring the surgical necessary physical accessories from the slave robot includes at least one of: acquiring all medical instruments from the slave robot (of manipulator), acquiring the sterile drape from the slave robot, acquiring the puncture device that guides the medical instrument to insert into the patient's body from the slave robot (of manipulator). The acquiring the surgical necessary physical accessories from the patient's body includes at least one of: acquiring all medical instruments from the patient's body, acquiring the puncture device from the patient's body, and acquiring auxiliary equipment such as a monitor or a breathing machine from the patient's body. The acquiring the slave robot to recover to the preoperative preparation state includes acquiring the mechanical arm of the slave robot to recover to the retracted state, or the support feet of the slave robot chassis to reset.

[0440] Regarding (2), in the surgery, the provision of the image is crucial, and the image platform closing to the delay time often means that the surgery is completed. Among them, the acquiring the field of view from the surgical field to switch to the non-surgical field to the delay time includes: acquiring the field of view from the surgical field to switch to the non-surgical field includes acquiring the endoscope from the patient's body, or the image collected by the endoscope does not include human tissue or organs.

[0441] Regarding (3), the acquiring the doctor or the patient to evacuate the surgical environment to the delay time includes: acquiring the doctor to evacuate the surgical environment includes acquiring the doctor to leave the remote doctor console. Acquiring the patient to evacuate the surgical environment includes acquiring the patient to evacuate from the operating bed or the operating bed carrying the patient to be removed from the operating room.

[0442] Regarding the surgical abnormal interruption, it includes detecting the network state abnormality of the network channel, for example, including detecting the network connectivity abnormality or the network communication quality abnormality. Further, the confirmation of the network state abnormality can be superimposed with a judgment condition, which includes, for example, the cumulative time of the network state abnormality reaching a time threshold, or the cumulative number of the network state abnormality reaching a number threshold.

[0443] Regarding the two-end device agreeing to switch the pairing, it includes that either end initiates a switch pairing request to the opposite end, the opposite end accepts the switch pairing request, and the opposite end device can notify the end that initiates the switch pairing request.

[0444] In some embodiments, the device and the second opposite end device meet the real-time pairing establishment condition includes: the second opposite end device and the device have established a reservation pairing. The establishment of the reservation pairing between the two can speed up the speed of the switch pairing.

[0445] In some embodiments, when the device pairs with any peer device in real time, the device further establishes a network channel with the peer device. The establishing of the network channel includes establishing all network channels and establishing only target network channels based on configured control permissions. When the device un-pairs with any peer device in real time, the device further destroys all network channels or target network channels established with the peer device.

[0446] In some embodiments, when the device pairs with any peer device in real time, the device and the peer device further acquire configured control permissions. At least one of the device and the peer device can create a target network channel based on the acquired configured control permissions. When the device un-pairs with any peer device in real time, the device and the peer device further release the configured control permissions.

[0447] In some embodiments, in 802, the device establishes a real-time pairing with a first peer device, and further establishes a first master-slave mapping relationship between the device and the first peer device. In 803, the device un-pairs with the first peer device, and further breaks the first master-slave mapping relationship between the device and the first peer device. When the first master-slave mapping relationship is broken, master-slave control is interrupted, including master-slave motion control or master-slave energy control. In order to ensure safety, when master-slave motion control is interrupted, the manipulator assembly can be configured to maintain at least one of the current position and the current posture. When master-slave energy control is interrupted, the energy platform can be configured to reduce or turn off energy output. In 803, the device establishes a real-time pairing with a second peer device, and further establishes a second master-slave mapping relationship between the device and the second peer device. For example, the device and the peer device can include a remote physician console and a slave robot or an energy platform. When the device and the peer device perform motion control or energy control, they usually control each other according to the master-slave mapping relationship. Motion control occurs between the remote physician console and the slave robot. The remote physician console can control the motion of the slave robot, for example, in master-slave control. The slave robot can also control the motion of the remote physician console, for example, in master-slave alignment of a laparoscopic surgical robot system. Energy control occurs between the remote physician console and the energy platform. The remote physician console controls the on-off, current adjustment, voltage condition, power adjustment, or frequency adjustment of the energy platform.

[0448] When the device is paired with the first peer device in real time, the device and the first peer device establish a first master-slave mapping relationship, and control is performed through the first master-slave mapping relationship. When the device cancels the real-time pairing with the first peer device and establishes real-time pairing with a second peer device, because the device types of the first peer device and the second peer device can be different, different device types usually correspond to different mechanical arm configurations or energy output modes, etc., and control between the two through the first master-slave mapping relationship can be inappropriate, and it is necessary to reestablish a master-slave mapping relationship adapted to the device and the second peer device. Assuming that the reestablished master-slave mapping relationship is a second master-slave mapping relationship, in order to ensure the smooth implementation of the remote surgery, it is necessary to switch the first master-slave mapping relationship to the second master-slave mapping relationship when the device switches from real-time pairing with the first peer device to real-time pairing with the second peer device.

[0449] In some embodiments, the master-slave mapping relationship between the device and the peer device includes at least one of first master-slave mapping logic and second master-slave mapping logic. The first master-slave mapping logic includes master-slave mapping logic in which the device controls the peer device. The second master-slave mapping logic includes master-slave mapping logic in which the peer device controls the device.

[0450] In some embodiments, the master-slave mapping logic comprises master-slave motion mapping logic. Take an example of a device being a remote physician console and a peer device being a slave surgical robot. Different slave surgical robot types usually have different manipulator assembly structures, and their controlled structures such as manipulator assemblies for performing surgery are usually different. Different manipulator assemblies can involve different link parameters. In master-slave control, different kinematic models are determined according to different manipulator assemblies, and then the manipulator assemblies are controlled based on the corresponding kinematic models. Different slave surgical robot types may, due to different manipulator assembly structures, need to control the manipulator assemblies according to different motion scaling ratios when executing the same pose instruction. Different slave surgical robot types often include different numbers or different arrangements of manipulator assemblies. In master-slave control, an operating part usually controls one manipulator assembly. When it is necessary to switch the manipulator assembly controlled by the operating part, the manipulator assembly may need to be switched according to specific instrument switching logic. Different slave surgical robot types may correspond to different instrument switching logic. In master-slave control, different slave surgical robot types may have different instrument operation logic for controlling different medical instruments to move. For example, in a single-hole laparoscopic slave surgical robot, controlling the movement of two operating parts simultaneously generates a control instruction for controlling the movement of an endoscope. In a multi-hole laparoscopic slave surgical robot, controlling the movement of one operating part generates a control instruction for controlling the movement of an endoscope. Thus, the master-slave motion mapping logic can include at least one of the kinematic models, motion scaling ratios, instrument switching logic, and instrument operation logic involved in master-slave control. The kinematic models involved in master-slave control at least include the kinematic models of the controlled structures such as the manipulator assemblies in the slave surgical robot. They can also include the kinematic models of the mechanical arms in the operating parts of the remote physician console.

[0451] In some embodiments, the master-slave mapping logic comprises master-slave energy mapping logic. Take an example of a device being a remote physician console and a peer device being an energy platform. Different energy platform types can have different types of energy output forms. The remote physician console sends energy control instructions to the energy platform, and the energy platform outputs energy based on the energy control instructions, which includes specific conversion relationships. For example, some conversion relationships include current size conversion, some conversion relationships include power size conversion, and some conversion relationships include frequency size conversion. Thus, the master-slave energy mapping logic can include energy output conversion relationships in master-slave control.

[0452] In some embodiments, for example in a laparoscopic surgical robot system, the remote physician console includes a first type of remote physician console and a second type of remote physician console. The first type of remote physician console has an operating part in the form of a mechanical link, and the second type of remote physician console has an operating part in the form of magnetic navigation. For example, when the device refers to a remote physician console and the peer device refers to a slave surgical robot:

[0453] If the remote physician console is a first type remote physician console, the remote physician console can control the slave robot in master-slave (follow) mode, and the slave robot can also control the remote physician console in master-slave alignment. The former requires first master-slave mapping logic, and the latter requires second master-slave mapping logic, i.e., the master-slave mapping relationship includes the first master-slave mapping logic and the second master-slave mapping logic.

[0454] If the remote physician console is a second type remote physician console, the remote physician console can control the slave robot in master-slave (follow) mode, but the slave robot cannot control the remote physician console in master-slave alignment, and only the former requires first master-slave mapping logic, i.e., the master-slave mapping relationship can only include the first master-slave mapping logic.

[0455] In some embodiments, the device information of the remote physician console and the patient surgery platform can include the information included in the master-slave mapping logic described above. When registering with the server, these information can be stored in the server, and when pairing, these information can be sent to the opposite device. Further, the master-slave mapping relationship is established based on the master-slave mapping logic of at least one of the two ends.

[0456] In some embodiments, in the scenario where the device is a remote physician console and the opposite device is a patient surgery platform, when the device is paired with any opposite device in real time, the device also includes updating the UI interface in the display screen of the device. Specifically, when the device is paired with a first opposite device in real time, the display screen of the device displays a first UI interface; when the device switches from real-time pairing with the first opposite device to real-time pairing with a second opposite device, the display screen of the device displays a second UI interface. The first UI interface is associated with the information of the first opposite device, and the second UI interface is associated with the information of the second opposite device. Different opposite devices can have at least one difference in the number, type, and state of the devices, resulting in different information of different opposite devices, which needs to be accurately displayed accordingly. For example, the first opposite device includes a single-port endoscopic slave robot, and the second opposite device includes a multi-port endoscopic slave robot. At least the change in the model and state of the slave robot can cause the UI interfaces of the two to be different.

[0457] In some embodiments, in the case where the remote physician console has an operating part of a mechanical arm, when the device is paired with any opposite device in real time, in order to facilitate and quickly complete preoperative preparation, one or more remote physician consoles that have obtained operator permissions can be automatically aligned in master-slave mode. Referring to FIG. 39, the method of master-slave alignment includes:

[0458] 8031, the patient surgery platform sends the current pose of the medical instrument at the end of a manipulator assembly in the slave robot in the endoscope coordinate system to the remote physician console that has operator permissions for the manipulator assembly.

[0459] The operator authority mainly refers to a motion control authority. The transmission of the current pose can be based on a motion control network channel created by the remote physician console and the patient surgery platform according to the operator authority.

[0460] 8032, the remote physician console acquires the current pose of the medical instrument tip in the endoscope coordinate system sent by the patient surgery platform.

[0461] 8033, based on the current pose of the medical instrument tip in the endoscope coordinate system, the remote physician console converts the current pose of the medical instrument tip in the endoscope coordinate system into a target pose of the operating part in the display screen coordinate system of the remote physician console.

[0462] 8034, the remote physician console analyzes the target pose based on inverse kinematics to obtain target joint parameters of each joint of the mechanical arm in the operating part.

[0463] 8035, the remote physician console controls the motion of the corresponding joint in the mechanical arm based on the target joint parameters of each joint, so that the end of the operating part reaches the target pose in the display screen coordinate system, so that the pose of the operating part is aligned with the pose of the medical instrument tip, and the master-slave alignment is realized.

[0464] Through the above 8031-8035, the operating part and the medical instrument can be aligned in pose, which is beneficial to quickly complete the preoperative preparation, and is beneficial to the doctor to realize the hand-eye intuition when controlling the associated medical instrument through the operating part.

[0465] <Exchange of control authority>

[0466] In some embodiments, in some surgery scenarios such as teaching scenarios, multiple remote physician consoles can be paired with one patient surgery platform in real time at the same time, wherein different remote physician consoles are configured to have different control authorities. In a teaching scenario, it is assumed that a first remote physician console and a second remote physician console are paired with the same patient surgery platform in real time, the first remote physician console is configured to have one of the operator authority and the observer authority, and the second remote physician console is configured to have the other one of the operator authority and the observer authority. The remote physician console with the operator authority focuses on “teaching”, and the remote physician console with the observer authority focuses on “learning”; or, the remote physician console with the operator authority focuses on “practice”, and the remote physician console with the observer authority focuses on “guidance”. In order to quickly switch between teaching and learning, or between practice and guidance, the present disclosure further provides a method for quickly switching control authorities, as shown in FIG. 40, which comprises:

[0467] 901, the first remote physician console sends a control authority exchange request to the patient surgery platform.

[0468] The control authority exchange request is used to request to exchange control authority with a target second remote physician console in the at least one second remote physician console. The first remote physician console and the at least one second remote physician console have different control authorities, for example, the first remote physician console has a first control authority, and the at least one second remote physician console has a second control authority. The plurality of second remote physician consoles can have different second control authorities.

[0469] 902, the patient surgery platform receives the control authority exchange request sent by the first remote physician console and forwards it to the target second remote physician console.

[0470] Since the first remote physician console and the second remote physician console do not communicate directly, the patient surgery platform can act as a server to relay relevant information.

[0471] 903, the target second remote physician console receives the control authority exchange request and obtains a response instruction generated by the control authority exchange request.

[0472] The second remote physician console can generate and play a notification based on the received control authority exchange request to facilitate the physician to respond to the control authority exchange request based on the notification.

[0473] The content of the notification can be whether the second remote physician console accepts the control authority exchange request initiated by the first remote physician console. The notification can be played in the form of a UI interface or in the form of a sound. The physician can respond to the control authority exchange request through the operable control of the UI interface or through voice recognition. The response includes acceptance or rejection.

[0474] 904, the target second remote physician console sends the response instruction to the patient surgery platform.

[0475] 905, the patient surgery platform receives the response instruction, and when the response instruction is an affirmative response instruction, configures the first control authority of the first remote physician console as the second control authority of the target second physician console, and configures the second control authority of the target second remote physician console as the second control authority of the first physician console.

[0476] The response instruction includes an affirmative response instruction or a negative response instruction. The affirmative response instruction indicates that the target second remote physician console obtains the response instruction generated by accepting the control authority exchange request. The negative response instruction indicates that the target second remote physician console obtains the response instruction generated by rejecting the control authority exchange request.

[0477] Further, in response to the instruction being a negative response instruction, the patient surgery platform does not re-allocate the control authority of the first remote physician console and the target second remote physician console, both of which maintain the original control authority.

[0478] In S905, the patient surgery platform generally needs to receive a positive response instruction within a preset time range to re-configure the control authority of the first remote physician console and the target second remote physician console. The preset time range generally refers to the timing of the patient surgery platform from forwarding the exchange control authority request to the target second remote physician console, for example, the preset time range is within 3 minutes.

[0479] 906, the patient surgery platform sends the second control authority to the first remote physician console and sends the first control authority to the target second remote physician console.

[0480] 907, the first remote physician console acquires the second control authority, and the target second remote physician console acquires the first control authority.

[0481] In some embodiments, the first remote physician console includes a first display screen, and the second remote physician console includes a second display screen. The method can include:

[0482] The first remote physician console can generate a control on the display interface of the first display screen in response to acquiring a phase change of the surgery process, the control being used to trigger sending an exchange control authority request to the patient surgery platform. The phase change of the surgery process can involve the need for control authority exchange.

[0483] Further, in S901, the first remote physician console can send an exchange control authority request to the patient surgery platform in response to acquiring that the control is triggered. Of course, when the first remote physician console does not acquire that the control is triggered, it will not send an exchange control authority request to the patient surgery platform and still operate under the original control authority.

[0484] In some embodiments, the phase change of the surgery process includes at least one of the following: a change in the type of target surgery object in the surgery process; and a change in the type of target surgery action in the surgery process.

[0485] Exemplarily, the target surgery object types include important target surgery objects and non-important target surgery objects. For example, the important target surgery objects include at least one of an artery, a heart, a liver, a spleen, a kidney, a stomach, a lung, and a gallbladder. For example, the non-important target surgery objects include at least one of a vein, fat, a large intestine, and a small intestine. Generally, different types of target surgery objects can be associated with different experienced doctors to perform surgery. The type of the target surgery object can be determined by recognizing the surgery image transmitted by the first remote doctor console to the patient surgery platform.

[0486] Exemplarily, the target surgery action types include important target surgery actions and non-important target surgery actions. For example, the important target surgery actions include at least one of resection and hemostasis. For example, the non-important target surgery actions include at least one of suturing and drainage. Different types of target surgery actions can be associated with different experienced doctors to perform surgery. Generally, different types of medical instruments are usually used to perform different surgery actions. The type of the target surgery action can be determined by recognizing the type of the currently used medical instrument transmitted by the first remote doctor console to the patient surgery platform.

[0487] Generally, the important target surgery objects or the important target surgery actions require more experienced doctors to perform surgery, and the non-important target surgery objects or the non-important target surgery actions can be performed by relatively less experienced doctors. Therefore, the control authority exchange prompt can be given based on the stage change of the surgery process, so as to facilitate the doctor to determine whether to exchange the control authority according to the actual needs.

[0488] In other embodiments, the control can also be generated in the first display screen without being based on the stage change of the surgery process, and the control is used for the doctor to trigger the request for exchanging the control authority to the patient surgery platform according to the actual needs.

[0489] In some embodiments, to ensure that the request for exchanging the control authority sent by the first remote doctor console is reliable, the request can be encapsulated by using a custom protocol agreed between the first remote doctor console and the patient surgery platform, and the request is unpacked on the patient surgery platform side to realize authentication, which will not be repeated here.

[0490] In some embodiments, the prompt can be generated in the first remote doctor console or the target second remote doctor console based on the situation of the control authority exchange. When the control authority exchange is successful, the successful exchange of the authority is prompted on both ends. When the control authority is maintained, it is prompted on both ends that the authority is not exchanged.

[0491] In some embodiments, the two remote physician consoles capable of exchanging control authority are usually two remote physician consoles of the same device type, or two remote physician consoles with the same device function. In some embodiments, the method can include enabling or disabling the function of any remote physician console to initiate an exchange of control authority request based on identifying the device type or device function of the two remote physician consoles that are real-time paired with the patient surgery platform. For example, when the device type and / or device function of the two remote physician consoles do not match, the function of any remote physician console to initiate an exchange of control authority request is disabled; when the device type and / or device function of the two remote physician consoles match, the function of any remote physician console to initiate an exchange of control authority request is enabled. Different configuration interfaces can be provided for the two remote physician consoles according to such results, when enabled, a configuration interface including a control that can trigger an exchange of control authority request is provided; when disabled, a configuration interface not including a control that can trigger an exchange of control authority request is provided.

[0492] When the control authority is released, the remote physician console with operator authority disconnects the master-slave control, including disconnecting the master-slave motion control or the master-slave energy control. When the master-slave motion control is disconnected, the manipulator assembly can be configured to maintain at least one of the current position and attitude. When the master-slave energy control is disconnected, the energy platform can be configured to reduce or shut down energy output to ensure safety.

[0493] In this embodiment, the first remote physician console and the second remote physician console can maintain pairing with the patient surgery platform without being unpaired, and can seamlessly exchange control authority throughout the entire surgery process, improving the rapid switching of teaching and learning, or practice and guidance modes, and improving the efficiency of the surgery.

[0494] In some embodiments, a first remote physician console can also establish real-time pairing with multiple patient surgery platforms to watch or guide the surgery processes of different patient surgery platforms. However, at the same time, the first remote physician console can at most obtain operator authority for one of the patient surgery platforms, and the first remote physician console can simultaneously watch multiple videos sent by different patient surgery platforms on its display screen. Meanwhile, the patient surgery platforms can also establish real-time pairing with multiple second remote physician consoles, and similarly, a second remote physician console can at most obtain operator authority for one of the patient surgery platforms. When the first remote physician console and the second remote physician console have different control authorities for the same patient surgery platform, such as one having operator authority and one having observer authority, the two can exchange control authority based on the foregoing embodiments, which will not be repeated here.

[0495] In the above embodiments, at least one of the following can be performed based on the change of the pairing relationship or the change of the control authority: establishing a network channel between the two ends of the pairing, establishing a master-slave mapping relationship, performing pairing success detection, and updating a UI interface. For brevity, reference can be made to the foregoing description, and the details will not be repeated here.

[0496] The various devices described above, including the first device, the second device, the server, or the portable mobile terminal, can include a memory and at least one processor, which, when at least one program stored in the memory is processed by the at least one processor, causes the at least one processor to implement the method described in any one of the above embodiments.

[0497] The present disclosure also provides a computer-readable storage medium having instructions stored therein, which, when executed on at least one processor, can implement the steps in the various method embodiments described above. The memory 503 can include a high-speed RAM memory and can also include a non-volatile memory such as at least one disk memory. The processor can be a central processing unit CPU, or an application specific integrated circuit ASIC, or one or more integrated circuits configured to implement embodiments of the present disclosure, or a graphics processing unit GPU. The one or more processors included in the control device can be the same type of processor, such as one or more CPUs, or one or more GPUs, or different types of processors, such as one or more CPUs and one or more GPUs.

[0498] The present disclosure also provides a computer program product that, when executed on a device, causes the device to perform steps that can implement the steps in the various method embodiments described above.

[0499] The technical features of the above-described embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described, but as long as the combinations of the technical features do not contradict each other, they should be considered within the scope of the present disclosure.

[0500] The above-described embodiments only express several implementations of the present disclosure, which are described in detail and specifically, but should not be construed as limiting the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present disclosure, a number of modifications and improvements can be made, which are within the scope of the present disclosure. Therefore, the scope of protection of the present disclosure should be subject to the appended claims.

Claims

1. A method of device interconnection, characterized by, The method is applied to a first device in a remote surgery robot system, the system comprising a server, and the first device and a second device which can communicate with the server, the first device comprising one of a remote physician console and a patient surgery platform which can be manipulated by the remote physician console, the second device comprising the other of the remote physician console and the patient surgery platform, the method comprising: receiving second device information of a pairable second device sent by the server, the pairable second device comprising a second device registered to the server and having a matching custom protocol agreed with the first device; obtaining target second device information of a target pairable second device determined based on the second device information, and sending a first pairing request to the server, the first pairing request comprising first device information of the first device and the target second device information, the first pairing request being used to request to establish pairing with the target pairable second device; receiving device network information of the target pairable second device sent by the server when detecting that the first pairing request and a second pairing request sent by the target pairable second device match, the second pairing request comprising target second device information of the target pairable second device and target first device information of a target pairable first device determined by the target pairable second device based on received first device information of a pairable first device sent by the server, the second pairing request being used to request to establish pairing with the target pairable first device, the pairable first device comprising a first device registered to the server and having a matching custom protocol agreed with the target pairable second device, wherein the server does not forward the first pairing request to the target pairable second device, and the server does not forward the second pairing request to the first device when detecting whether the first pairing request and the second pairing request match; establishing pairing with the target pairable second device based on the device network information to realize direct communication with the target pairable second device.

2. The method of claim 1, further comprising: creating a target network channel based on the device network information; evaluating a communication state of the target network channel; determining that pairing between the first device and the second device is successful based on the evaluated communication state of the target network channel being normal.

3. The method of claim 2, the first device being a remote physician console and the second device being a patient surgery platform, the method further comprising: creating a corresponding target network channel based on a UDP protocol when control authority of the remote physician console is configured as observer authority; creating a corresponding target network channel based on the UDP protocol for observer authority included in operator authority when the control authority is operator authority, and creating a corresponding target network channel based on a TCP protocol for other authority included in the operator authority. 4.The method of claim 1, wherein the first device is a remote physician console and the second device is a patient procedure platform, and the method further comprises: obtaining a possession permission and a non-possession permission when the remote physician console is configured as an operator permission; determining a controlled object from the patient procedure platform based on a first permission attribution of the possession permission, the first permission attribution being associated with the controlled object in the patient procedure platform; determining a non-controlled object from the patient procedure platform based on a second permission attribution of the non-possession permission, the second permission attribution being associated with the non-controlled object in the patient procedure platform; and determining that the remote physician console and the patient procedure platform are successfully paired when the possession permission and the non-possession permission are successfully configured. 5.The method of claim 4, further comprising: sending a control instruction to the patient procedure platform; and determining that the possession permission and the non-possession permission are successfully configured when a controlled object responds to the control instruction and a state of the non-controlled object does not change. 6.The method of claim 1, wherein the second device is a plurality of second devices including a first counterpart device and a second counterpart device, and a pairing between the first device and the plurality of second devices is a pre-appointment pairing, and the method further comprises: obtaining pairing information of the plurality of second devices paired with the first device based on the pre-appointment pairing, the pairing information including a pairing order of the plurality of second devices and device network information; establishing a real-time pairing with the first counterpart device based on the device network information of the first counterpart device when the first counterpart device is determined to have a priority pairing order based on the pairing order; and in response to detecting that a switching pairing condition is met, releasing the real-time pairing with the first counterpart device and establishing a real-time pairing with the second counterpart device based on the device network information of the second counterpart device. After the step of establishing the pairing with the target second device based on the device network information, the method further comprises: establishing a master-slave mapping relationship between the first device and the target second device, the master-slave mapping relationship including at least one of a kinematics model, a motion scaling ratio, an instrument switching logic, and an instrument operation logic involved when the first device and the target second device perform master-slave control. The detecting that the first pairing request and a second pairing request sent by the target second device match includes detecting that first device information included in the first pairing request matches target first device information included in the second pairing request. Before the receiving the second device information of the second device that can be paired sent by the server, the method comprises: ​ ​ ​ ​ ​ ​ ​ 7. The method of claim 1, wherein, ​ ​ 8. The method of claim 1, wherein, ​ 9. The method of claim 1, wherein, ​ sending a first request to the server, the first request being used to request the server to send second device information of the pairable second device, the first request comprising a first screening condition for the server to match the second device information of the pairable second device from the second devices registered to the server, wherein the first screening condition is associated with the first device and the second device, and the first screening condition being satisfied comprises at least one of the following: the second device supporting a custom protocol of pairing matches the first device supporting a custom protocol of pairing; a device function supported by the second device matches a device function supported by the first device; a surgery type supported by the second device matches a surgery type supported by the first device; a device type of the second device matches a device type of the first device; and a planned surgery time of the second device does not conflict with a pre-arranged pairing time of the first device.

10. The method of claim 1, wherein, Before receiving the second device information of the pairable second device sent by the server, the method comprises: sending a first request to the server, the first request being used to request the server to send second device information of the pairable second device, the first request comprising a second screening condition for the server to match the second device information of the pairable second device from the second devices registered to the server, wherein the second screening condition is associated with the second device, and the second screening condition being satisfied comprises at least one of the following: an online state of the second device satisfies an expected online state; a network state of the second device satisfies an expected network state; a pairing state of the second device satisfies an expected pairing state; a device positioning of the second device satisfies an expected device positioning; a control authority allocation state of the second device satisfies an expected control authority; and a power-on self-test state of the second device satisfies an expected power-on self-test state, the power-on self-test state being associated with patient surgery information.

11. The method of claim 1, wherein, The first device is the remote physician console, the remote physician console comprising an operation part with a mechanical arm, and the second device is the patient surgery platform, the patient surgery platform comprising a slave robot, the slave robot comprising a plurality of manipulator assemblies, one of the manipulator assemblies comprising a medical instrument and another of the manipulator assemblies comprising an endoscope, and after the step of establishing pairing with the target second device based on the device network information, the method further comprises: obtaining a current pose of the medical instrument tip in an endoscope coordinate system of the endoscope; converting the current pose into a target pose of the operation part in a display screen coordinate system of a display screen in the remote physician console; controlling movement of the mechanical arm based on the target pose to align the pose of the operation part with the pose of the medical instrument tip.

12. A method of device interconnection, characterized by, The method is applied to a server in a remote surgery robot system, the system comprising the server, and a first device and a second device in communication with the server, the first device comprising one of a remote physician console and a patient surgery platform manipulatable by the remote physician console, the second device comprising the other of the remote physician console and the patient surgery platform, the method comprising: receiving a first pairing request sent by the first device, the first pairing request comprising first device information of the first device, and target second device information associated with a target second device determined by the first device based on received second device information of a pairable second device sent by the server, the first pairing request being used to request to establish pairing with the target second device, the pairable second device comprising a second device registered to the server and having a matched custom protocol agreed with the first device; the server not forwarding the first pairing request to the target second device; receiving a second pairing request sent by the target second device, the second pairing request comprising second device information of the target second device, and target first device information associated with a target first device determined by the target second device based on received first device information of a pairable first device sent by the server, the second pairing request being used to request to establish pairing with the target first device, the pairable first device comprising the first device registered to the server and having a same communication protocol with the server as a communication protocol used by the second device to communicate with the server; upon detecting that the first pairing request matches the second pairing request, sending device network information of a counterpart device to at least one of the first device and the target second device for the first device and the target second device to establish pairing, thereby enabling direct communication between the first device and the target second device.

13. A tele-surgical robotic system, comprising: The system comprises a server, and a first device and a second device in communication with the server, the first device comprising one of a remote physician console and a patient surgery platform manipulatable by the remote physician console, the second device comprising the other of the remote physician console and the patient surgery platform, the first device or the server being configured to implement the method of any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium has stored therein instructions which, when executed on at least one processor, implement the method of any one of claims 1 to 12.

Images