Remote surgical robot system

Abstract

The utility model relates to a remote surgical robot system. The system comprises a first receiving unit which is used for receiving second equipment information capable of being paired with second equipment; the display unit is used for displaying the second equipment information; the determining unit is used for determining target second equipment information of target second equipment based on the second equipment information and sending a first pairing request to the server; the second receiving unit is used for receiving the device network information of the target second device sent by the server when the server obtains a positive response to the first pairing request; and the establishing unit is used for establishing pairing with the target second equipment based on the equipment network information. And when one of the remote doctor console and the patient operation platform initiates a pairing request and the other initiates a positive response to the pairing request, the server sends the device network information of the opposite-end device to at least one of the devices initiating and accepting pairing so as to establish pairing, so that the pairing efficiency can be improved, and the user experience is improved. And the doctor can conveniently find the patient or the patient.

Description

TECHNICAL FIELD

[0001] The utility model relates to surgical robot technical field especially, it relates to a remote operation robot system. BACKGROUND

[0002] In recent years, surgical robot technology is booming, and different application scenarios of surgical robots such as orthopedic surgery robots, interventional surgery robots and endoscopic surgery robots have been produced. For doctors, they have the advantages of easy operation and high operation precision. For patients, they have the advantages of small trauma, light pain and fast recovery, and are widely accepted by doctors and patients.

[0003] The surgical robot includes a doctor console and a patient surgery platform. The doctor console includes a display screen and an operation part, and the patient surgery platform includes one or more manipulator assemblies, the manipulator assembly includes a manipulator and a medical instrument detachably assembled on the manipulator. When in use, the doctor generates a control instruction under the operation part by manipulating the operation part under the surgical field provided by the imaging device in the medical instrument displayed on the display screen, and the manipulator drives the medical instrument to perform a surgical action based on the control instruction.

[0004] With the maturity of high-speed communication network technologies such as 5G and the Internet, which have low latency and high bandwidth, remote surgery robot technology combining the advantages of surgical robot technology and high-speed communication network technology has emerged. Remote surgery robot technology allows doctors to be far away from patients, for example, it allows doctors to perform super-remote surgery on patients in different countries or different provinces and cities, thereby maximizing the breakthrough of space constraints on surgery implementation and effectively alleviating the problem of uneven distribution of medical resources.

[0005] The remote surgery robot system includes a remote doctor console and a patient surgery platform, which are usually deployed in different countries, different provinces and cities, or different jurisdictions of the same city. Before performing super-remote surgery, it is necessary to establish communication between the remote doctor console and the patient surgery platform, especially when a large number of remote doctor consoles and patient surgery platforms are deployed. How to conveniently and reliably establish communication between the target remote doctor console and the target patient surgery platform becomes a technical problem to be solved. SUMMARY

[0006] Therefore, it is necessary to provide a remote surgery robot system and a method for equipment interconnection, which can conveniently and reliably establish pairing between a remote doctor console and a patient surgery platform.

[0007] In one aspect, the utility model provides a kind of remote surgery robot system, the system includes server, and the first device and the second device can be communicated with the server, the first device includes one of remote physician console and patient surgery platform that can be manipulated by the remote physician console, the second device includes the other of the remote physician console and the patient surgery platform, the first device includes:

[0008] First receiving unit, for receiving the second device information of the second device matched by the server based on the first request and sent, the first request is used to request the server to send the second device information of the second device, the second device includes the second device registered to the server, and the second device information of the second device matched between the first device is agreed to custom protocol;

[0009] Display unit, for displaying the configuration interface with first control, also for displaying the second device information of the second device received by the first receiving unit in configuration interface, the first control is used to trigger the first device to send the first request to the server;

[0010] Determination unit, for determining the target second device information of the target second device expected to be paired based on the second device information displayed in the configuration interface of the display unit, sends the pairing request to the server, the pairing request includes the first device information of the first device and the target second device information, and the pairing request is used to request to establish pairing with the target second device;

[0011] Second receiving unit, for receiving the device network information of the target second device sent by the server when the server obtains the positive response instruction generated by the target second device to the server forwarding the pairing request positively responds;And

[0012] Establishment unit, for establishing pairing with the target second device based on the device network information received by the second receiving unit, to realize direct communication with the target second device.

[0013] Wherein, the second device information of the second device is displayed in the configuration interface through list control, and the second device information of one of the second device of the second device is associated with a list option of the list control;The determination unit is used to determine that the second device information of the target list option associated with the target list option is the target second device information of the target second device expected to be paired in response to detecting that the target list option is selected.

[0014] The second device information of the second device capable of pairing is displayed on the configuration interface through the first dragging control, the second device information of one of the second devices capable of pairing is associated with one of the first dragging controls, and the configuration interface further comprises a window control.

[0015] The establishing unit is further configured to establish a master-slave mapping relationship between the first device and the target second device, and the master-slave mapping relationship comprises at least one of kinematics models, motion scaling ratios, instrument switching logic, and instrument operation logic involved when the first device and the target second device perform master-slave control.

[0016] The remote surgery robot system has the following beneficial effects:

[0017] When one of the remote doctor console and the patient surgery platform initiates a pairing request and the other positively responds to the pairing request, the server sends device network information of a peer device to at least one of the devices initiating and accepting pairing, to establish pairing, thereby improving pairing efficiency and facilitating doctors to find patients or patients to find doctors. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 FIG. 1 is a structural schematic diagram of a master-slave surgery robot suitable for a remote surgery robot system according to the present application;

[0019] Figure 2 FIG. 2 is a structural schematic diagram of a surgery instrument of the master-slave surgery robot suitable for the remote surgery robot system according to the present application; Figure 1

[0020] Figure 3 FIG. 3 is a structural schematic diagram of another embodiment of the slave robot in the master-slave surgery robot suitable for the remote surgery robot system according to the present application;

[0021] Figure 4 FIG. 4 is a structural schematic diagram of another embodiment of the master-slave surgery robot suitable for the remote surgery robot system according to the present application;

[0022] Figure 5 FIG. 5 is a network topology diagram of an embodiment of the remote surgery robot system according to the present application;

[0023] Figure 6 FIG. 6 is a pairing scene schematic diagram of the remote surgery robot system according to the present application;

[0024] ​Figure 7 Flow chart of an embodiment of the device registration method of the present application;

[0025] Figure 8 Structure schematic view of an embodiment of the custom protocol for communication between devices of the present application;

[0026] Figure 9 Flow chart of an embodiment of the device interconnection method of the present application;

[0027] Figure 10 Flow chart of another embodiment of the device interconnection method of the present application;

[0028] Figure 11 Flow chart of another embodiment of the device interconnection method of the present application;

[0029] Figure 12 Flow chart of another embodiment of the device interconnection method of the present application;

[0030] Figures 13 to 15 Schematic view of the interactive interface of an embodiment of the device interconnection method of the present application;

[0031] Figures 16 to 19 Schematic view of the interactive interface of another embodiment of the device interconnection method of the present application;

[0032] Figures 20 to 22 Schematic view of the interactive interface of another embodiment of the device interconnection method of the present application;

[0033] Figure 23 Flow chart of another embodiment of the device interconnection method of the present application;

[0034] Figure 24 Flow chart of another embodiment of the device interconnection method of the present application;

[0035] Figures 25 to 28 Schematic view of the interactive interface of an embodiment of the device interconnection method of the present application;

[0036] Figures 29 to 31 Schematic view of the interactive interface of another embodiment of the device interconnection method of the present application;

[0037] Figures 32 to 34 Schematic view of the interactive interface of another embodiment of the device interconnection method of the present application;

[0038] Figure 35 Flow chart of another embodiment of the device interconnection method of the present application;

[0039] Figure 36 Flow chart of another embodiment of the device interconnection method of the present application;

[0040] Figure 37 a flow chart of another embodiment of the method for interconnecting the devices of the present application;

[0041] Figure 38 a flow chart of another embodiment of the method for interconnecting the devices of the present application;

[0042] Figure 39 a flow chart of another embodiment of the method for interconnecting the devices of the present application;

[0043] Figure 40 a flow chart of another embodiment of the method for interconnecting the devices of the present application. DETAILED DESCRIPTION

[0044] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the accompanying drawings. The preferred embodiments of the present application are shown in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described in the present application. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0045] It should be noted that when an element is referred to as being "provided on" another element, it can be directly on the other element or there can be an intervening element. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or there can be an intervening element. When an element is referred to as being "coupled" to another element, it can be directly coupled to the other element or there can be an intervening element. The terms "vertical", "horizontal", "left", "right", and similar expressions used in the present application are for illustrative purposes only and are not intended to be the only embodiment. The terms "distal end", "proximal end" used in the present application are directional terms commonly used in the medical device field, where "distal end" means the end farthest from the operator during surgery, and "proximal end" means the end closest to the operator during surgery. The terms "first / second" and the like used in the present application indicate a component and a plurality of components having common characteristics.

[0046] Unless otherwise defined, all technical and scientific terms used in the present application have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used in the present application are only for the purpose of describing the specific embodiments and are not intended to limit the present application. The term "and / or" used in the present application includes any and all combinations of one or more related listed items. The term "multiple" used in the present application includes two or more.

[0047] This utility model discloses a remote surgical robot system comprising a master-slave surgical robot suitable for performing ultra-long-distance surgeries. The master-slave surgical robot includes a remote physician control console and a patient surgical platform. The patient surgical platform may include different types of slave robots, including but not limited to single-port laparoscopic slave robots, multi-port laparoscopic slave robots, bronchial interventional slave robots, vascular interventional slave robots, and orthopedic slave robots. These different types of slave robots have different structural characteristics and may be similar or different in their applicability to various surgical procedures.

[0048] For example, in Figure 1 The illustrated master-slave surgical robot includes a remote doctor console 100 and a patient surgical platform 200, the latter comprising a single-port laparoscopic slave robot. The remote doctor console 100 sends control commands to the patient surgical platform 200 based on the doctor's instructions to control the platform, and also displays images acquired by the platform. The patient surgical platform 200 responds to the control commands sent by the remote doctor console 100 and performs corresponding operations; furthermore, it acquires images within the body.

[0049] The patient surgical platform 200 includes a robotic arm 210 and a manipulator assembly disposed on the robotic arm 210. The manipulator assembly includes a manipulator 220 disposed on the robotic arm 210 and, for example, a device disposed on the manipulator 220. Figure 2 The surgical instrument 230 is shown. The patient surgical platform 200 also includes a trocar 240 with a long axis 231 of the surgical instrument 230 mounted on it. When responding to control commands from the remote physician console 100, the robotic arm 210 is used to adjust the position of the surgical instrument 230, the manipulator 220 is used to drive the surgical instrument 230 to perform corresponding operations, and the end effector 232 of the surgical instrument 230 is used to extend into the body and perform surgical operations and / or acquire in vivo images through its distal end instrument.

[0050] Figure 3Another patient surgery platform is illustrated, which is 200' and includes a multi-lumen endoscope slave robot. The patient surgery platform 200' includes a mechanical arm, which includes a main arm 210' and a plurality of adjustment arms 220' arranged in the main arm 210' and orienting a platform 215', and a plurality of manipulator assemblies 230' arranged in the plurality of adjustment arms 220'. The main arm 210' can adjust the poses of the adjustment arms 220' and the manipulator assemblies 230', and the adjustment arms 220' can adjust the poses of the manipulator assemblies 230'. The manipulator assemblies 230' include manipulators 240' and medical instruments 250' detachably arranged in the manipulators 240'. The manipulator assemblies 230' include parallelogram mechanisms, and the manipulators 240' can be rotated around a remote center (RC) by parallelogram principle. The medical instruments 250' can be inserted into a patient through different puncture devices 500' respectively. Figure 1 The illustrated remote physician console 100 can also be used to manipulate the manipulator assemblies 230' in the patient surgery platform 200' as Figure 3 The illustrated remote physician console 100 can also be used to manipulate the manipulator assemblies 230' in the patient surgery platform 200' as

[0051] Illustratively, the physician can manipulate the manipulator assemblies 230' in the patient surgery platform 200' as Figure 4 An interventional surgery robot 300 is illustrated, which is also a natural orifice transluminal endoscopic surgery robot and includes a remote physician console and a patient surgery platform 320. The remote physician console includes a handle 310' and an image cart 330 connected with each other, and / or the patient surgery platform 320 also includes the image cart 330. The patient surgery platform 320 is connected with a catheter instrument 340, a sensor system 350, a control system 360 for controlling the catheter instrument 340, the sensor system 350 and the image cart 330, etc. The physician can trigger control instructions by operating the handle 310' when performing various procedures on a patient beside the patient surgery platform 320, and send the control instructions to the patient surgery platform 320 for driving, so as to control the catheter instrument 340 to advance, retract and bend, etc.

[0052] The patient procedure platform 320 can generally be moved to the bedside of a patient for engaging the catheter instrument 340 and controlling the catheter instrument 340 to be raised or lowered in a vertical direction, or to be translated in a horizontal direction, or to be moved in a non-vertical and non-horizontal direction under control instructions, so as to provide a better preoperative preparation angle for the operation of the catheter instrument 340. The control instructions can be triggered by a physician through operating the patient procedure platform 320, or by the physician directly through clicking or pressing a button provided on the patient procedure platform 320. Of course, in other embodiments, the control instructions can also be voice control or triggered by a force feedback mechanism.

[0053] As shown in Figure 4 Further, the patient procedure platform 320 can include a base 321, a sliding seat 322 movable up and down along the base 321, and two mechanical arms 323 fixedly connected to the sliding seat 322. The mechanical arm 323 can include a plurality of arm segments coupled at joints, which provide the mechanical arm 323 with a plurality of degrees of freedom, for example, seven degrees of freedom corresponding to seven arm segments. The end of the mechanical arm 323 is provided with a manipulator (not shown in the figure), which is used to engage the catheter instrument 340 and control the end of the catheter instrument 340 to be correspondingly bent and turned under the driving action of the manipulator. The two mechanical arms 323 can be identical or partially identical in structure, one mechanical arm 323 being used to engage the inner catheter instrument 341 and the other mechanical arm 323 being used to engage the outer catheter instrument 342. When installed, the outer catheter instrument 342 can be installed first, and then the catheter of the inner catheter instrument 341 is inserted into the catheter of the outer catheter instrument 342.

[0054] The sensor system 350 has one or more subsystems for receiving information about the catheter instrument 340. The subsystems can include a position sensor system, a shape sensor system for determining the position, orientation, velocity, speed, pose, and / or shape of the end of the catheter instrument 340 and / or along one or more segments of the catheter that can constitute the catheter instrument 340, and / or a visualization system for capturing images from the end of the catheter instrument 340.

[0055] The image cart 330 can be provided with a display system 331 and an irrigation system (not shown) and the like. The display system 331 is used to display images or representations of the surgical site and the catheter instrument 340 generated by the subsystems of the sensor system 350. Real-time images of the surgical site and the catheter instrument 340 captured by the visualization system can also be displayed. Image data from imaging techniques such as computed tomography (CT), magnetic resonance imaging (MRI), optical coherence tomography (OCT), and ultrasound and the like can also be used to present images of the preoperatively or intraoperatively recorded surgical site.

[0056] Wherein, the preoperative or intraoperative image data can be presented as two-dimensional, three-dimensional or four-dimensional (e.g. time-based or velocity-based information) images and / or as images from models created from preoperative or intraoperative image data sets, and virtual navigation images can also be displayed. In the virtual navigation images, the actual position of the catheter instrument 340 is registered with the preoperative images to present the operator with a virtual image of the catheter instrument 340 within the surgical site from the outside.

[0057] The control system 360 includes at least one memory and at least one processor. It can be understood that the control system 360 can be integrated in the patient surgery platform 320 or the image cart 330, or can be independently provided. The control system 360 can support wireless communication protocols such as IEEE 802.11, IrDA, Bluetooth, HomeRF, DECT and wireless telemetry and the like. The control system 360 can transmit one or more signals instructing the catheter instrument 340 to move, which are generated by the manipulator moving the catheter instrument 340. The catheter instrument 340 can extend to the surgical position in the body via an opening of the natural cavity of the patient or a surgical incision.

[0058] Further, the control system 360 can include a mechanical control system (not shown) for controlling the movement of the catheter instrument 340, and thus can be integrated in the patient surgery platform 320, and an image processing system (not shown) for virtual navigation path planning, and thus can be integrated in the image cart 330. Of course, the various subsystems of the control system 360 are not limited to the specific cases listed above, and can be reasonably arranged according to actual conditions.

[0059] The image processing system can image the surgical site using the imaging techniques described above based on images of the surgical site recorded preoperatively or intraoperatively. Software that can also be used in conjunction with manual input converts the recorded images into two- or three-dimensional composite images of the anatomical organ or segment in question. During the virtual navigation procedure, the sensor system 350 can be used to calculate the position of the catheter instrument 340 relative to the patient's anatomy, which can be used to generate an external tracking image and an internal virtual image of the patient's anatomy, enabling registration of the actual position of the catheter instrument 340 with the preoperative image, so that the virtual image of the catheter instrument 340 within the surgical site can be presented externally to the operator.

[0060] The internal catheter instrument 341 and the external catheter instrument 342 have substantially the same structure, each having an elongated flexible internal catheter 41 and an external catheter 42, wherein the diameter of the external catheter 42 is slightly larger than that of the internal catheter 41, so that the internal catheter 41 can pass through the external catheter 42 and provide certain support for the internal catheter 41, so that the internal catheter 41 can reach the target position in the patient's body to facilitate tissue or cell sampling or other operations from the target position.

[0061] Some movements of the handle 310 can cause corresponding movement of the catheter instrument 340. For example, when the physician moves the direction dial of the handle 310 up or down, the movement of the direction dial of the handle 310 can be mapped to the corresponding pitch movement of the tip of the catheter instrument 340; when the physician moves the direction dial of the handle 310 left or right, the movement of the direction dial of the handle 310 can be mapped to the corresponding yaw movement of the tip of the catheter instrument 340. In this embodiment, the handle 310 can control the tip of the catheter instrument 340 to move within a 360° spatial range.

[0062] The remote surgery robot system of the utility model further includes a server. Figure 5 As shown in the figure, the same or different master-slave surgery robots remote physician consoles A1-A3 and patient surgery platforms B1-B3 are connected to the server S to communicate with the server S. The server S can be used to manage information sent by the remote physician consoles A1-A3 and the patient surgery platforms B1-B3, and can be used to relay the transmission of information between the remote physician consoles A1-A3 and the patient surgery platforms B1-B3. The server S can also be used to realize the interconnection of devices (i.e. pairing) between the remote physician consoles A1-A3 and the patient surgery platforms B1-B3. Pairing includes connection between two devices to enable communication.

[0063] In the super-long distance surgery, the remote doctor console and the patient surgery platform are respectively deployed at different locations. For example, they can be deployed in different countries, different provinces and cities, or different districts of the same city.

[0064] The server can also be deployed at a location different from any one of the remote doctor console and the patient surgery platform, for example, the server, the remote doctor console and the patient surgery platform can be respectively deployed in different cities. The server can also be deployed at a location same as one of the remote doctor console and the patient surgery platform, for example, the server can be deployed independently of the remote doctor console or the patient surgery platform located at the same location, and for example, the server can be integrated into the remote doctor console or the patient surgery platform located at the same location. It can be understood that the server can also be deployed in the cloud.

[0065] In the remote surgery robot system of the utility model, the remote doctor console, the patient surgery platform and the server are respectively deployed at least one.

[0066] The remote doctor console can include at least one. The patient surgery platform can also include at least one. The server can also include at least one.

[0067] When the server includes one, each remote doctor console and each patient surgery platform are connected to the one server. When the server includes multiple, the servers are connected through a network topology, and different remote doctor consoles and different patient surgery platforms can be connected to any same or different server. The network topology between the multiple servers includes at least one of star topology, ring topology, bus topology, tree topology, mesh topology, virtual local area network and wireless topology.

[0068] The remote doctor console and the server, the patient surgery platform and the server, and the servers can be connected through the same type or different type of high-speed communication network. For example, these types of high-speed communication network can include at least one of internet broadband, internet private network, 5G network and 5G private network.

[0069] For example, the remote doctor consoles A1-A3 and the patient surgery platforms B1-B3 can realize one-to-one, one-to-many and many-to-one under the action of the server S. The paired remote doctor console and the patient surgery platform can communicate end-to-end without the help of the server's relay function. For example, from the perspective of the remote doctor console, these pairings are as shown in Figure 6

[0070] ​One-to-one pairing refers to that one remote physician console is paired with one patient surgery platform. For example, for remote physician consoles A2 and A3, they are respectively paired with one patient surgery platform, such as A2 is paired with B2 or A3 is paired with B1, thus for A2 and A3, it is one-to-one pairing.

[0071] One-to-many pairing refers to that one remote physician console is paired with many patient surgery platforms. For example, for remote physician console A1, it is paired with patient surgery platforms B2 and B3, thus for A1, it is one-to-many pairing.

[0072] Many-to-one pairing refers to that many remote physician consoles are paired with one patient surgery platform. For example, for remote physician consoles A1 and A2, they are paired with one patient surgery platform B2, thus for A1 and A2, it is many-to-one pairing.

[0073] <Registration of device>

[0074] There are many types and models of remote physician consoles and patient surgery platforms, and there are also many fake and inferior ones. Inappropriate pairing of remote physician consoles and patient surgery platforms can easily lead to surgical risks. In order to reduce the surgical risks that may be caused by inappropriate pairing between remote physician consoles and patient surgery platforms as much as possible, the remote physician consoles or patient surgery platforms that are not allowed to be paired (i.e. not allowed to be used) can be filtered by adopting the method of device registration. That is, only the remote physician consoles and patient surgery platforms that have completed registration on the server are allowed to be used for pairing.

[0075] The method of registering remote physician consoles and patient surgery platforms to the server is the same. For simplicity of description, any remote physician console and any patient surgery platform can be described as a device. In some embodiments, referring to Figure 7 , the method of registering a device to a server includes:

[0076] 101. After the device establishes a connection with the server, the device sends a registration request to the server.

[0077] The process of establishing a connection between the device and the server includes: the device sends a connection establishment request to the server; the server receives the connection establishment request sent by the device, attempts to establish a connection with the device, and returns a connection state to the device. Wherein, when the returned connection state is connection success, the device and the server establish a long connection; otherwise, the device and the server do not establish a long connection. When the returned connection state is connection failure, the connection process can be re-executed.

[0078] 102. The server receives the registration request sent by the device, and authenticates the registration request based on the judgment rules agreed between the server and the device.

[0079] 103, in response to the server determining that the authentication is successful, the server accepts the registration of the device and returns a registration success status to the device.

[0080] The server can maintain the registration information of the registered devices through a registration list or the like, and the registration information includes the device information of the devices.

[0081] 104, in response to the server determining that the authentication is failed, the server rejects the registration of the device and returns a registration failure status to the device.

[0082] In some embodiments, in 102, the agreed judgment rule includes a first judgment rule, and the first judgment rule includes a self-defined protocol agreed between the device and the server. The self-defined protocol includes a protocol header and a protocol body. The protocol header is the message header, and the protocol body is the message data part. For different self-defined protocols, the format specified by the protocol header is usually the same, but the format specified by the protocol body is usually different. The difference of the protocol body can be reflected in at least one of the number of fields, the type of fields, the arrangement order of fields and the data exchange format. Figure 8 A self-defined protocol based on TCP protocol is illustrated.

[0083] The registration request is a data message encapsulated according to the format specified by the self-defined protocol. After the server receives the registration request, the registration request is decapsulated according to the format specified by the agreed self-defined protocol. If the decapsulation is successful, it can be determined that the authentication is successful; if the decapsulation fails, it can be determined that the authentication is failed.

[0084] In the initial state, for example, before the device is registered to the server, different devices only store the device information of the local end and the device information of the server. The registration request of the device includes the device information of the server and the device information of the local end. The device information of the server includes at least one of the IP address and the port number of the server. The device information of the local end includes device basic information and device network information. The device basic information includes at least one of the device unique identification (ID or UDID), the device name, the device type, the device manufacturer name, the device function, the suitable operation type and the device position. The device network information includes at least one of the IP address and the port number of the device.

[0085] The registration request of the device is encapsulated into a data packet according to the format specified by the custom protocol. Taking the custom protocol as the TCP protocol for example, the port number of the server and the port number of the device are encapsulated in the protocol header, and the other device information of the device is encapsulated in the protocol body. The protocol body includes multiple fields, and the device information of the device is stored in the corresponding field, for example, the fields include at least one of the fields of the device unique identifier (ID or UDID), the device name (name), the device type (type), the device manufacturer name (manufacture), the device function (function), and the IP address. The fields of the protocol body can also include a time stamp (time) field for storing the time when the registration request of the device is sent to the server.

[0086] In some embodiments, different devices can communicate with each other through the same or different custom protocols. For example, the remote doctor console and the server can communicate through a first custom protocol agreed upon; the patient surgery platform and the server can communicate through a second custom protocol agreed upon; and the remote doctor console and the patient surgery platform can communicate through a third custom protocol agreed upon. The three custom protocols can be the same or different. In some embodiments, one remote doctor console can be configured to support communication with multiple patient surgery platforms through the same or different custom protocols respectively, and one patient surgery platform can also be configured to support communication with multiple remote doctor consoles through the same or different custom protocols respectively.

[0087] In some embodiments, for example, when the remote doctor console or the patient surgery platform communicates with the server through the custom protocol agreed upon, the protocol body of the custom protocol can include at least one field for storing the custom protocol supported by the remote doctor console or the patient surgery platform for pairing. In 103, when the server accepts the registration of the device, the custom protocol that the corresponding device can support for pairing can also be stored in the server.

[0088] For example, the device functions of the fields in the protocol body are described. For the remote physician console, the device functions include the operable functions; for the patient surgery platform, the device functions include the operable functions. Different remote physician consoles or different patient surgery platforms can have the same or different device functions. The remote physician console and the patient surgery platform expected to be used in pairs need to match at least the device core functions in the device functions to be allowed to be used in pairs. For example, the device core functions include at least one of the motion control function and the energy control function, both of which are related to the device hardware conditions, and the corresponding device core functions are supported only when the corresponding device hardware conditions are met. For example, for a patient surgery platform including a slave robot and an energy platform, the patient surgery platform supports the motion control function based on the slave robot and supports the energy control function based on the energy platform; for a remote physician console including a hand operation part and a foot pedal part, the remote physician console supports the motion control function based on the hand operation part and supports the energy control function based on the foot pedal part; for the patient surgery platform, if the remote physician console cannot support the motion control function or the energy control function due to the limitation of its device hardware conditions, it can not be allowed to be paired with the patient surgery platform, or at least it is not allowed to be configured to have full operator control authority over the patient surgery platform, at which time it can have observer authority. In some embodiments, the motion control function or the energy control function can be further classified, and the device core functions correspond to more specific core functions. The remote physician console and the patient surgery platform expected to be used in pairs need to match at least the more specific core functions to be allowed to be used in pairs, which is also closely related to the device hardware conditions of the device, which will not be described in detail here. Among them, the device functions including the device core functions under the device function field can be recorded by associated coding, for example, recorded by numbers. Among them, the device core functions and the device non-core functions in the device functions can be added with different flag records to facilitate the matching of the device core functions.

[0089] In 103, the server accepts the registration of the device, and the server stores the registration information of the device, which includes the data stored in at least part of the fields in the custom protocol body, such as one or more of the device information, the device support pairing custom protocol, etc., for subsequent pairing. After 103, after the server accepts the registration of the device, a login password can be assigned to the device, which is used for simple authentication of the device when it logs in to the server later, so that the server does not need to authenticate the device based on the agreed judgment rules such as the custom protocol or the encryption algorithm.

[0090] In some embodiments, the agreed judgment rule can include a second judgment rule, and the second judgment rule includes an encryption algorithm agreed between the device and the server. The registration request can be a data packet encapsulated according to a format defined by the custom protocol, or a data packet encapsulated according to a format defined by a general protocol. In some embodiments, the encryption algorithm can encrypt at least part of the data in the protocol body in the data packet to generate a first check value, and the first check value is stored in an authentication field of the protocol body. The data in the protocol body can not include the data stored in the authentication field. After the server receives the registration request, the server can obtain all the data of the data packet by decapsulating the registration request according to the format defined by the custom protocol or the general protocol. The server can generate a second check value by encrypting the data in the protocol body based on the agreed encryption algorithm. The server detects whether the first check value and the second check value are the same. If they are the same, it indicates that the data is complete and legal, and the authentication is successful. If they are different, it indicates that the data is missing or illegal, and the authentication fails.

[0091] Alternatively, when the encryption algorithm can be reversely solved, the server can also not generate the second check value, but directly decrypt the first check value stored in the authentication field obtained by decapsulating the registration request based on the reverse solution of the encryption algorithm to obtain the verification data. The server compares the verification data obtained by decryption with the corresponding at least part of the data in the protocol body. If they are the same, it indicates that the data is complete and legal, and the authentication is successful. If they are different, it indicates that the data is missing or illegal, and the authentication fails.

[0092] For example, the encryption algorithm can be a base64 encryption algorithm or a hash algorithm. The data used for encryption in the protocol body can be the data of all fields in the protocol body, or the data of the fields selected according to a certain rule. For example, the rule can be the fields selected at a mutual interval, or the first few continuous fields, or the last few continuous fields, etc. In one example, one or more of the device information associated with the device in the protocol body can be selected for encryption. For example, the unique identifier, the device name and the timestamp in the protocol body can be selected as the data of the fields to be encrypted. In one example, the data of the selected fields can be concatenated into a string and then subjected to the encryption processing.

[0093] In some embodiments, the registration request can be subjected to double authentication in combination with the first judgment rule and the second judgment rule, which can prevent unauthorized devices from being allowed to access the remote surgery robot system described in the present application, thereby ensuring the safety and reliability of the remote surgery.

[0094] In the dual authentication scenario, the registration request is usually a data packet encapsulated according to the format specified in the custom protocol. The server decapsulates the registration request based on the format specified in the custom protocol, performs the first authentication, and if the decapsulation is successful, determines that the first authentication is successful and enters the second authentication; if the decapsulation fails, it is determined that the first authentication fails and the second authentication is not entered. The server will reject the registration of the device. In the second authentication, the server generates a second check value by encrypting the data in the protocol body based on the agreed encryption algorithm. The server detects whether the first check value stored in the authentication field in the data packet corresponding to the registration request and the second check value are the same. If they are the same, it is determined that the second authentication is successful, and when the two authentications are successful, the server will accept the registration of the device; if they are different, it is determined that the second authentication fails, and the server will reject the registration of the device.

[0095] Alternatively, in the second authentication, the server can also not generate the second check value as described above, but directly decrypt the first check value stored in the authentication field obtained by decapsulating the registration request based on the inverse solution of the encryption algorithm to obtain the verification data. The server compares the decrypted verification data with at least part of the corresponding data in the protocol body. If they are the same, it is determined that the authentication is successful; if they are different, it is determined that the authentication fails.

[0096] In some embodiments, patient surgery platforms can include multiple types, and the types of data exchanged between the remote physician console and the patient surgery platform can be different when the same remote physician console manipulates different patient surgery platforms to perform the same or different types of surgery. The remote physician console can also include multiple types, and the types of data exchanged between the remote physician console and the patient surgery platform can also be different when different remote physician consoles manipulate the same patient surgery platform to perform the same or different types of surgery. Among them, the types of data exchanged between the remote physician console and the patient surgery platform can be different in the field types in the protocol body of the custom protocol. It can be understood that due to the different field types in the protocol body, the custom protocol including the protocol body is also different.

[0097] In some embodiments, the server can serve as an open platform and be provided for use by multiple remote physician consoles and multiple patient surgery platforms. Different judgment rules can be agreed between the server and different devices for authentication during registration, and devices that can be successfully authenticated can be allowed to register and subsequently be paired for use in super-remote surgery. For example, when the judgment rule includes a custom protocol, different judgment rules correspond to different custom protocols. For another example, when the judgment rule includes an encryption algorithm, different judgment rules correspond to different encryption algorithms, wherein encrypting data corresponding to different field types using the same named encryption algorithm can be understood as different encryption algorithms.

[0098] For example, the server authenticates the registration request sent by the device using the custom protocol. Since the server does not know whether the server and the device have agreed on a custom protocol when the server receives the registration request sent by any device, and does not know which custom protocol has been agreed on, the server usually needs to call all known custom protocols respectively to try to unpack the registration request sent by the current device. If the registration request can be unpacked based on a custom protocol, it is determined that the authentication is successful; if the registration request cannot be unpacked based on any custom protocol, it is determined that the authentication fails.

[0099] For example, the server receives the registration request and tries to unpack the registration request using all known custom protocols in sequence. In some embodiments, it is assumed that the device includes a first device and a second device, the first device and the server agree on a first custom protocol as a judgment rule, the second device and the server agree on a second custom protocol as a judgment rule, and all known custom protocols include the first custom protocol and the second custom protocol. When the server receives the registration request, the server first tries to unpack the registration request received currently using the first custom protocol, and then tries to unpack the registration request received currently using the second custom protocol if the unpacking fails.

[0100] For example, the server receives the first registration request sent by the first device, and first unpacks the first registration request based on the first custom protocol. At this time, the unpacking is successful, and it is determined that the authentication is successful. For another example, the server receives the second registration request sent by the second device, and first unpacks the second registration request based on the first custom protocol. At this time, the unpacking fails, and the server then unpacks the second registration request based on the second custom protocol. At this time, the unpacking is successful, and it is determined that the authentication is successful. For another example, the server can also receive the third registration request sent by the third device. However, the third device can not have agreed on a custom protocol with the server. In this case, the server receives the third registration request sent by the third device, first unpacks the third registration request based on the first custom protocol, and then unpacks the third registration request based on the second custom protocol. At this time, the unpacking still fails, and it is determined that the authentication fails.

[0101] In some embodiments, the core devices of the patient surgery platform include slave robots, an image platform, and an energy platform. The slave robots include manipulators and medical instruments mounted on the manipulators, and a remote physician console controls the slave robots to control the medical instruments by controlling the motion of the manipulators. The image platform is used to process multimedia data, such as processing video or audio, where the video can be from an imaging device in the medical instruments, such as an endoscope, and the audio can be from the remote physician console or sound input from the slave robots, including voice intercom. The energy platform is used to control energy instruments in the medical instruments, such as electrotome, ultrasonic knife, microwave ablation knife, radiofrequency ablation knife, anastomat, plasma knife, etc. The image platform and the energy platform can be separately arranged or integrated on a movable trolley. For the convenience of description, the image platform and the energy platform can be collectively referred to as an image energy processing platform. In some embodiments, the patient surgery platform can further include auxiliary devices, such as a monitor, a breathing machine, a CT machine, an EM electromagnetic positioning system, a pneumoperitoneum machine, a surgery bed, or a shadowless lamp, etc.

[0102] Generally, the type of the tele-surgery robotic system can depend on the type of the slave robot in the master-slave surgery robot, and different slave robots usually have different structural characteristics. Various types of slave robots include, but are not limited to, single-hole laparoscopic slave robots, multi-hole laparoscopic slave robots, bronchial interventional slave robots, vascular interventional slave robots, and orthopedic slave robots.

[0103] In some embodiments, the type of the remote physician console can depend on the type of the operation part, which is used to control the action of the medical instruments in the slave robot, including motion control, clamping control, cutting control, coagulation control, suturing control, etc. The operation part includes a hand operation part, and the type of the hand operation part includes, but is not limited to, a gamepad type operation part, a mechanical linkage type operation part, a magnetic navigation type operation part, a trackball type operation part, etc.

[0104] In some embodiments, the remote physician console and the patient surgery platform that are not registered to the server or have different communication protocols are generally prohibited from being paired for use. The pairing requires the same communication protocol, which generally refers to the same custom protocol, mainly including the same format specified by the custom protocol.

[0105] In some embodiments, the remote physician console and the patient surgery platform that are registered to the server and have the same communication protocol are generally allowed to be paired for use. Without being limited to the similarity or difference of the type of the remote physician console or the patient surgery platform, when the basic conditions that the remote physician console and the patient surgery platform are registered to the server and the remote physician console and the patient surgery platform have the same communication protocol are met, the remote physician console and the patient surgery platform can be paired.

[0106] In some embodiments, a custom protocol can be agreed between a type of remote physician console and a type of patient surgery platform, such that the type of remote physician console is dedicated for pairing use with the type of patient surgery platform, i.e., the remote physician console is dedicated for manipulating a specific type of patient surgery platform.

[0107] In some embodiments, a custom protocol can be agreed between a type of remote physician console and a type of patient surgery platform, such that the type of remote physician console is dedicated for pairing use with the type of patient surgery platform, i.e., the remote physician console is dedicated for manipulating a specific type of patient surgery platform.

[0108] For example, a manufacturer A manufactures a universal remote physician console Al, and manufactures a patient surgery platform A2 with a first type of patient surgery platform, e.g., a patient surgery platform with a single-port laparoscopic teleoperational robot, and a patient surgery platform A3 with a second type of patient surgery platform, e.g., a patient surgery platform with a multi-port laparoscopic teleoperational robot, a same custom protocol can be agreed between the remote physician console Al and the patient surgery platforms A2, A3, or different custom protocols can be agreed respectively, and the remote physician console Al can be configured for manipulating one or more of the patient surgery platforms A2 and A3.

[0109] For example, a remote physician console Bl is manufactured by manufacturer B, and a patient surgery platform B2 having a first type and a patient surgery platform B3 having a second type are manufactured by manufacturer B, and a same custom protocol, such as a first custom protocol, can be agreed between the remote physician console Bl and the patient surgery platforms B2, B3. A remote physician console Cl is manufactured by manufacturer C, and a patient surgery platform C2 having a first type and a patient surgery platform C3 having a second type are manufactured by manufacturer C, and a same custom protocol, such as a second custom protocol, different from the first custom protocol, can be agreed between the remote physician console Cl and the patient surgery platforms C2, C3. The remote physician consoles Bl and Cl are, for example, both remote physician consoles having mechanical linkage type operation portions, the patient surgery platforms B2 and C2 are, for example, both single-port laparoscopic slave robots, and the patient surgery platforms B3 and C3 are, for example, both multi-port laparoscopic slave robots. The remote physician console Bl manufactured by manufacturer B can be configured to be exclusively used for manipulating one or more of the patient surgery platforms B2 and B3 manufactured by manufacturer B, and cannot be configured to be used for manipulating the patient surgery platforms C2 and C3 manufactured by manufacturer C. Meanwhile, the remote physician console Cl manufactured by manufacturer C can be configured to be exclusively used for manipulating one or more of the patient surgery platforms C2 and C3 manufactured by manufacturer C, and cannot be configured to be used for manipulating the patient surgery platforms B2 and B3 manufactured by manufacturer B.

[0110] For example, a remote physician console B1 manufactured by manufacturer B and a remote physician console C1 manufactured by manufacturer C can be configured to be used to manipulate one or more types of patient surgical platforms manufactured by manufacturer B and manufacturer C, respectively. For example, the remote physician console B1 and the remote physician console C1 can be configured to be used to manipulate one or more types of patient surgical platforms manufactured by manufacturer B and manufacturer C, respectively, when the remote physician console B1 and the remote physician console C1 are configured to be used to manipulate the patient surgical platforms in accordance with a first custom protocol and a second custom protocol, respectively. The first custom protocol and the second custom protocol can be the same or different. For example, the remote physician console B1 and the remote physician console C1 can be configured to be used to manipulate one or more types of patient surgical platforms manufactured by manufacturer B and manufacturer C, respectively, when the remote physician console B1 and the remote physician console C1 are configured to be used to manipulate the patient surgical platforms in accordance with a first custom protocol and a second custom protocol, respectively, and the first custom protocol and the second custom protocol are the same. For example, the remote physician console B1 and the remote physician console C1 can be configured to be used to manipulate one or more types of patient surgical platforms manufactured by manufacturer B and manufacturer C, respectively, when the remote physician console B1 and the remote physician console C1 are configured to be used to manipulate the patient surgical platforms in accordance with a first custom protocol and a second custom protocol, respectively, and the first custom protocol and the second custom protocol are different.

[0111] In some embodiments, a custom protocol can be established between a type of patient surgical platform and a type of remote physician console, or a plurality of custom protocols can be established between a type of patient surgical platform and a plurality of types of remote physician console, respectively, such that the type of patient surgical platform can be configured to be manipulated by one or more types of remote physician console.

[0112] For example, a manufacturer A manufactures a first type of patient surgery platform Al, and the manufacturer A also manufactures a first type of remote physician console A2 and a second type of remote physician console A3. The first type of remote physician console A2 can be, for example, a remote physician console having an operation part in the form of electromagnetic navigation, and the second type of remote physician console A3 can be, for example, a remote physician console having an operation part in the form of mechanical linkage. A same custom protocol can be agreed between the patient surgery platform Al and the remote physician consoles A2 and A3, or different custom protocols can be agreed respectively. The patient surgery platform Al can be configured to be manipulatable by one or more of the remote physician consoles A2 and A3.

[0113] For example, a manufacturer B manufactures a first type of patient surgery platform Bl, and the manufacturer B also manufactures a first type of remote physician console B2 and a second type of remote physician console B3. A same custom protocol, such as a first custom protocol, can be agreed between the patient surgery platform Bl and the remote physician consoles B2 and B3. A manufacturer C manufactures a first type of patient surgery platform Cl, and the manufacturer B also manufactures a first type of remote physician console C2 and a second type of remote physician console C3. A same custom protocol, such as a second custom protocol, can be agreed between the patient surgery platform Cl and the remote physician consoles C2 and C3, which is different from the first custom protocol. The patient surgery platforms Bl and Cl can be, for example, each a single-port laparoscopic tele-surgical robot. The remote physician consoles B2 and C2 can be, for example, each a remote physician console having an operation part in the form of electromagnetic navigation. The remote physician consoles B3 and C3 can be, for example, each a remote physician console having an operation part in the form of mechanical linkage. The patient surgery platform Bl can be configured to be manipulatable by one or more of the remote physician consoles B2 and B3. Meanwhile, the patient surgery platform Cl can be configured to be manipulatable by one or more of the remote physician consoles C2 and C3.

[0114] For example, a patient surgery platform manufactured by manufacturer B can be configured to be manipulated by one or more types of remote physician consoles manufactured by different manufacturers. Still taking manufacturer B as an example, manufacturer B manufactures a first type of patient surgery platform B1, and also manufactures a first type of remote physician console B2 and a second type of remote physician console B3; manufacturer C manufactures a first type of patient surgery platform C1, and also manufactures a first type of remote physician console C2 and a second type of remote physician console C3; the patient surgery platforms B1 and C1 may, for example, each be a single-hole laparoscopic robot, the remote physician consoles B2 and C2 may, for example, each be a remote physician console having an electromagnetic navigation form of operating part, and the remote physician consoles B3 and C3 may, for example, each be a remote physician console having a mechanical linkage form of operating part. By way of example, a first custom protocol is agreed between the patient surgery platform B1 and the remote physician consoles B2, B3, C2, C3, and a second custom protocol is agreed between the patient surgery platform C1 and the remote physician consoles B2, B3, C2, C3, the first custom protocol may be the same as or different from the second custom protocol, the patient surgery platform B1 manufactured by manufacturer B can be configured to be manipulated by one or more of the remote physician consoles B2 and B3 manufactured by manufacturer B, and can also be configured to be manipulated by one or more of the remote physician consoles C2 and C3 manufactured by manufacturer C; the patient surgery platform C1 manufactured by manufacturer C can be configured to be manipulated by one or more of the remote physician consoles C2 and C3 manufactured by manufacturer C, and can also be configured to be manipulated by one or more of the remote physician consoles B2 and B3 manufactured by manufacturer B.

[0115] It can be understood that whether a remote physician console and a patient surgery platform can be paired for use mainly depends on whether the two have been registered to the server and whether the communication protocols of the two are the same. Even if a remote physician console and a patient surgery platform are manufactured by different manufacturers, as long as the two are registered to the server and the communication protocols of the two are the same, the two can be paired for use, and at least the observer permission can be allowed to be used. The present application describes more conditions for allowing pairing by way of example in the following.

[0116] <Device maintenance>

[0117] When a remote physician console and a patient surgery platform are successfully registered to the server, the server will store the device information of these devices. However, as the number of remote physician consoles and patient surgery platforms successfully registered to the server increases, if the server does not regularly maintain these remote physician consoles and patient surgery platforms, the existence of a large number of inactive remote physician consoles or patient surgery platforms will waste the storage space of the server and also reduce the efficiency of pairing between active remote physician consoles and patient surgery platforms.

[0118] In some embodiments, the server can clear the registration information of the inactive remote physician console and patient surgery platform registered in the server according to a period, for example, the registration information of the inactive devices can be removed from the device list maintained by the server. When clearing the inactive devices, it can be detected whether the device to be cleared is currently in a pairing state, and if not, the clearing program is started; if it is currently in a pairing state, the device to be cleared can be converted to an active device, or the clearing program can be started after the pairing state of the device to be cleared is released, so as to prevent the device to be cleared from being paired for use when the clearing program is started, which may adversely interrupt the super-remote surgery and affect the implementation of the surgery.

[0119] Inactivity includes login inactivity and pairing inactivity. Login inactivity includes login frequency less than a preset login frequency or login duration less than a preset login duration within a server-set inactivity period. Pairing inactivity includes successful pairing frequency less than a preset pairing frequency or use duration after successful pairing less than a preset pairing use duration within the inactivity period. The server can clear the inactive devices, for example, once a week; the inactivity period is the time from successful registration of the device to the server, for example, 1 year; the preset login frequency is the number of times the device logs in to the server recorded by the server, for example, 1 time; the preset login duration is the cumulative time of the device logging in to the server, for example, 1 hour; the preset pairing frequency is the number of times the device establishes pairing with other devices recorded by the server, for example, 1 time; and the preset pairing use duration is the cumulative time of the device establishing pairing with other devices to releasing pairing recorded by the server, for example, 1 hour.

[0120] The server clears the registration information of the inactive remote physician console and patient surgery platform, effectively saving the resources of the server, improving the efficiency of pairing between the remote physician console and the patient surgery platform, and avoiding illegal use of the inactive remote physician console and patient surgery platform, which affects the safety of super-remote surgery.

[0121] <Device interconnection>

[0122] In some embodiments, the registered remote physician console and patient surgery platform can be interconnected, that is, paired. The present application discusses three pairing modes, and the pairing initiator is different in different pairing modes. The first mode is a double-end call mode, which means that the remote physician console and the patient surgery platform both initiate pairing. The second mode is a single-end call mode, which means that one end of the remote physician console and the patient surgery platform initiates pairing. The third mode is an administrator direct connection mode, which means that the server initiates pairing.

[0123] In the first mode and the second mode, the pairing initiator needs to log in to the server to initiate pairing. Among them, the pairing initiator can quickly log in to the server by using the login password assigned by the server when registration is completed. The pairing initiator includes at least one of the remote doctor console and the patient surgery platform. In the third mode, the pairing initiator is the server, and the user needs to log in to the server with an administrator account and password to obtain an administrator permission, which can be used to configure the judgment rule agreed with each device for the server, or can be used to directly pair the devices.

[0124] For simplicity of description, one of the remote doctor console and the patient surgery platform can be referred to as a first device, and the other as a second device. The pairing methods in the three modes are introduced below.

[0125] <First mode - double-end mutual call>

[0126] In some embodiments, referring to Figure 9 The method comprises:

[0127] 201. The first device sends a first request to the server.

[0128] The first request is used to request the server to send second device information of a second device that can be paired with the first device.

[0129] In an embodiment, the communication protocol, such as a custom protocol, agreed between all first devices and all second devices registered to the server is the same. Any first device and any second device can communicate after pairing. In this case, the first request is simply used to request the server to send the second device information of the second device that can be paired.

[0130] In an embodiment, the communication protocol, such as a custom protocol, agreed between at least some first devices and second devices registered to the server is different. Between the first device and the second device with different communication protocols, they cannot communicate after pairing. In this case, in order to improve compatibility and pairing efficiency, the server can match the custom protocol supported by the first device for pairing with the custom protocol supported by the second device for pairing based on the relevant information carried by the first request, such as the first device, and filter out the second device related to the custom protocol supported by the first device for pairing from the second device as the second device that can be paired, and provide the second device information of the second device that can be paired.

[0131] Among them, the second device that can be paired can be a second device that meets several conditions. For example, the second device that can be paired includes online second devices, i.e. second devices logged in to the server. For example, the second device that can be paired includes second devices in an unpaired state.

[0132] The first request is encapsulated into a data packet in a format specified by a self-defined protocol agreed between the first device and the server and sent to the server.

[0133] 202. The server receives the first request and sends second device information of a pairable second device to the first device based on the first request.

[0134] After receiving the first request, the server decapsulates the first request based on the self-defined protocol agreed with the first device, and obtains a self-defined protocol supported by the first device for pairing. The self-defined protocol supported by the first device for pairing is used for communication with the second device and is different from the self-defined protocol agreed with the server.

[0135] The server matches a second device having the same self-defined protocol supported for pairing from the registered second devices as the pairable second device based on the obtained self-defined protocol supported by the first device for pairing, and obtains second device information associated with the pairable second device.

[0136] In an embodiment, the second device information includes at least one of a plurality of device information of the corresponding second device, and the at least one device information can uniquely represent the corresponding second device. Considering security and simplicity, the second device information can only include a unique identifier of the corresponding second device, without providing device network information of the device. The unique identifier can be named in combination with device location information, for example, "XX hospital XX patient operation platform", which can provide more useful information without affecting communication security.

[0137] 203. The first device receives the second device information of the pairable second device sent by the server.

[0138] After receiving the second device information, the first device can display the second device information in at least one of a plurality of forms such as a list on a display screen of the first device for the doctor to select for pairing.

[0139] 204. The first device obtains target second device information of a target second device determined based on the second device information, and sends a first pairing request to the server.

[0140] The target second device is a second device in the pairable second devices that the doctor expects to pair with the first device. The first pairing request is used to request to establish pairing with the target second device, and the first pairing request includes device information of the first device and the target second device information of the target second device. In an embodiment, the device information of the first device and the target second device information can only include a unique identifier of the corresponding device.

[0141] The first pairing request is encapsulated into a data packet according to a format defined by a self-defined protocol agreed between the first device and the server, and is sent to the server.

[0142] 205. The server receives the first pairing request sent by the first device.

[0143] After receiving the first pairing request, the server can return a signaling to the first device indicating that the first pairing request is received.

[0144] 206. The target second device sends a second request to the server.

[0145] The second request is used to request the server to send first device information of the first device that can be paired with the target second device.

[0146] The first device that can be paired can include at least part of the first devices registered to the server. Also, in order to improve the pairing efficiency, the first device that can be paired can include the first devices registered to the server and having the same communication protocol as the second device. To achieve this purpose, the second request can include the self-defined protocol supported by the second device for pairing. The second request is encapsulated into a data packet according to a format defined by a self-defined protocol agreed between the target second device and the server, and is sent to the server.

[0147] 207. The server receives the second request and sends the first device information of the first device that can be paired to the target second device based on the second request.

[0148] After receiving the second request, the server decapsulates the second request based on the self-defined protocol agreed with the target second device for pairing, and obtains the self-defined protocol supported by the second device for pairing.

[0149] Based on the obtained self-defined protocol supported by the second device for pairing, the server matches the first devices having the same self-defined protocol supported for pairing from the registered first devices as the first device that can be paired, and obtains the first device information associated with the first device that can be paired.

[0150] The first device information includes at least one of a plurality of device information of the corresponding first device, for example, can only include the unique identifier of the corresponding first device, without providing the device network information of the device.

[0151] 208. The target second device receives the first device information of the first device that can be paired sent by the server.

[0152] The description of 208 can refer to the description of 203, which will not be repeated here.

[0153] 209, the target second device acquires the target first device information of the target first device of the expected pairing determined based on the first device information, and sends a second pairing request to the server.

[0154] The target first device is the first device in the pairable first devices that the doctor expects to pair with the target second device. The second pairing request is used to request to establish pairing with the target first device, and the second pairing request includes the target second device information of the target second device and the target first device information of the target first device. In an embodiment, the first device information includes at least one of the various device information of the corresponding first device, and the at least one device information can uniquely represent the corresponding first device. For example, the first device information can only include the unique identifier of the corresponding device.

[0155] The second pairing request is also encapsulated into a data packet in a format specified by the custom protocol agreed by the target second device and the server and sent to the server.

[0156] 210, the server receives the second pairing request sent by the target second device.

[0157] After receiving the second pairing request, the server can return a signaling to the target second device indicating that the second pairing request is received.

[0158] Since the first device and the target second device requesting pairing are registered to the server, and the target second device and the target first device requesting pairing are registered to the server, the server can return a received pairing request to the first device or the target second device, indicating that the pairing request is allowed. The server can return a paired request ok signaling to the device initiating the pairing request.

[0159] 211, the server determines whether the two peer devices expected to be paired match based on the first pairing request and the second pairing request.

[0160] The two peer devices refer to the first device and the target second device, which are peer devices to each other. The server detects the matching of the first pairing request and the second pairing request.

[0161] In the 211, the server can determine whether the two peer devices expected to be paired match based on the information included in the first pairing request and the information included in the second pairing request. For example, the server can determine whether the two peer devices expected to be paired match based on detecting whether the target first device information matches the first device information of the first device. If the target first device information matches the first device information of the first device, it is determined that the two peer devices expected to be paired match; otherwise, it is determined that the two peer devices expected to be paired do not match.

[0162] 204 can comprise determining at least one target second device information associated with a target second device, for example, comprising a plurality of times, the first device can send a plurality of first pairing requests to the server, each first pairing request is used to request to establish pairing with a target second device. 209 can comprise determining at least one target first device information associated with a target first device, for example, comprising a plurality of times, the target second device can send a plurality of second pairing requests to the server, each second pairing request is used to request to establish pairing with a target first device.

[0163] In 211, if the server determines that the two counterpart devices of the expected pairing match, go to 212; otherwise, go to 214.

[0164] 212, the server sends the device network information of the counterpart device to at least one of the first device and the target second device.

[0165] The device network information can include at least one of an IP address, a port number, and a custom protocol supported by the device for pairing.

[0166] In 212, the server can send the device network information of the target second device to the first device. Alternatively, the server can send the device network information of the first device to the target second device. Alternatively, the server can send the device network information of the target second device to the first device, and send the device network information of the first device to the target second device.

[0167] 213, at least one of the first device and the target second device receives the device network information of the counterpart device sent by the server, and establishes pairing with the counterpart device based on the device network information.

[0168] Among them, the first device receives the device network information of the target second device, and establishes pairing with the target second device based on the device network information. Alternatively, the target second device receives the device network information of the first device, and establishes pairing with the first device based on the device network information. Alternatively, both are performed simultaneously.

[0169] Among them, after the pairing is completed, the first device and the target second device can directly communicate based on any custom protocol supported by both for pairing.

[0170] 214, the server sends a pairing failure notification to the first device and the target second device.

[0171] In 204 or 209 above, the corresponding device can send the corresponding pairing request to the server in response to the triggering of an event, for example, including that the control for sending the pairing request is triggered, or the corresponding target device information is determined to reach a preset delay time, which can be 0, or other time not equal to 0.

[0172] In some embodiments, the pairing implemented by 213 is real-time pairing, after which the first device and the target second device can communicate in real time, and the communication between the two devices does not need to be relayed through the server, but is directly end-to-end communication. In some embodiments, the pairing implemented by 213 is pre-appointment pairing, after which the first device and the target second device enter and implement real-time pairing when a pre-appointment condition is met. Exemplary pre-appointment conditions include arrival of a pre-appointment pairing time, a physical state of the remote doctor console or the patient surgery platform meeting pairing requirements, and the like.

[0173] In some embodiments, after 213, the first device and the target second device establish pairing, and the server returns a pairing success state to the server, and the server receives the pairing success state and sends a pairing success prompt to the first device and the target second device. In some embodiments, after 211, if the server determines that the opposite device expected to be paired does not match, the server can send a pairing failure prompt to the first device and the target second device. The pairing success or pairing failure prompt can be displayed in the form of a UI interface or sound.

[0174] In some embodiments, two devices can send pairing requests to the server at the same time or at different times. When any device sends a pairing request to the server, the server can start a timer to time the time taken by the opposite device expected by the device to send a pairing request to the server. If the opposite device also sends a pairing request to the server within a set waiting time range, and the server determines that the pairing request sent by the device matches the pairing request sent by the opposite device, it is determined that the device and the opposite device match, and is considered as pairing success. If the pairing request does not match or times out, it is determined that the device and the opposite device do not match, and is considered as pairing failure. For example, the set time can be 60s, 120s, 180s, etc. Setting an appropriate waiting time range can improve user experience and avoid unnecessary time waste during pairing.

[0175] <Second mode - single-end call>

[0176] In some embodiments, referring to Figure 10 The method comprises:

[0177] 301. The first device sends a first request to the server.

[0178] 302. The server receives the first request and sends second device information of a pairable second device to the first device based on the first request.

[0179] 303. The first device receives the second device information of the pairable second device sent by the server.

[0180] 304, the first device acquires target second device information of the target second device determined based on the second device information, and sends a first pairing request to the server.

[0181] 305, the server receives the first pairing request sent by the first device.

[0182] The server can return signaling that the first pairing request is received to the first device after receiving the first pairing request.

[0183] 301-305 in the second mode are the same as or similar to the principles of 201-205 in the first mode, which will not be repeated here.

[0184] 306, the server sends the first pairing request to the target second device associated with the target second device information.

[0185] 307, the target second device receives the first pairing request sent by the server, and acquires a response instruction generated in response to the first pairing request.

[0186] The target second device can generate and play a notification based on the first pairing request to facilitate the doctor to respond to the pairing request, that is, to respond.

[0187] The content of the notification is whether the target second device accepts the pairing initiated by the first device. In some embodiments, the notification can be played in the form of a UI interface or in the form of a sound.

[0188] In some embodiments, the user can make a decision on the notification that the target second device accepts or rejects the pairing of the first device. For example, when played in the form of a UI interface, the user can display "accept the pairing of the first device" and "reject the pairing of the first device" in the display screen of the second target device. When the user operates the "accept the pairing of the first device" operation control, it means accepting the pairing of the first device, or when the user operates the "reject the pairing of the first device" operation control, it means rejecting the pairing of the first device. For example, when played in the form of a sound, "whether to accept the pairing of the first device" can be played in the audio device of the second target device. The user can identify the "accept the pairing of the first device" input by the user through the voice recognition device set by the second target device, and generate a response instruction to accept the pairing of the first device, or identify the "reject the pairing of the first device" input by the user, and generate a response instruction to reject the pairing of the first device.

[0189] In some embodiments, the second target device can also make the decision of accepting or rejecting the pairing with the first device, i.e. by itself. For example, when the second target device receives the first pairing request sent by the server, the second target device can start a timer to count time. When the response waiting time range is reached and the second target device does not obtain the response to the notification from the user, the second target device generates a response instruction of rejecting the pairing with the first device by default, and sends the response instruction to the server. After receiving the response instruction, the server sends a notification of pairing failure to the first device. For another example, the second target device can be configured to store a white list associated with one or more first devices, and the second target device allows the first devices associated with the white list to pair with the second target device without the need of the response to the notification from the user. For example, in the white list scenario, if the first device is in the white list of the second target device, the user can actively respond to the notification within the response waiting time range, and the response includes the acceptance or rejection of the pairing with the first device. When the preset condition is met, such as when the response waiting time range is reached and the second target device does not obtain the response to the notification from the user, the second target device automatically generates a response instruction of accepting the pairing with the first device, and sends the response instruction to the server. After receiving the response instruction, the server sends a notification of pairing success to the first device.

[0190] 308, the second target device sends the response instruction to the server.

[0191] 309, the server sends the device network information of the opposite device to at least one of the first device and the second target device in response to receiving the positive response instruction.

[0192] If the response instruction indicates that the second target device accepts the pairing with the first device, i.e. the response instruction is a positive response instruction.

[0193] 310, at least one of the first device and the second target device receives the device network information of the opposite device sent by the server, and establishes the pairing with the opposite device based on the device network information.

[0194] For example, the server sends the device network information of the second target device to the first device, the first device receives the device network information, and establishes the pairing with the second target device based on the device network information.

[0195] 311, the server sends a prompt of pairing success to the first device and the second target device.

[0196] 312, the first device receives the prompt of pairing success sent by the server, and plays the prompt of pairing success; the second target device receives the prompt of pairing success sent by the server, and plays the prompt of pairing success.

[0197] 313, the server sends a pairing failure prompt to the first device in response to receiving the negative response instruction.

[0198] If the response instruction indicates that the target second device rejects the pairing of the first device, i.e., the response instruction is a negative response instruction.

[0199] In addition, the server can also send a pairing failure prompt to the target second device at the same time.

[0200] 314, the first device receives the pairing failure prompt sent by the server and plays the pairing failure prompt.

[0201] In some embodiments, the first device is a remote doctor console, and the target second device is a patient surgery platform, which is suitable for use when the doctor initiates matching with the patient. In some embodiments, the first device is a patient surgery platform, and the target second device is a remote doctor console, which is suitable for use when the patient initiates matching with the doctor. The two application scenarios can maximize the use of high-quality medical resources.

[0202] In the above embodiments, the request sent by the local device to the server for requesting a pairable opposite device, such as the first request sent by the first device to the server, or the second request sent by the second device to the server, can include or not include a filtering condition.

[0203] When these requests do not include any filtering condition, in order to facilitate the pairing of the local device with the target opposite device among the pairable opposite devices, the server can obtain various state information of the opposite devices and send them to the local device for prompting, such as displaying. These state information can include at least one of the following: custom protocol information supported by the opposite device for pairing, online state information, device type information, device function information, applicable surgery type information, network state information, device positioning information, power-on self-test state information, pairing state information, pre-booking pairing time information, and control authority allocation state information. The doctor can select the target opposite device for desired pairing according to these state information, however, too much state information is not conducive to quickly and accurately performing the desired pairing. Therefore, in order to further improve the pairing efficiency, the request can be configured to include a filtering condition, so that the server can accurately provide the pairable opposite devices based on the filtering condition, so that the doctor can select the target opposite device for desired pairing.

[0204] In some embodiments, the filtering condition can be configured based on the above-mentioned state information of the local device or the opposite device that can be obtained. Some filtering conditions are associated with the state information of the local device and the opposite device, and some filtering conditions are only associated with the state information of the opposite device. The filtering condition can include at least one of the following.

[0205] For example, the screening condition includes a custom protocol supported by the local device for pairing. The server can obtain the screening condition from the device information of the local device, and match, based on the screening condition, a counterpart device from the counterpart devices registered to the server, which matches the custom protocol supported by the local device for pairing, as a pairable counterpart device. By setting the screening condition, a pairable counterpart device that is actually pairable and used can be obtained.

[0206] For example, the screening condition includes a device type of the local device. The registration information of the local device or the counterpart device stored by the server can include the device type of the device. For the counterpart device being a patient surgery platform, the device type can be embodied in the type of the slave robot, which includes a single-hole laparoscopic slave robot, a multi-hole laparoscopic slave robot, a bronchial intervention slave robot, a vascular intervention slave robot, or an orthopedic slave robot, etc. For the counterpart device being a remote doctor console, the device type can be embodied in the type of the operation part, which includes a gamepad-form operation part, a mechanical linkage-form operation part, a magnetic navigation-form operation part, or a trackball-form operation part, etc. Different types of remote doctor consoles can be suitable for different types of patient surgery platforms. The server can obtain the screening condition from the device information of the local device, and match, based on the screening condition, a counterpart device from the counterpart devices registered to the server, which matches the device type of the local device, as a pairable counterpart device.

[0207] For example, the screening condition includes a device function supported by the local device. The registration information of the local device or the counterpart device stored by the server can include a device function supported by the device. The device function includes a device core function. Only when the remote doctor console and the patient surgery platform match in the device core function, pairing is allowed. Exemplarily, the device core function includes having a motion control function or having an energy control function. This is especially often considered when the control authority to be configured is an (full) operator authority. If the control authority to be configured is only an observer authority, as long as the two counterpart devices expected to be paired both include an observation function, they can be matched and allowed to be paired, regardless of whether the device core function is matched. The server can obtain the screening condition from the device information of the local device, and match, based on the screening condition, a counterpart device from the counterpart devices registered to the server, which matches the device function supported by the local device, as a pairable counterpart device. By setting the screening condition, a pairable counterpart device that is actually pairable and used can be obtained.

[0208] For example, the screening condition includes a surgery type supported by the local device, and the registration information of the local device or the remote device stored by the server can include the suitable surgery type of the device, different slave robots can be suitable for the same or different surgery types, for example, for thoracic or abdominal surgery, a (single-hole or multi-hole) endoscopic slave robot can be suitable; for bronchial or vascular or urinary tube surgery, a (bronchial or vascular) interventional slave robot can be suitable; for orthopedic surgery, an orthopedic slave robot can be suitable. Different remote physician consoles can be suitable for the same or different surgery types. The surgery type can also include more specific surgical procedures, such as urinary surgery, lung surgery, liver surgery, kidney surgery, knee replacement surgery, etc. For urinary surgery, a slave robot can be matched to a (vascular) interventional slave robot; for lung surgery, a slave robot can be matched to a (bronchial) interventional slave robot; for liver surgery or kidney surgery, a slave robot can be matched to a (single-hole or multi-hole) endoscopic slave robot; for knee replacement surgery, a slave robot can be matched to an orthopedic slave robot. Among them, the server can obtain the screening condition from the device information of the local device, and match the remote devices that match the surgery type supported by the local device from the remote devices registered to the server as the pairable remote devices based on the screening condition. By setting the screening condition, the pairable remote devices that can be truly paired and used can be obtained.

[0209] For example, the screening condition includes a pre-pairing time of the local device. The pre-pairing time of the local device can be input by the physician and obtained by the server. The remote device can be input by the physician with a surgery planning time, and the surgery planning time is obtained by the server, including an occupied time period and an idle time period. The server can match the pairable remote devices whose pre-pairing time does not conflict with the idle time period from the surgery planning time associated with the remote devices obtained by the server. Among them, the remote device in the unpaired state and without configuring the surgery planning time can be considered as the idle time period of the surgery planning time.

[0210] For example, the screening condition includes an expected online state of the remote device. When the server obtains that the remote device is currently logged in the server, the server can mark the remote device as online; when the server obtains that the remote device is currently not logged in the server, the server can mark the remote device as offline. The physician can configure the expected online state as the screening condition, so that the server matches the pairable remote devices that meet the online state, which is beneficial to quickly obtain the remote devices that can be paired in real time.

[0211] For example, the screening condition includes a desired network state of the peer device. The server can obtain the network state between the peer device and the server, and the network state can include a network type and a network communication state, and the network communication state includes normal and abnormal. The doctor can configure the desired network state as the screening condition, so that the server matches the peer device that meets the network state.

[0212] For example, the screening condition includes a desired pairing state of the peer device. The server can obtain and maintain the pairing state of the local device or the peer device, and the pairing state includes paired and unpaired. To ensure privacy and security, the peer device that can be paired is usually from the peer device in the unpaired state. In the remote teaching scenario, multiple local devices can be paired with one peer device, and the peer device in the paired state may appear. In this scenario, the peer device that can be paired is also allowed to come from the peer device in the paired state. The doctor can configure the desired pairing state as the screening condition, so that the server matches the appropriate peer device that can be paired.

[0213] For example, the screening condition includes a desired device location of the peer device. The registration information of the local device or the peer device stored by the server can include the device location of the device, which can be obtained by the positioning module configured for the device, such as a GPS positioning module, a Beidou positioning module, a Starlink positioning module, etc., or can be obtained by IP address resolution, or can be manually written. Various positioning methods can be used for mutual verification, and the verification is successful. The device location is used for screening, and the verification fails. The user can be prompted to manually modify the device location for screening. The device location can include multiple levels, such as continent level, country level, province level, city level, district level, street level, or building level. The doctor can configure the desired device location as the screening condition, so that the server matches the peer device that matches the device location.

[0214] For example, the screening condition includes a desired control authority allocation state of the peer device. The server can obtain the control authority allocation state configured by the peer device, and the control authority allocation state includes all control authorities allocated state, part of the control authority allocated state, and control authority unallocated state. For the all control authority allocated state, more suitable for pairing in the teaching scenario. For the part of the control authority allocated state, more suitable for pairing in the collaborative operation scenario. For the control authority unallocated state, suitable for pairing in any scenario. The doctor can configure the desired control authority as the screening condition, so that the server matches the appropriate peer device that can be paired.

[0215] For example, the screening condition includes the power-on self-test state expected by the terminal device. When the server communicates with the local terminal device or the opposite terminal device, the power-on self-test state of the device can be obtained, which can include various levels, for example, it can include that the software and hardware of the device are all normal or the core functions of the software and hardware of the device are all normal. Generally, the matched opposite terminal device is expected to be normal in software and hardware. However, when the opposite terminal device expected to be paired does not meet the requirement of being normal in software and hardware, in order to ensure that the ultra-long distance surgery can be implemented, as long as the core functions of the software and hardware are normal, the opposite terminal device can also be paired for use. The power-on self-test state can be associated with the patient condition information. For example, when the opposite terminal device is a patient surgery platform, the device software and hardware functions include core devices such as mobile robots and energy platforms, and auxiliary devices such as monitors or breathing machines. When the patient condition is not serious and the auxiliary device does not need to be applied, even if the auxiliary device software and hardware functions are abnormal, as long as the core device software and hardware functions are normal, it can also be paired for use, which can meet the minimum pairing requirement and improve the pairing success rate. Wherein, whether the auxiliary device needs to be applied can be obtained from the patient condition information input by the local terminal device or the opposite terminal device, and the patient condition information can include the definition of the core device and the auxiliary device to be used. The doctor can configure the expected power-on self-test state for the server to match the appropriate pairable opposite terminal device.

[0216] Wherein, each of the above screening conditions can be used alone or in combination, and the opposite terminal device that meets one or more of the above screening conditions can be used as a pairable opposite terminal device.

[0217] <Third mode - administrator direct connection>

[0218] In some embodiments, the server can obtain all first devices and second devices that complete registration, and perform expected pairing based on the obtained first devices and second devices, which can realize batch configuration of devices in multiple places and improve pairing efficiency. However, in actual situations, different communication protocols can be agreed between these first devices and second devices, and since the communication protocol is invisible, the pairing led by the doctor is often trial and error, resulting in low pairing efficiency.

[0219] In some embodiments, the server can configure different tags for different custom protocols supported by the corresponding device for pairing, and when the device information of the corresponding device is displayed on the display screen of the device or the server, the tag can be displayed together to facilitate the doctor to identify the devices that can be paired with each other. Wherein, the custom protocol supported by each device for pairing can include that different tags can be configured for the custom protocol of the device respectively. For example, the first device and the second device with the same tag can be paired with each other, and the first device and the second device with different tags cannot be paired with each other. Wherein, different tags can be represented by different numerical codes, characters, words, colors or symbols, etc.

[0220] In some embodiments, the physician can utilize the server itself for administrator direct connection; or, the physician can utilize any device that can log into the server with administrator permission, including a remote physician console, a patient surgery platform, a desktop computer, or a portable mobile terminal, including a cell phone, a tablet computer, or a laptop computer, for administrator direct connection. In some embodiments, any of these devices that can be used for administrator direct connection can be registered to the server, and the registration of these devices can be authenticated by a custom protocol or an encryption algorithm as described above to improve security. In some embodiments, the mobile terminal device can be bound to the remote physician console or the patient surgery platform to participate in the pairing configuration of the above-mentioned multiple modes as the remote physician console or the patient surgery platform.

[0221] In some embodiments, to facilitate administrator direct connection using the server, the utility model provides a method, referring to Figure 11 The method comprises:

[0222] 401, obtaining first device information of a first device registered to the server.

[0223] 402, obtaining target first device information associated with a target first device determined based on the first device information.

[0224] The target first device is used for pairing with a second device.

[0225] 403, based on the target first device information, matching second device information of a pairable second device that can be paired with the target first device from a second device registered to the server.

[0226] As described above, the pairable second device that can be paired with the target first device is usually compatible with the custom protocol supported by the target first device for pairing.

[0227] 404, obtaining target second device information associated with a target second device determined based on the second device information.

[0228] 405, obtaining device network information of at least one of the target first device and the target second device.

[0229] 406, sending the device network information to a peer device associated with the network information.

[0230] 407, the peer device receives the network information, and establishes pairing with a local device associated with the network information based on the network information, to realize direct communication.

[0231] For example, in 406, the device network information includes the device network information of the target first device, the device network information is sent to the target second device. Further in 407, the target second device establishes the pairing with the target first device based on the device network information of the target first device.

[0232] For example, in 406, the device network information includes the device network information of the target second device, the device network information is sent to the target first device. Further in 407, the target first device establishes the pairing with the target second device based on the device network information of the target second device.

[0233] For example, in 406, the device network information includes the device network information of the target first device and the device network information of the target second device, the device network information of the target first device is sent to the target second device, and the device network information of the target second device is sent to the target first device. Further in 407, the target second device establishes the pairing with the target first device based on the device network information of the target first device, and the target first device establishes the pairing with the target second device based on the device network information of the target second device.

[0234] Through the above 401-406, the pairing can be quickly performed, and the pairing efficiency can be improved.

[0235] After 406, the target first device and the target second device establish the pairing, at least one of the target first device and the target second device returns a pairing success status to the server, and the server receives the pairing success status to update the pairing status maintained by the server. The server can send a pairing success prompt to the target first device and the target second device.

[0236] In some embodiments, the target first device can include one, the target second device can include at least one, and the pairable second device can include a second device supporting a custom protocol for pairing compatible with the custom protocol for pairing supported by the target first device.

[0237] For example, assuming that the custom protocol for pairing supported by the target first device is A, and the custom protocol for pairing supported by the second device includes A, the second device can be regarded as a pairable second device.

[0238] In some embodiments, the target first device can include at least one, the target second device can include one, and the pairable second device can include a second device supporting a custom protocol for pairing compatible with the custom protocol for pairing supported by each target first device.

[0239] For example, assuming that the custom protocol for pairing supported by one target first device is A, and the custom protocol for pairing supported by another target first device is B, and the custom protocol for pairing supported by the second device is compatible with A and B, the second device can be regarded as a pairable second device.

[0240] For example, the second device can be a pairable second device when the custom protocol for pairing supported in the second device is at least compatible with the custom protocol for pairing supported in one of the target first devices.

[0241] For example, assuming that a custom protocol for pairing supported in a target first device is A and B, and a custom protocol for pairing supported in another target first device is C and D, the second device can be a pairable second device when the custom protocol for pairing supported in the second device is at least compatible with at least one of A and B and at least compatible with at least one of C and D. For example, the second device can be a pairable second device when the custom protocol for pairing supported in the second device includes A and C, or A and D, or B and C, or B and D. Of course, the second device can also be a pairable second device when the custom protocol for pairing supported in the second device includes A, B and C, or A, C and D, or B, C and D, or A, B, C and D.

[0242] For example, assuming that a custom protocol for pairing supported in a target first device is A and B, and a custom protocol for pairing supported in another target first device is B and C, the second device can be a pairable second device when the custom protocol for pairing supported in the second device is at least compatible with at least one of A and B and at least compatible with at least one of B and C. For example, the second device can be a pairable second device when the custom protocol for pairing supported in the second device includes B, or A and B, or B and C, or A and C. Of course, the second device can also be a pairable second device when the custom protocol for pairing supported in the second device includes A, B and C.

[0243] In some embodiments, the screening conditions can be configured to obtain suitable devices when determining the first device for pairing and the second device for pairing with the first device. The screening conditions are described above and will not be repeated here.

[0244] Based on the pairing methods in the first mode to the third mode, one-to-one, one-to-many, or many-to-one pairing between any remote doctor console and patient surgery platform can be quickly achieved. The user operation can be simplified, the labor cost and time cost can be saved, and the complexity of remote surgery can be reduced, the universality of remote surgery can be improved, and the establishment of a super-remote surgery medical network can be facilitated.

[0245] In some embodiments, in the above pairing modes, the server can save pairing information of the first device and the second device for establishing pairing, and assign a unique pairing identifier to the first device and the second device. The first device and the second device can attempt to establish pairing and communicate according to the pairing identifier, and mutual communication is prohibited before pairing is successful. The pairing identifier can be carried in the signaling (i.e., data) of each subsequent communication for communication authentication, and a receiving device has the right to refuse communication services for devices with mismatched pairing identifiers.

[0246] In some embodiments, the server can acquire the pairing status of all devices in real time, and can maintain a pairing table according to the pairing status, which records the paired, pairing or unpaired status of the devices. The server can set the status of the device that has completed real-time pairing as paired. The server can set the status of the device that is scheduled to be paired or is establishing real-time pairing as pairing. The server can also delete the pairing information of the device that fails to pair or is unpaired, and set the status of the device that fails to pair or is unpaired as unpaired. Wherein, with the update of the pairing table, the second device information of the second device that can be paired, or the first device information of the first device and the second device information of the second device that can be paired with each other can be updated and sent to the related devices, for example, the former is updated and sent to the first device, and the latter is updated and sent to the server, to facilitate pairing.

[0247] In some embodiments, the first device, the second device or the server can include a display screen and an input device, and the visual pairing of the first device and the second device can be quickly realized by using the display screen and the input device. The display screen and the input device can be independent components, and the input device can include a mouse, a keyboard, etc., wherein, on the side of the remote doctor console, the operating part thereof can also be used as a mouse in the pairing stage. The display screen and the input device can also be integrated components, for example, a touch screen.

[0248] <UI interaction of double-end mutual call>

[0249] In some embodiments, the utility model provides a method for pairing, referring to Figure 12 , the method comprises:

[0250] 501, the first device displays a first configuration interface on the first display screen thereof, and in response to obtaining the triggering of a first control in the first configuration interface, sends a first request to the server.

[0251] 502, the server receives the first request sent by the first device, and sends the second device information of the second device that can be paired to the first device based on the first request.

[0252] 503, the first device receives the second device information of the second device that can be paired sent by the server, and displays the second device information in the first configuration interface.

[0253] 504, the first device acquires the target second device information of the target second device that is expected to be paired based on the second device information, and sends a first pairing request to the server.

[0254] The first device can automatically trigger the first pairing request to the server immediately after determining the target second device information. The first device can trigger the first pairing request to the server in response to an event after determining the target second device information. The event includes that a timer of the first device reaches a delay time since determining the target second device information, or a control in the first configuration interface is triggered, the control being a control for controlling sending of the pairing request. The first device triggers the first pairing request to the server after obtaining that the timer reaches the delay time or obtaining that the control is triggered.

[0255] 505. The server receives the first pairing request sent by the first device.

[0256] The server can return a signaling that the first pairing request is received to the first device after receiving the first pairing request.

[0257] 506. The target second device displays a second configuration interface on a second display screen thereof, and sends a second request to the server in response to obtaining a trigger of a first control in the second configuration interface.

[0258] 507. The server receives the second request sent by the target second device, and sends first device information of a first device that can be paired to the target second device based on the second request.

[0259] 508. The target second device receives the first device information of the first device that can be paired sent by the server, and displays the first device information in the second configuration interface.

[0260] 509. The target second device obtains target first device information determined based on the first device information, and sends a second pairing request to the server.

[0261] The target second device can automatically trigger the second pairing request to the server immediately after determining the target first device information. The target second device can trigger the second pairing request to the server in response to an event after determining the target first device information. The event includes that a timer of the target second device reaches a delay time since determining the target first device information, or a control in the second configuration interface is triggered, the control being a control for controlling sending of the pairing request. The target second device triggers the second pairing request to the server after obtaining that the timer reaches the delay time or obtaining that the control is triggered.

[0262] 510. The server receives the second pairing request sent by the target second device.

[0263] The server can return a signaling that the second pairing request is received to the target second device after receiving the second pairing request.

[0264] 511, the server determines whether the two peer devices expected to be paired match based on the first pairing request and the second pairing request.

[0265] If the server determines that the two peer devices expected to be paired match, go to 512; otherwise, go to 515.

[0266] 512, the server sends the device network information of the peer device to at least one of the first device and the target second device.

[0267] 513, at least one of the first device and the target second device receives the device network information of the peer device sent by the server, and establishes pairing with the peer device based on the device network information.

[0268] 514, after the first device and the target second device establish pairing, a message indicating that the pairing is successfully established is displayed on the first configuration interface and the second configuration interface, respectively.

[0269] 515, the server sends a notification indicating that the pairing is failed to the first device and the target second device.

[0270] 516, the first device and the target second device receive the notification indicating that the pairing is failed sent by the server, and display a message indicating that the pairing is failed on the first configuration interface and the second configuration interface, respectively.

[0271] For example, the message indicating that the pairing is successfully established or the message indicating that the pairing is failed can be displayed by popping up a dialog box on the configuration interface, or can be displayed by a fixed or floating status bar in the configuration interface.

[0272] In some embodiments, in combination with Figures 13 to 15 Referring to 501 or 506, the first control 1 can be a button, when the button 1 is clicked, it indicates that the first control 1 is triggered, and the corresponding device sends a corresponding request to the server to request the device information of the pairable peer device. In 503 or 508, the corresponding device information sent by the server can be displayed on the corresponding configuration interface through the second control 2, which can be a list control. The list control 2 includes at least one list option, and one list option is associated with the device information of one device. In 504 or 509, the corresponding device can determine the device information associated with the one or more list options selected as target device information when detecting that the one or more list options are selected, that is, determine the target peer device expected to be paired by the corresponding device. For example, taking the configuration interface of a certain remote doctor console such as the remote doctor console of hospital A as an example, as shown in Figure 13 , the first control 1 is triggered, and the remote doctor console of hospital A sends a request to request the pairable peer device; and as shown in Figure 14At this time, the server matches the A-E hospital patient surgery platforms and displays them in the list control 2 in the configuration interface; as shown in Figure 15 When the list option associated with the B hospital patient surgery platform is selected and confirmed, the B hospital patient surgery platform is the counterpart device that the A hospital remote doctor console expects to pair with, and sends a pairing request. Similarly, if the A hospital remote doctor console is determined to be the counterpart device that the B hospital patient surgery platform expects to pair with in the configuration interface of the B hospital patient surgery platform and sends another pairing request, the two devices match and pairing can be established; otherwise, the two devices do not match and pairing cannot be established.

[0273] In this embodiment, when the corresponding device obtains that pairing fails, it can automatically restore the list option to the initial state, such as the unselected state, to facilitate subsequent pairing of the corresponding device with other counterpart devices. The corresponding device can also release pairing with the counterpart device when it obtains that the list option is restored to the initial state, such as the unselected state.

[0274] In some embodiments, in combination with Figures 16 to 19 Referring to 501 or 506, the first control 1' can be a window control, and the corresponding device can generate the second control 2' in the respective configuration interface according to the device information of the corresponding device. The second control 2' can be a drag control, which has an initial position in the respective configuration interface. When the second control 2' is moved into the window of the window control 1', it indicates that the first control 1' is triggered, and the corresponding device sends a corresponding request to the server to request the device information of the counterpart devices that can be paired. In 503 or 508, the corresponding device information can be displayed in the respective configuration interface through the third control 3'. The third control 3' can be a drag control, which has an initial position in the respective configuration interface. The number of third controls 3' is the same as the number of counterpart devices that can be paired, and each third control 3' corresponds to a counterpart device. In 504 or 509, when the corresponding device detects that the third control 3' is moved into the window of the window control 1', it determines that the device information associated with the third control 3' is the target device information, i.e., the target counterpart device that the corresponding device expects to pair with. For example, the configuration interface of a certain remote doctor console, such as the A hospital remote doctor console, is taken as an example for illustration, as shown in Figure 16 and Figure 17 When the second control 2' representing the A hospital remote doctor console is moved into the window of the window control 1', the A hospital remote doctor console sends a request to request the counterpart devices that can be paired; as shown in Figure 18 At this time, the server matches the A-E hospital patient surgery platforms and displays them in the list control 2 in the configuration interface; as shown in Figure 19When the drag control 3' associated with the B hospital patient surgery platform is moved into the window of the window control 1' and is confirmed, the B hospital patient surgery platform is the peer device desired to be paired by the A hospital remote physician console, and a pairing request is sent. Similarly, if the A hospital remote physician console is determined to be the peer device desired to be paired in the corresponding configuration interface of the B hospital patient surgery platform, and another pairing request is sent, the two are matched, and pairing can be established; otherwise, the two are not matched, and pairing cannot be established.

[0275] In this embodiment, referring to Figures 20 to 22 , the window of the window control 1' can include a first area 11' and a second area 12', the first area 11' includes a first sub-area 11a' and a second sub-area 11b' for accommodating the second control 2' respectively, and the second area 12' is for accommodating the third control 3', wherein these areas can also be implemented by windows. In the case that one or more remote physician consoles can be paired with one patient surgery platform, the second control 2' is associated with the remote physician console, and the third control 3' is associated with the patient surgery platform. The first sub-area 11a' and the second sub-area 11b' can be associated with different control authorities, or can also be associated with different pairing sequences. For example, in response to the third control 3' being moved to the second area 12', the patient surgery platform associated with the third control 3' sends a first request to the server, and the pairable remote physician consoles that can be paired with the patient surgery platform can be matched and displayed in the configuration interface of the patient surgery platform. The pairable remote physician consoles can be displayed in the form of drag controls outside the window.

[0276] For example, the first sub-area 11a' can be associated with the first control authority of the remote physician console accommodated therein to the patient surgery platform accommodated in the second area 12', and the second sub-area 11b' can be associated with the second control authority of the remote physician console accommodated therein to the patient surgery platform accommodated in the second area 12', the first control authority being different from the second control authority. For example, the first control authority is the full operator authority, and the second control authority is the observer authority. When the second control 2' associated with the first remote physician console is moved to the first sub-area 11a', the first remote physician console is configured to have full operator authority to the patient surgery platform accommodated in the second area 12'; when the second control 2' associated with the second remote physician console is moved to the second sub-area 11b', the second remote physician console is configured to have observer authority to the patient surgery platform accommodated in the second area 12'. As Figure 22For the same patient surgery platform, such as the patient surgery platform of hospital A, the remote physician console of hospital B has different control permissions or pairing orders relative to the remote physician console of hospital A. In this example, the second sub-area 11b' can include one or more, suitable for the quick pairing of a remote physician console with one patient surgery platform, which has one full operator permission and multiple observer permissions. In this way, the quick allocation of control permissions can be realized at the time of pairing.

[0277] For example, the first sub-area can be associated with a first pairing order of the remote physician console housed therein and the patient surgery platform housed in the second area, the second sub-area can be associated with a second pairing order of the remote physician console housed therein and the patient surgery platform housed in the second area, and the first pairing order and the second pairing order are different. For example, the first pairing order is prior to the second pairing order, when the second control associated with the first remote physician console is moved to the first sub-area, the first remote physician console is configured to have the first pairing order with the patient surgery platform housed in the second area; when the second control associated with the second remote physician console is moved to the second sub-area, the second remote physician console is configured to have the second pairing order with the patient surgery platform housed in the second area. In this example, the first area can include more sub-areas, and different sub-areas have different pairing orders. In this way, the quick setting of the pairing order can be realized at the time of pairing. In this example, the first area can also be used to house different patient surgery platforms, and the second area is used to house remote physician consoles, and the quick setting of the pairing order between different patient surgery platforms and the same remote physician console can also be realized. For example, in response to the drag control associated with the patient surgery platform being moved to the second area, the patient surgery platform sends a first request to the server, and the server can match the pairable remote physician console that can be paired with the patient surgery platform and display it in the configuration interface of the patient surgery platform. The pairable remote physician console can be displayed in the form of a drag control outside the window.

[0278] Wherein, whether the second control or the third control is moved into the first control can be determined by detecting whether the pixel coordinates of the boundary of the second control or the third control fall within the pixel coordinates of the boundary of the first control. Whether the second control or the third control is moved into the corresponding area can also be determined by detecting whether the pixel coordinates of the boundary of the second control or the third control fall within the pixel coordinates of the boundary of the corresponding area.

[0279] In this embodiment, the drag control can have the same or different size before and after being moved. For example, the drag control can have a larger or smaller size relative to before being moved when it is moved into the first control.

[0280] In this embodiment, when the corresponding device obtains the pairing failure, the corresponding device can automatically remove at least the third control from the first control and reset the third control to the position before being removed, so as to facilitate the subsequent pairing of the corresponding device and other peer devices. The corresponding device can also cancel the pairing with the peer device when the corresponding device obtains that the second control and the third control are removed from the first control. When the second control or the third control is manually removed, the second control or the third control can be automatically restored to the position before being removed once the second control or the third control is removed from the first control.

[0281] In some embodiments, in 501 or 506, the first control can be a button, and when the button is clicked, it indicates that the first control is triggered, and the corresponding device sends a corresponding request to the server to request the device information of the pairable peer device. The corresponding device can generate the second control according to the device information of the corresponding device. In 503 or 508, the corresponding device information sent by the server can be displayed on the corresponding configuration interface through the third control, and the number of the third controls is the same as the number of the pairable peer devices, each third control corresponds to a peer device, and the positions of the second control and the third control in the configuration interface are fixed. In 504 or 509, when the display screen of the corresponding device is a touch screen, the corresponding device determines that the device information associated with the third control corresponding to the start or end of the contact and continuous movement of the finger on the touch screen as the target device information, i.e., determines the target peer device that the corresponding device expects to pair, when the corresponding device detects that the finger on the touch screen moves from one of the second control and the third control and continuously moves to the other of the second control and the third control. When the display screen of the corresponding device is a touch screen or a non-touch screen, the corresponding device determines that the device information associated with the third control corresponding to the start or end of the continuous movement of the cursor on the display screen as the target device information, i.e., determines the target peer device that the corresponding device expects to pair, when the corresponding device detects that the cursor on the display screen moves from one of the second control and the third control and continuously moves to the other of the second control and the third control. In 504 or 509, when the display screen of the corresponding device is a touch screen, the corresponding device can also determine that the device information associated with the third control included in the closed track formed by the continuous movement of the finger around the third control on the touch screen as the target device information, i.e., determines the target peer device that the corresponding device expects to pair, when the corresponding device detects that the finger on the touch screen continuously moves around the third control to form a closed track such as a circular, square, triangular, or other closed track. When the display screen of the corresponding device is a touch screen or a non-touch screen, the corresponding device can also determine that the device information associated with the third control included in the closed track formed by the continuous movement of the cursor around the third control on the display screen as the target device information, i.e., determines the target peer device that the corresponding device expects to pair, when the corresponding device detects that the cursor on the display screen continuously moves around the third control to form a closed track. In 504 or 509, the corresponding device can also determine that the peer device as the target peer device when the corresponding device detects that the number of clicks or the time of staying of the finger or the cursor on the space corresponding to the peer device reaches a threshold.

[0282] In this embodiment, the movement track of the finger or cursor on the corresponding display screen can be displayed on the corresponding configuration interface to identify the pairing relationship.

[0283] In some embodiments, each configuration interface can include a control for configuring the aforementioned filtering condition to accurately match the pairable opposite device. The control can include a text box, a list, or a check box, etc. For example, the text box can input the filtering condition. The list can select the filtering condition. The check box can check the filtering condition. Before performing 501 or 506, the doctor can configure the filtering condition. In 501 or 506, the corresponding device can encapsulate the configured filtering condition into the corresponding request and send it to the server, so that the server can filter out the pairable opposite device that meets the condition based on the corresponding request.

[0284] <UI interaction of single-end call>

[0285] In some embodiments, the utility model provides a method for pairing, referring to Figure 23 , the method comprises:

[0286] 531, the first device displays a first configuration interface on its first display screen, and in response to obtaining the triggering of a first control in the first configuration interface, sends a first request to the server.

[0287] 532, the server receives the first request sent by the first device, and based on the first request, sends the second device information of the pairable second device to the first device.

[0288] 533, the first device receives the second device information of the pairable second device sent by the server, and displays the second device information in the first configuration interface.

[0289] 534, the first device obtains the target second device information of the expected pairing second target device determined based on the second device information, and sends a first pairing request to the server.

[0290] 535, the server receives the first pairing request sent by the first device.

[0291] After receiving the first pairing request, the server can return the signaling of receiving the first pairing request to the first device.

[0292] 536, the server sends the first pairing request to the target second device associated with the target second device information.

[0293] 537, the target second device receives the first pairing request sent by the server, and displays a second configuration interface in the second display screen of the target second device, and generates a first control and a second control in the second configuration interface based on the first pairing request.

[0294] The first control may be, for example, a pop-up dialog box for notifying the doctor whether to accept the pairing of the first device. The second control may be, for example, a button, including a first button and a second button, the first button being triggered to accept the pairing of the first device, and the second button being triggered to reject the pairing of the first device.

[0295] 538, the target second device acquires the triggering of the second control, and sends a response instruction generated by the triggering of the second control to the server.

[0296] If it is detected that the first button is triggered, it is determined that the target second device accepts the pairing of the first device, and 539 is entered; if it is detected that the second button is triggered, it is determined that the target second device rejects the pairing of the first device, and 543 is entered.

[0297] 539, the server sends the device network information of the opposite device to at least one of the first device and the target second device.

[0298] 540, at least one of the first device and the target second device receives the device network information of the opposite device sent by the server, and establishes pairing with the opposite device based on the device network information.

[0299] 541, the server sends a pairing success message to the first device and the target second device.

[0300] 542, the first device receives the pairing success message sent by the server, and displays pairing success on the first configuration interface; the target second device receives the pairing success message sent by the server, and displays pairing success on the second configuration interface.

[0301] 543, the server sends a pairing failure message to the first device.

[0302] 544, the first device receives the pairing failure message sent by the server, and displays pairing failure on the first configuration interface.

[0303] The pairing success or pairing failure can be displayed by popping up a dialog box on the corresponding configuration interface.

[0304] The method of 531-535 is the same as that of 501-505 of the foregoing embodiment, and will not be repeated here. The method of pairing failure and unpairing can also be the same as the principle described in the foregoing embodiment, and will not be repeated here.

[0305] <UI interaction of administrator direct connection>

[0306] In some embodiments, the utility model provides a method for pairing, referring to Figure 24 , the method comprises:

[0307] 601, display the acquired first device information of the first device registered to the server in a configuration interface of the display screen.

[0308] 602, acquire target first device information associated with the target first device determined based on the first device information.

[0309] 603, based on the target first device information, match second device information of a second device registered to the server and capable of pairing with the target first device, and display in the configuration interface.

[0310] 604, acquire target second device information associated with the target second device determined based on the second device information.

[0311] 605, acquire device network information of at least one of the target first device and the target second device.

[0312] 606, send the device network information to the opposite device associated with the network information.

[0313] 607, the opposite device receives the network information and establishes pairing with the local device associated with the network information based on the network information to realize direct communication.

[0314] In some embodiments, referring to Figures 25 to 28 , the first device information can be displayed in the configuration interface through the list control 1a. Different first device information is associated with different list options of the list control 1a. When it is detected that the target list option in the list option is selected, the first device information associated with the target list option can be determined as the target first device information of the target first device expected to be used for pairing.

[0315] At the same time or slightly lagging, when it is detected that the target list option in the list option 1a is selected, the second device information of the second device capable of pairing with the target first device can be matched from the second device registered to the server based on the target first device information associated with the target list option. The matched second device information of the second device capable of pairing can be displayed in the configuration interface through another list control 1b for the doctor to configure. Different second device information is associated with different list options of the other list control 1b. When it is detected that the target list option in the other list option 1b is selected, the second device information associated with the target list option can be determined as the target second device information of the target second device expected to be used for pairing.

[0316] In some embodiments, after the target first device and the target second device are determined, the device network information of at least one of the target first device and the target second device can be acquired in response to a triggering event. The triggering event includes that a timer started from the determination of the target second device reaches a delay time, or a control in the configuration interface is triggered, the control being a control for triggering the acquisition of the device network information of at least one of the target first device and the target second device, for example, the control being the confirmation button 1c in Figure 28 . In the case that the server acquires that the timer reaches the delay time, or acquires that the control is triggered, the device network information of at least one of the target first device and the target second device is acquired.

[0317] In some embodiments, after the target first device and the target second device are paired, at least one of the target first device and the target second device can return a pairing success status to the server. The server receives the pairing success status, displays a message of successful pairing in the configuration interface, and sends a message of successful pairing to the target first device and the target second device. The configuration interface in the target first device and the target second device can display the message of successful pairing. For example, the message of successful pairing can be displayed by a pop-up dialog box.

[0318] In some embodiments, referring to Figures 29 to 31 , the first device information can be displayed in the configuration interface by the first drag control 1a'. Different first device information is associated with different first drag controls 1a'. The configuration interface can include a window control 1c' having a window. Upon detecting that the target first drag control in the first drag control 1a' moves into the window of the window control 1c', the first device information of the first device associated with the target first drag control can be determined as the target first device information of the target first device for pairing.

[0319] Simultaneously or slightly delayed, upon detecting that the target first drag control in the first drag control 1a' moves into the window of the window control 1c', the second device information of the pairable second device that can be paired with the target first device can be matched from the second devices registered to the server based on the target first device information associated with the target first drag control. The matched second device information of the pairable second device can be displayed in the configuration interface outside the window by the second drag control 1b', and different second device information is associated with different second drag controls 1b'. Upon detecting that the target second drag control in the second drag control 1b' moves into the window 1c', the second device information of the second device associated with the target second drag control can be determined as the target second device information of the target second device for pairing. For example, as Figure 29 , the first devices that can be paired include the remote doctor consoles of the hospitals A-D; and as Figure 30When the first drag control 1a' associated with the remote doctor console of the A hospital is moved to the window 1c', the server automatically matches the second device information of the second device that is compatible with the custom protocol supported by the remote doctor console of the A hospital for pairing, such as the patient surgery platforms of the A-E hospitals; as shown in Figure 31 When the second drag control 1b' associated with the patient surgery platform of the B hospital is moved to the window 1c', it is determined that the patient surgery platform of the B hospital is the device expected to be paired with the remote doctor console of the A hospital.

[0320] In the above embodiment, referring to Figures 32 to 34 The window control 1c' can include a first area 11c' and a second area 12c'. The first area 11c' can include a first sub-area 111c' and a second sub-area 112c', the first sub-area 111c' and the second sub-area 112c' are configured to accommodate the first drag control 1a', and the second area 12c' is configured to accommodate the second drag control 1b'. The first sub-area 111c' and the second sub-area 112c' are configured to associate different control authorities or different pairing sequences. Among them, in the case that the first drag control 1a' is associated with the remote doctor console and the second drag control 1b' is associated with the patient surgery platform, when the second drag control 1b' is moved to the second area 12c', one first drag control 1a' is moved to the first sub-area 111c', and another first drag control 1a' is moved to the second sub-area 112c', the remote doctor console associated with different first drag controls 1a' can have different control authorities or pairing sequences for the patient surgery platform associated with the second drag control 1b'. For example, as shown in Figure 32 The first device that can be paired includes the remote doctor consoles of the A-D hospitals; as shown in Figure 33 When the first drag controls 1a' associated with the remote doctor consoles of the A and B hospitals are moved to the first sub-area 111c' and the second sub-area 112c' respectively, the server automatically matches the second device information of the second device that is compatible with the custom protocol supported by the remote doctor consoles of the A and B hospitals for pairing, such as the patient surgery platforms of the A-C hospitals; as shown in Figure 34 When the second drag control 1b' associated with the patient surgery platform of the B hospital is moved to the second area 12c', it is determined that the patient surgery platform of the B hospital is the device expected to be paired with the remote doctor consoles of the A and B hospitals. Among them, if the first sub-area 111c' and the second sub-area 112c' are associated with different control authorities or different pairing sequences, the remote doctor consoles of the A and B hospitals are configured to have different control authorities or different pairing sequences for the patient surgery platform of the B hospital.

[0321] <Paired successfully detection>

[0322] In some embodiments, when the remote physician console and the patient surgery platform are paired in real time, a pairing success detection can be performed to ensure that the remote surgery can be carried out smoothly. In this regard, at least one network channel (i.e., communication link) established by the remote physician console and the patient surgery platform when paired in real time can be utilized to determine whether the pairing between the two is truly successful.

[0323] In a super-remote surgery process, multiple types of network information can be involved in the interaction between the remote physician console and the patient surgery platform. For example, these types of network information include first network information involved in the interaction between the remote physician console and the core devices in the patient surgery platform, and second network information involved in the interaction between the remote physician console and the auxiliary devices in the patient surgery platform.

[0324] The first network information can include different types of network information, such as at least one of motion control information, energy control information, multimedia interaction information, and human-computer interface interaction information. In this regard, the motion control information can be usually issued by the remote physician console and executed by the slave robot, such as master-slave following control; the motion control information can also be issued by the slave robot and executed by the remote physician console, such as when the remote physician console includes an operating part in the form of a mechanical linkage, the slave robot issues motion control information for the pose of its medical instrument, and the operating part of the remote physician console executes control for aligning the pose of the medical instrument. The energy control information can be usually issued by the remote physician console and executed by the energy platform, such as controlling the conduction of a monopolar circuit, the conduction of a bipolar circuit, the conduction of an ultrasonic transducer, the conduction of a radio frequency circuit, or the conduction of a microwave circuit, etc. The multimedia interaction information includes video interaction information and audio interaction information, the video interaction information is usually collected by an imaging device such as an endoscope, processed by the image platform, and sent to the display screen of the remote physician console for display; the audio interaction information is usually collected by audio intercom devices respectively configured at both ends of the remote physician console and the slave robot, processed by the image platform, and sent to the opposite end device for playing. The human-computer interaction information is usually operated by the remote physician console on the touch screen of the display screen, and the interface changed after the operation is displayed on the display screen of the slave robot, such as image annotation.

[0325] The second network information can also include different types of network information, and the remote physician console usually has different network information interactions with different auxiliary devices, and the remote physician console has at least one network information interaction with the same auxiliary device. For example, when the auxiliary device includes a monitor, these interaction information includes at least one of heart rate, pulse, and blood pressure information; for example, when the auxiliary device includes a surgical bed, the remote physician console can be configured to control the position or pose of the surgical bed, and these interaction information includes motion control information of the surgical bed by the remote physician console.

[0326] In some embodiments, when the remote physician console and the patient surgery platform are paired, a network channel is created between the remote physician console and the patient surgery platform for communication of network information between the two, based on the device network information sent by the server. In a teleoperation surgery, different network channels can be created to meet the transmission requirements of different types of network information between the remote physician console and the patient surgery platform. By creating different network channels for the transmission of different types of network information, a plurality of advantages can be achieved, such as facilitating bandwidth optimization, facilitating priority management, facilitating data isolation to improve security, facilitating fault isolation to improve the fault tolerance of the network, and facilitating compliance of specific data transmission.

[0327] The different network channels can be created by configuring the port numbers of the remote physician console and the patient surgery platform to distinguish the transmission of different types of network information. For example, a motion control network channel can be created for the transmission of motion control information, an energy control network channel can be created for the transmission of energy control information, a multimedia interaction network channel can be created for the transmission of multimedia interaction information, and a human-computer interface interaction network channel can be created for the transmission of human-computer interface interaction information.

[0328] In some embodiments, the remote physician console or the patient surgery platform can create different network channels for the transmission of network information of corresponding data types between the remote physician console and the patient surgery platform according to different configured control permissions, wherein different control permissions can be associated with the transmission of different data types. In some embodiments, different network channels can be created for the transmission of network information between corresponding device types in the remote physician console and the patient surgery platform according to different physician-patient information input in the remote physician console or the patient surgery platform, wherein different physician-patient information can be associated with the transmission of network information between different device types. For example, the physician-patient information can include the type of surgery or the surgical equipment to be used, and different physician-patient information can require the use of different core devices or auxiliary devices. Therefore, different network channels can be created based on the number or type of devices to be connected.

[0329] In some embodiments, the utility model also provides a method for detecting whether the pairing is successful, referring to Figure 35 The method comprises:

[0330] 711, the server obtains the information of the target network channel expected to be established between the first device and the second device, and based on the obtained target network channel information, sends the device network information of the opposite device associated with the target network channel to at least one of the first device and the second device.

[0331] The first device and the second device have a pairing relationship. Different network channels usually have different port numbers. The server sends a port number associated with a target network channel to at least one of the first device and the second device.

[0332] 712. At least one of the first device and the second device receives the device network information associated with the target network channel of the opposite device sent by the server, and creates the target network channel with the opposite device based on the device network information.

[0333] 713. At least one of the first device and the second device evaluates the communication state of the target network channel.

[0334] In some embodiments, the communication state of the target network channel can be evaluated by detecting network connectivity. For example, a heartbeat mechanism can be used to detect the network connectivity of the target network channel. Using the heartbeat mechanism requires creating a sending end and a receiving end at both ends of the target network channel. One of the first device and the second device can act as the sending end, and the other can act as the receiving end. When the sending end and the receiving end are used as a pair, both can use TCP protocol or UDP protocol. When using the heartbeat mechanism for detection, the sending end sends a message to the receiving end through the target network channel. If the receiving end returns a signaling indicating that the message has been received within a predetermined time, it indicates that the network connection between the sending end and the receiving end is normal, i.e., the network connectivity of the target network channel between the two is normal. If the receiving end does not return the signaling indicating that the message has been received within the predetermined time, it indicates that the network connection between the sending end and the receiving end is disconnected, i.e., the network connectivity of the target network channel between the two is abnormal.

[0335] In some embodiments, the target network channel includes one or more. When the target network channel includes multiple, different sending ends and receiving ends can be created for different target network channels. In other words, when the target network channel includes a first target network channel and a second target network channel, for the first target network channel, the first device can act as the sending end and the second device can act as the receiving end; for the second target network channel, the second device can act as the sending end and the first device can act as the receiving end.

[0336] In some embodiments, the communication status of the target network channel can also be evaluated by detecting network communication quality, for example, by detecting at least one of network delay, jitter, packet loss and out-of-order of packets transmitted in the target network channel. For example, the network communication quality can be evaluated by evaluating network delay. Also, a sending end and a receiving end are created at both ends of the target network channel, the sending end sends a preset type of instruction to the receiving end through the target network channel, the preset type of instruction can be related to the type of network information to be transmitted by the target network, the instruction is encapsulated into a packet according to a custom protocol agreed between the devices at both ends of the target network channel, the packet includes a sending timestamp, and the receiving end returns a signaling to the sending end through the target network channel after receiving the packet, the signaling includes a receiving timestamp when the receiving end receives the packet, the sending end calculates the difference between the receiving timestamp and the sending timestamp, and compares the difference with a preset delay threshold corresponding to the target network channel. For example, when the difference is less than the preset delay threshold, it is determined that the network communication quality is good; when the difference exceeds a first range of the preset delay threshold, it is determined that the network communication quality is fair; and when the difference exceeds a second range of the preset delay threshold, it is determined that the network communication quality is poor. The first range is smaller than the second range. In some embodiments, when the network communication quality is a first quality, for example, good or fair, it can be determined that the network communication quality of the target network channel is normal; and when the network communication quality is a second quality, for example, poor, it can be determined that the network communication quality of the target network channel is abnormal.

[0337] 714, when the communication status of the target network channel is normal, at least one of the first device and the second device determines that the pairing of the first device and the second device is successful.

[0338] 715, when the communication status of the target network channel is abnormal, at least one of the first device and the second device determines that the pairing of the first device and the second device fails.

[0339] The target network channel can include multiple target network channels. Whether the pairing between the first device and the second device is successful generally needs to consider the communication status of multiple target network channels.

[0340] For example, network connectivity is a key indicator for evaluating the communication status. If the network connectivity of all target network channels is normal, it can be determined that the pairing of the first device and the second device is successful; if the network connectivity of one or more target network channels is abnormal, it can be determined that the pairing of the first device and the second device fails. For example, network communication quality is a key indicator for evaluating the communication status. If the network communication quality of all target network channels is normal, it can be determined that the pairing of the first device and the second device is successful; if the network communication quality of one or more target network channels is abnormal, it can be determined that the pairing of the first device and the second device fails.

[0341] In some embodiments, for example, when the first device or the second device is configured to have the operator's authority, the pairing success of the first device and the second device can be evaluated in combination with the priority of the network information transmitted by the target network channel and the communication state of the target network channel. Some target network channels are used to transmit network information with high priority, such as motion control information, energy control information, and video information. Some target network channels are used to transmit network information with low priority, such as audio information and human-computer interaction information. In order to improve the pairing success rate, the system can have a high degree of redundancy. For example, as long as the communication state of all target network channels used to transmit network information with high priority is normal, and the communication state of all target network channels used to transmit network information with low priority is abnormal, it can be determined that the first device and the second device are paired successfully. For example, as long as the communication state of any target network channel used to transmit network information with high priority is abnormal, it can be determined that the first device and the second device are paired unsuccessfully.

[0342] After 714 or 715, a pairing success or pairing failure prompt can be performed in the first device and the second device.

[0343] In 711, at least one of the first device and the second device can obtain the configured control authority or the doctor inputted patient information, and can determine the information of the target network channel expected to be established between the first device and the second device based on the control authority or the patient information, and send the determined information of the target network channel to the server. The server can also obtain the configured control authority or the doctor inputted patient information, and can determine the information of the target network channel expected to be established between the first device and the second device based on the control authority or the patient information.

[0344] In some embodiments, 711 can be replaced by that the server sends the device network information of the opposite device to at least one of the first device and the second device. The device network information includes all possible network channels that can be created.

[0345] 712 can be replaced by that at least one of the first device and the second device receives the device network information of the opposite device, and can determine the device network information associated with the target network channel from the device network information based on the configured control authority or the patient information, and further create the target network channel with the opposite device based on the device network information of the target network channel for communication.

[0346] 712 Alternatively, at least one of the first device and the second device receives device network information of the counterpart device, and creates all network channels with the counterpart device for communication based on the device network information. 713 Alternatively, at least one of the first device and the second device determines a target network channel from the all network channels based on the configured control authority or the doctor-patient information, and further evaluates the communication status of the target network channel. In this embodiment, the first device and the second device create all network channels, but only the communication status of the necessary network channel, i.e., the target network channel, is detected.

[0347] In some embodiments, upon determining that the first device and the second device are successfully paired, the corresponding device, such as the first device or the second device, can allow or prohibit the use of the corresponding control authority according to the different network communication qualities of the plurality of target network channels; or can allow or disable the implementation of the corresponding surgical procedure according to the different network communication qualities of the plurality of target network channels. For example, when the network communication quality of the plurality of target network channels used for transmitting motion control information, energy control information, and video information is better, the relatively dangerous surgical procedure, such as surgery on arteries and organs such as lungs, livers, kidneys, and stomachs, can be allowed to be performed; for example, when the network communication quality of any one of the plurality of target network channels used for transmitting motion control information, energy control information, and video information is general, only the relatively safe surgical procedure, such as surgery on skin and fat, can be allowed to be performed.

[0348] In some embodiments, the control authority of the remote doctor console over the patient surgery platform includes, but is not limited to, the control authority over at least one of the slave robot, the image platform, and the energy platform.

[0349] For example, the control authority includes operator authority and observer authority, and the operator authority has a higher level than the observer authority. The operator authority covers the observer authority, i.e., the observer authority is part of the operator authority. Therefore, the network channels required to be enabled by the operator authority include the network channels required to be enabled by the observer authority, and the number of the network channels required to be enabled by the operator authority is more than the number of the network channels required to be enabled by the observer authority.

[0350] In some embodiments, the observer permission is applicable to an observer identity, and is mainly applied to a teaching scenario. A trainee can learn by observing a doctor's operation implementation process, and the doctor can also observe the trainee's operation implementation process to guide. The observer permission is mainly reflected in the image platform. When configuring the observer permission, a network channel between the remote doctor console and the image platform usually needs to be created. The image platform can process multimedia interactive information, and can also process human-computer interface interactive information. The multimedia interactive information includes, for example, video information and audio information, and the human-computer interface interactive information includes, for example, annotation information or annotation information of a surgical image collected by a doctor on an endoscope and displayed on a display screen of the remote doctor console. The video information can be sourced from an endoscope or an external camera electrically connected to the image platform and mounted on a slave robot. The external camera is used to monitor the working state of at least part of the equipment of the patient operation platform. After the video information and the human-computer interface interactive information are processed by the image platform, they are transmitted to the remote doctor console through the network channel between the image platform and the remote doctor console and displayed on the display screen of the remote doctor console. The audio information can be sourced from an audio device electrically connected to the image platform and mounted on the slave robot. The audio device includes an input device such as a microphone and an output device such as a loudspeaker. The remote doctor console also includes an audio device. The input device of the remote doctor console and the slave robot can collect sound input such as voice talkback of both ends of the user, and after processing such as noise reduction by the image platform, the sound input is transmitted to the output device of the opposite end device through the network channel between the remote doctor console and the image platform and played. Different network channels can be created between the remote doctor console and the image platform to transmit different types of multimedia interactive information according to the types of the multimedia interactive information. For example, under the observer permission, when different network channels are created, three network channels can be created respectively, one for transmitting video information, one for transmitting human-computer interface interactive information, and one for transmitting audio information.

[0351] In some embodiments, the operator authority is applicable to an operator identity, and the operator includes a physician who performs a surgery. The operator authority can be embodied in various devices in a patient surgery platform, including an imaging platform, a slave robot, and an energy platform. The physician performing the surgery can include motion control of a medical instrument (including an energy instrument) assembled on the slave robot under a surgical view provided by the imaging platform, and energy control of the energy instrument assembled on the slave robot through control of the energy platform. During the surgery, the physician can mark or annotate the surgical image, or communicate with a physician or a patient on the side of the slave robot. Therefore, under the operator authority, the remote physician console can create multiple network channels with the imaging platform, the slave robot, the energy platform, and auxiliary devices for transmission of different types of network information. In some embodiments, the remote physician console and any device can create multiple network channels according to different types of signals transmitted therebetween or different physical communication links. In some embodiments, for example, for a laparoscopic slave robot or an interventional slave robot, the slave robot includes multiple manipulators, each of which is used to assemble a medical instrument, and the motion control of the medical instrument by the remote physician console is mainly motion control of the manipulators, and the motion control of different manipulators by the remote physician console is independent of each other, and their physical communication links are different, so that multiple motion control network channels associated with different manipulators can be created when the motion control network channel between the remote physician console and the slave robot is created. In some embodiments, the energy platform can include processing of different types of energy control signals, and multiple energy control network channels associated with different types of energy control signals can be created when the energy control network channel between the remote physician console and the energy platform is created.

[0352] In some embodiments, different control authorities can be configured more finely. When a certain specific authority is allowed, any device can open or create a network channel corresponding to the authority. When a certain specific authority is prohibited, any device can close or destroy a network channel corresponding to the authority. Disabling a certain specific authority can help to shield interference on the one hand, and to release bandwidth resources when the network bandwidth is insufficient on the other hand. For example, when the visual authority with high network bandwidth occupation is disabled, the released bandwidth resources can help to ensure transmission of other information.

[0353] In some embodiments, for the observer authority, the visual authority, the audible and speakable authority, and the markable authority can be included for a first network channel for transmission of video information, a second network channel for transmission of audio information, and a third network channel for transmission of human-computer interface interaction information, respectively. For example, the observer authority can be configured to include the visual authority and the audible and speakable authority, and the first network channel and the second network channel are opened or created, and the third network channel is closed or destroyed.

[0354] In some embodiments, for operator permissions, partial operator permissions and full operator permissions can be included, both of which can override observer permissions. A remote physician console can obtain partial operator permissions to a patient surgical platform, which is suitable for use when multiple remote physician consoles are needed to cooperatively manipulate a patient surgical platform to perform surgery. A remote physician console can also obtain full operator permissions to a patient surgical platform, which is suitable for use when a single remote physician console is needed to independently manipulate a patient surgical platform to perform surgery.

[0355] In some embodiments, partial operator permissions include, for example, permissions to control a subset of manipulators or a subset of types of medical instruments from a slave robot. For example, a slave robot can include a first manipulator, a second manipulator, a third manipulator, and a fourth manipulator, one of which can be equipped with a medical instrument, and the medical instruments equipped by different manipulators can be the same or different, partial operator permissions can be configured such that a remote physician console can only control the first and second manipulators, and the third and fourth manipulators not configured can be configured to be controlled by one or more other remote physician consoles, wherein the configuration of partial operator permissions can be implemented based on the unique identification of the manipulators, such as the number of the manipulators. For another example, the medical instruments can include first, second, third, and fourth types of medical instruments, the types of medical instruments are recorded in the storage device thereof, and are identified by a read-write device provided in the manipulator when the medical instrument is installed to the manipulator of the slave robot, and can be sent to the remote physician console through a motion control network channel between the slave robot and the remote physician console. The remote physician console can determine whether it can control the medical instrument according to the type of the medical instrument obtained, wherein partial operator permissions can be configured such that a remote physician console can only control the first and second types of medical instruments, and the third and fourth types of medical instruments not configured can be configured to be controlled by one or more other remote physician consoles. For example, the first type of medical instrument is surgical forceps, the second type of medical instrument is an endoscope, the third type of medical instrument is an electrotome, and the fourth type of medical instrument is an anastomat.

[0356] In some embodiments, full operator permissions include, for example, permissions to control all manipulators or all types of medical instruments from a slave robot.

[0357] In some embodiments, after 715, when prompting the pairing failure, the target network channel with abnormal communication status can be prompted to facilitate the doctor to detect or maintain the target network channel, or to facilitate the doctor to reconfigure the control authority to avoid the pairing failure caused by the target network channel with the corresponding abnormal communication status. For the latter, for example, in the scenario of a teaching doctor, when the doctor learns from the prompt of the pairing failure that the network channel corresponding to the visual permission of the observer authority is normal, and the network channel corresponding to the audible and speaking permission is abnormal, the doctor can reconfigure the observer authority to only include the visual permission to ensure the pairing success, and even if the network channel corresponding to the audible and speaking permission is closed or destroyed, the implementation of the teaching function will not be greatly hindered.

[0358] In some embodiments, when creating the target network channel with the peer device, different target network channels with different communication protocol standards can be created according to the different control authorities obtained. For example, when the control authority is the observer authority, the target network channel with the UDP protocol standard can be created, such as the multimedia network channel and the human-computer interface interaction network channel using the UDP protocol standard, which has faster signaling transmission speed. For another example, when the control authority is the operator authority, for the observer authority covered by the operator authority, the target network channel with the UDP protocol standard can be created, and for other authorities, the target network channel with the TCP protocol standard can be created, such as the motion control network channel and the energy control network channel using the TCP protocol standard, which has more reliable signaling transmission and less packet loss and out-of-order, and can ensure the reliable implementation of the remote surgery. Of course, the target network channels can also have unified communication protocol standards.

[0359] In some embodiments, the target network channel is created according to the doctor-patient information, which can include the type of surgery expected to be implemented by the doctor or the patient. For example, when the type of surgery is laparoscopic surgery, the target network channel between the remote doctor console and the corresponding device of the (single-hole / multi-hole) laparoscope slave robot can be created; when the type of surgery is (bronchial / vessel) interventional surgery, the target network channel between the remote doctor console and the corresponding device of the (bronchial / vessel) interventional slave robot can be created. The doctor-patient information can include the surgical equipment required for the surgery. For example, when the doctor-patient information includes the surgical equipment required for the slave robot, the monitor, and the ventilator, the target network channel between the remote doctor console and the slave robot, the monitor, and the ventilator can be created; when the doctor-patient information includes the surgical equipment required for the slave robot without the monitor and the ventilator, the target network channel between the remote doctor console and the slave robot can be created.

[0360] In some embodiments, the configuration of control authorities can also be included in the whole pairing phase. Whether the control authorities are successfully configured can affect whether the telesurgery can be reliably implemented, and the successful configuration of the control authorities is an important reference to measure whether the pairing is successful. Among the control authorities, the successful configuration of the operator authority is particularly important to ensure the reliable implementation of the telesurgery. In most cases, a telesurgery mainly controls the core equipment in the patient surgery platform, including the motion control of the slave robot by the remote physician console and the energy control of the energy platform by the remote physician console. Therefore, the operator authority usually involves the motion control authority and the energy control authority. In some cases, however, the remote physician console paired with the patient surgery platform in real time can include one or more, and when multiple remote physician consoles are included, one or more remote physician consoles can be configured to have only observer authority, and one or more remote physician consoles can be configured to have operator authority. Among them, one remote physician console can be configured to have full operator authority, that is, both the motion control authority of the slave robot and the energy control authority of the energy platform are obtained by the remote physician console. One remote physician console can be configured to have partial operator authority, for example, including one of the motion control authority of the slave robot and the energy control authority of the energy platform obtained by the remote physician console; or, including partial motion control authority of the slave robot obtained by the remote physician console; or, including partial energy control authority of the energy platform obtained by the remote physician console, wherein the energy platform can include multiple energy output links, and different energy output links can be controlled by different remote physician consoles, that is, the multiple energy output links can include multiple energy control authorities.

[0361] In a master-slave surgical robot system, the operator authority is usually transparent to the operation end and the execution end, that is, the operation end and the execution end usually know the operator authority they have, for example, the operation end knows which operations he can control the execution end to perform, and the execution end knows which operations he can perform controlled by the operation end. Among them, one of the operation end and the execution end is a remote physician console, and the other can be a patient surgery platform. For example, in master-slave control, the operation end is a remote physician console, and the execution end is a slave robot or an energy platform of a patient surgery platform; and in master-slave alignment, the operation end is a slave robot, and the execution end is a remote physician console.

[0362] In some embodiments, the successful configuration of the operator authority includes at least two aspects. The first aspect includes that the possession (i.e. permission) of the authority is a successful configuration, indicating that the object desired to be operated can be operated. The second aspect includes that the non-possession (i.e. prohibition) of the authority is a successful configuration, indicating that the object desired not to be operated cannot be operated. Whether the operator authority is successfully configured includes the detection of at least one of the two aspects. Based on this, for the remote physician console and the patient surgery platform that have established real-time pairing, the utility model provides a kind of method for detecting whether the operator authority is successfully configured, and the method comprises:

[0363] 721, at least one of remote physician console and patient surgery platform obtains the possession authority in configured operator authority.

[0364] The possession authority mainly includes authority attribution, and the authority attribution includes association with controlled object, and the operation end can only control the controlled object in the execution end, and cannot control the non-controlled object. From the remote physician console, the controlled object can include the specific manipulator component in the slave robot or the manipulator component associated with the specific type of medical instrument desired to be controlled;The controlled object can include the specific energy output link in the energy platform desired to be controlled by the remote physician console. From the patient surgery platform, the controlled object can include the specific manipulator component in the slave robot or the manipulator component associated with the specific type of medical instrument desired to be controlled by the remote physician console;The controlled object can include the specific energy output link in the energy platform desired to be controlled by the remote physician console. Thus, whether it is the remote physician console or from the patient surgery platform, the authority attribution of both ends is corresponding. Wherein, "specific" includes all or part.

[0365] Different authority attribution can be closely associated with the authority type of operator authority. For example, when the controlled object includes manipulator component, the authority type is usually associated with motion control authority;For example, when the controlled object includes specific energy output link, the authority type is usually associated with energy control authority. That is, when configuring the operator authority, the authority attribution is usually configured, and the associated authority type can be configured by default according to the authority attribution.

[0366] In addition, based on the difference of the controlled object authority type, different types of control instructions are usually required for control. For example, when the authority type is motion control authority, motion control instructions are required to control the controlled object;When the authority type is energy control authority, energy control instructions are required to control the controlled object.

[0367] 722, the remote physician console determines the controlled object from the patient surgery platform based on the authority attribution of the possession authority, and determines the target control instruction associated with the controlled object from the preset control instruction set.

[0368] Even if two slave robots in two patient surgery platforms are of the same type, due to differences in machining and mechanical assembly, the two slave robots can be slightly different in kinematics, and thus a preset control instruction set can be designed for each patient surgery platform. The preset control instruction set includes motion control instructions simulating the operation of the operation part of the remote physician console and energy control instructions simulating the operation of the foot pedal assembly of the remote physician console, both of which can include at least one control instruction.

[0369] For example, in a laparoscopic slave robot, different motion control instructions can be designed for different manipulator assemblies in the slave robot. The motion control instructions are mainly used to control the manipulators in the manipulator assemblies to control the medical instruments mounted on the manipulators, and the motion control instructions are usually a pose instruction, which includes at least one of a position instruction and an attitude instruction.

[0370] In some embodiments, the slave robot includes a main arm and a plurality of manipulators connected to the distal end of the main arm. When the slave robot is powered on and initialized, the main arm is unfolded to drive the manipulators to a first initial pose fixed relative to a base coordinate system. When the medical instrument is mounted on the manipulator, the initialization operation of the manipulator on the medical instrument also enables the medical instrument to be kept in a second initial pose fixed relative to the base coordinate system, and the second initial pose is different from the first initial pose. The medical instrument includes an endoscope and a surgical instrument. Generally, the motion of the endoscope is performed relative to the base coordinate system, and the motion of the surgical instrument is performed relative to the endoscope coordinate system. However, when detecting whether the motion control authority is configured successfully, the control mode can be simplified, for example, the remote physician console can be defined to control the motion of all types of medical instruments relative to the base coordinate system.

[0371] The kinematic models of the manipulators in the same manipulator assembly of the slave robot are basically the same, and the kinematic models of the medical instruments can be the same or different.

[0372] Assuming that the kinematic models of different medical instruments suitable for the slave robot are substantially the same, the target poses of the medical instruments relative to the base coordinate system can be determined based on the current joint parameters of the joints in the manipulator and the target joint increments of at least part of the joints, in the case that the medical instruments are in the second initial poses relative to the base coordinate system, and based on the kinematic model of the manipulator and the kinematic models of the medical instruments. The target joint increments are pre-designed and can be a small value, for example, when the joint parameters are joint angles, the target joint increments can be designed as 0.1°, 0.5°, 1°, 1.5°, 2°, or other values between any two of the foregoing, etc., to reduce the amplitude of the subsequent motion for verifying whether the motion control authority is successfully configured. Of course, the target poses of the manipulator assemblies can also be directly specified poses relative to the base coordinate system, and the target poses are different from the corresponding second initial poses, for example, the target poses are upward, downward, leftward, rightward, forward or backward poses relative to the second initial poses. In this case, the target poses of the manipulator assemblies will not change because the manipulator is installed with different medical instruments, that is, each manipulator assembly is associated with a relatively fixed target pose. The target poses designed here are target motion control instructions for detecting whether the motion control authority is successfully configured. Further, in step 722, the steps of determining the controlled object from the patient surgery platform and determining the target control instruction associated with the controlled object from the preset control instruction set include: determining the controlled manipulator assembly from the slave robot, and determining the target motion control instruction associated with the controlled manipulator assembly from the preset control instruction set based on the controlled manipulator assembly.

[0373] Assuming that the kinematic models of different medical instruments suitable for the slave robot are substantially different, the target poses of different medical instruments installed on different manipulators can be pre-designed based on the similar method described above, in the case that the manipulator and the medical instrument are in the corresponding initial pose state. The target poses of the manipulator assemblies are different when the same medical instrument is installed on different manipulators, and the target poses of the manipulator assemblies are also different when different medical instruments are installed on the same manipulator. That is, the target pose of the manipulator assembly is associated with the medical instrument itself and the manipulator on which the medical instrument is installed. Further, in step 722, the steps of determining the controlled object from the patient surgery platform and determining the target control instruction associated with the controlled object from the preset control instruction set include: determining the controlled manipulator assembly from the slave robot, identifying the medical instrument in the controlled manipulator assembly, and determining the target motion control instruction associated with the controlled manipulator assembly from the preset control instruction set based on the controlled manipulator assembly and the medical instrument installed thereon.

[0374] In some embodiments, the energy control instruction is relatively simple in design. When the energy platform includes multiple energy output links, the energy control instruction includes multiple energy control instructions corresponding to the multiple energy output links. These energy control instructions can include switch control instructions, current regulation instructions, voltage regulation instructions, power regulation instructions, or frequency regulation instructions, etc. Further, in 722, the step of determining a controlled object from the patient surgery platform and determining a target control instruction associated with the controlled object from the preset control instruction set includes determining a controlled energy output link from the energy platform and determining a target energy control instruction associated with the controlled energy output link from the preset control instruction set.

[0375] 723, the remote physician console sends the target control instruction to the patient surgery platform.

[0376] The communication between the remote physician console and the patient surgery platform is achieved through a network channel established between the two, and the communication includes the transmission of the target control instruction and the transmission of the actual execution result of executing the target control instruction.

[0377] 724, the patient surgery platform receives the target control instruction, and the controlled object of the patient surgery platform executes the target control instruction.

[0378] In some embodiments, the physician on the controlled object side can observe the execution of the target control instruction by the controlled object and notify the remote physician console or the patient surgery platform of the observation result, which includes execution success or execution failure. Execution success can indicate that the possession permission configuration in the operator permission is successful, and execution failure can indicate that the possession permission configuration in the operator permission fails.

[0379] In some embodiments, referring to Figure 36 The method can further include:

[0380] 725, after the controlled object executes the target control instruction, the patient surgery platform returns the actual execution result to the remote physician console.

[0381] Corresponding to the target control instruction being a motion control instruction, the actual execution result can include the actual pose of the corresponding manipulator component relative to the base coordinate system. Corresponding to the target control instruction being an energy control instruction, the actual execution result can include the actual energy output state of the corresponding energy transmission link, and the actual energy output state includes an actual switch state, an actual current output value, an actual voltage output value, an actual power output value, or an actual frequency output value.

[0382] 726, the remote physician console receives the actual execution result and detects whether the actual execution result matches the expected execution result.

[0383] The expected execution result can be an execution result determined based on the target control instruction in an ideal state. Corresponding to the target control instruction being a motion control instruction, the expected execution result can include an expected pose of the corresponding manipulator component relative to the base coordinate system. Corresponding to the target control instruction being an energy control instruction, the expected execution result can include an expected energy output state of the corresponding energy transmission link, the expected energy output state being an expected switch state, an expected current output value, an expected voltage output value, an expected power output value, or an expected frequency output value.

[0384] 727, upon detecting that the actual execution result matches the expected execution result, determining that the possession right configuration is successful.

[0385] The actual execution result matching the expected execution result includes at least one layer of meaning. This layer of meaning includes that the controlled object will act in response to the target control instruction. This layer of meaning includes that the controlled object moves in response to the motion control instruction, or the controlled object outputs energy in response to the energy control instruction.

[0386] The actual execution result matching the expected execution result can also include another layer of meaning. This layer of meaning includes that the controlled object accurately acts in response to the target control instruction. This layer of meaning includes that, corresponding to the target control instruction being a motion control instruction, the deviation between the actual pose and the expected pose is within a deviation threshold in the same coordinate system; or, corresponding to the target control instruction being an energy control instruction, the actual energy output state and the expected energy output state should be consistent, for example, in an open state, when the actual energy output state is an actual switch state and the expected energy output state is an expected switch state; or, when the actual energy output state is an actual current output value, an actual voltage output value, an actual power output value, or an actual frequency output value, and the expected energy output state is an expected current output value, an expected voltage output value, an expected power output value, or an expected frequency output value, the deviation between the actual energy output state and the expected energy output state is within a deviation threshold.

[0387] 728, upon detecting that the actual execution result does not match the expected execution result, determining that the possession right configuration fails.

[0388] Through 725-728, it can be more intelligently determined whether the possession right in the operator right is configured successfully.

[0389] In some embodiments, referring to Figure 37 The method can further include detecting whether the non-possession right in the operator right is configured successfully.

[0390] 731, the remote doctor console and the patient surgery platform acquire the non-possession right in the configured operator right.

[0391] The operator's permissions include owned permissions and unowned permissions, the unowned permissions can be acquired independently or according to the owned permissions in the operator's permissions. As described above, the permission attribution of the owned permissions is associated with the controlled objects in the patient surgery platform, and the permission attribution of the unowned permissions is associated with the non-controlled objects in the patient surgery platform. It is assumed that different non-controlled objects need to send control instructions corresponding to the non-controlled objects if the non-controlled objects can be controlled, for example, the non-controlled objects are manipulator assemblies, and motion control instructions need to be sent, or the non-controlled objects are energy output links, and energy control instructions need to be sent.

[0392] 732, the remote physician console determines the non-controlled objects from the patient surgery platform based on the permission attribution of the unowned permissions, and determines the target control instructions associated with the non-controlled objects from the preset control instruction set.

[0393] The design of the preset control instruction set of the non-controlled objects can refer to the design of the preset control instruction set of the controlled objects, which will not be repeated here.

[0394] When the kinematics models of different medical instruments of the slave robot are basically the same, in 732, the step of determining the non-controlled objects from the patient surgery platform and determining the target control instructions associated with the non-controlled objects from the preset control instruction set includes: determining the non-controlled manipulator assemblies from the slave robot, and determining the target motion control instructions associated with the non-controlled manipulator assemblies from the preset control instruction set based on the non-controlled manipulator assemblies.

[0395] When the kinematics models of different medical instruments of the slave robot are basically different, in 732, the step of determining the non-controlled objects from the patient surgery platform and determining the target control instructions associated with the non-controlled objects from the preset control instruction set includes: determining the non-controlled manipulator assemblies from the slave robot, identifying the medical instruments in the non-controlled manipulator assemblies, and determining the target motion control instructions associated with the non-controlled manipulator assemblies from the preset control instruction set based on the non-controlled manipulator assemblies and the medical instruments assembled therewith.

[0396] In 732, the step of determining the non-controlled objects from the patient surgery platform and determining the target control instructions associated with the non-controlled objects from the preset control instruction set includes: determining the non-controlled energy output links from the energy platform, and determining the target energy control instructions associated with the non-controlled energy output links from the preset control instruction set.

[0397] 733, the remote physician console sends the target control instructions to the patient surgery platform.

[0398] 734, the patient surgery platform receives the target control instructions, and the non-controlled objects of the patient surgery platform execute the target control instructions.

[0399] In the case that the non-possessory permission configuration is successful, the non-controlled object usually does not receive the target control instruction, or even if it can receive the target control instruction, it does not execute the target control instruction. The reasons that the non-controlled object does not receive the target control instruction include that a network channel between the remote physician console and the non-controlled object is not created according to the configuration of the control permission, or even if the network channel between the remote physician console and the non-controlled object is created, the target control instruction is not transmitted between the two according to the configuration of the control permission. Accordingly, in 734, the patient surgery platform can not receive the target control instruction, or the non-controlled object of the patient surgery platform can not execute the target control instruction. That is, in any case, the state of the non-controlled object does not change when the non-possessory permission configuration is successful, including that the motion state does not change or the energy output state does not change. When the non-possessory permission configuration fails, the state of the non-controlled object changes, including that the motion state changes or the energy output state changes. In the abnormal state, the reasons that cause the non-possessory permission configuration to fail can include that the transmission of the target control instruction in the network channel is disturbed, or the network channel between the remote physician console and the patient surgery platform is incorrectly created.

[0400] Whether the state of the non-controlled object changes can be determined by the observation of the physician on the side of the non-controlled object. The physician can inform the remote physician console or the patient surgery platform of the observation result, so as to artificially determine whether the non-possessory permission configuration is successful.

[0401] In some embodiments, referring to Figure 17 The method can further include:

[0402] 735, after the non-controlled object executes the target control instruction, the patient surgery platform returns an actual execution result to the remote physician console.

[0403] Whether the non-controlled object executes the target control instruction or not, the patient surgery platform can return an actual execution result to the remote physician console. The actual execution result includes whether the state of the non-controlled object changes.

[0404] 736, the remote physician console receives the actual execution result and detects whether the actual execution result matches an expected execution result.

[0405] When the non-possessory permission, the expected execution result associated with the target control instruction usually includes that the state of the non-controlled object does not change. The state of the non-controlled object not changing includes that the motion state does not change or the energy output state does not change. The motion state not changing means that the pose of the non-controlled manipulator assembly does not change; the energy output state not changing means that the switch state of the non-controlled energy platform does not change, the current output value does not change, the voltage output value does not change, the power output value does not change, or the frequency output value does not change.

[0406] 737, when detecting that the actual execution result matches the expected execution result, determining that the un-owned permission configuration is successful.

[0407] For example, when the execution result corresponds to no change in the state of the uncontrolled object, and the expected execution result corresponds to no change in the state of the uncontrolled object, it can be determined that the un-owned permission configuration is successful.

[0408] 738, when detecting that the actual execution result does not match the expected execution result, determining that the un-owned permission configuration fails.

[0409] For example, when the execution result corresponds to a change in the state of the uncontrolled object, and the expected execution result corresponds to no change in the state of the uncontrolled object, it can be determined that the un-owned permission configuration fails.

[0410] Through 735-738, it can be determined whether the un-owned permission in the operator's permission is configured successfully.

[0411] In some embodiments, when one or more remote physician consoles are paired with the patient surgery platform in real time, for any remote physician console, when both the owned permission and the un-owned permission of the remote physician console are configured successfully, it can be confirmed that the remote physician console is actually paired successfully with the patient surgery platform; when the owned permission or the un-owned permission of the remote physician console is configured unsuccessfully, it can be confirmed that the remote physician console is actually paired unsuccessfully with the patient surgery platform. Such pairing success detection can fully guarantee the smooth implementation of the super-remote surgery.

[0412] <Switching of pairing>

[0413] In some embodiments, a device (local device) can be paired with multiple remote devices. For example, one remote medical console can be paired with multiple patient surgery platforms. For another example, one patient surgery platform can be paired with multiple remote medical consoles. The medical doctor can configure the one device to establish pairing with multiple remote devices simultaneously, and the established pairings between the one device and the multiple remote devices can be pre-arranged pairings. The one device can obtain pairing information of the multiple remote devices from a server, and the pairing information can include pairing orders of the multiple remote devices and device network information. The pairing orders can also be obtained by the one device from itself since the one device can record the pairing orders of the one device and the multiple remote devices. The pairing orders can be default, for example, the pairing order of the first configured pairing is earlier than the pairing order of the second configured pairing. The pairing orders can be preset, for example, the first remote device can be preset to have an earlier pairing order and the second remote device can be preset to have a later pairing order. The pairing orders can be determined by pre-arranged pairing time, for example, the one device having a relatively earlier pre-arranged pairing time can have an earlier pairing order. The pairing orders can be determined according to the emergency degree of surgery, for example, the medical doctor can input medical information of the patient surgery platform and the remote medical console in the same time range, for example, 0-30 minutes, and the medical information can include the emergency degree of surgery. The patient surgery platform and the remote medical console having a higher emergency degree of surgery can be configured to have an earlier pairing order by default. When the time range is exceeded, the pairing orders can be set according to the default pairing mode.

[0414] In some surgical scenarios, to ensure the safety and privacy of surgery, the one device can be paired with at most one remote device in real time and pre-arranged pairing with other remote devices at the same time. For example, the one remote medical console can be paired with at most one patient surgery platform in real time and pre-arranged pairing with other patient surgery platforms at the same time. For another example, the one patient surgery platform can be paired with at most one remote medical console in real time and pre-arranged pairing with other remote medical consoles at the same time.

[0415] In these surgical scenarios, the utility model provides a method for quickly switching pairing, referring to Figure 38 The method comprises the following steps.

[0416] 801, the device obtains pairing information of multiple remote devices which have established pre-arranged pairings with the device.

[0417] The pairing information includes pairing orders of the multiple remote devices and device network information. The multiple remote devices include a first remote device and a second remote device.

[0418] 802, when the device determines that the first remote device has a priority pairing order among the multiple remote devices based on the pairing orders in the pairing information, the device establishes real-time pairing with the first remote device based on the device network information of the first remote device in the pairing information.

[0419] 803, when detecting that the switching pairing condition is satisfied, the device cancels the real-time pairing with the first counterpart device, and establishes the real-time pairing with the second counterpart device based on the device network information of the second counterpart device in the pairing information.

[0420] When the device cancels the real-time pairing with the first counterpart device, the device can update the pairing information, for example, can mark that the first counterpart device has been paired, or can delete the device network information of the first counterpart device to avoid real-time pairing with the first counterpart device again. The device can send the updated pairing information to the server for the server to maintain the pairing table, such as marking the pairing state of the related device.

[0421] In this embodiment, the switching pairing condition is satisfied to allow the device to establish a new real-time pairing with the second counterpart device, which can prevent accidental interruption of the ongoing super-remote surgery of the first counterpart device and ensure the continuous implementation of the super-remote surgery.

[0422] In some embodiments, the switching pairing condition is satisfied at least in two aspects: the device and the first counterpart device satisfy the condition of cancelling the real-time pairing; and the device and the second counterpart device satisfy the condition of establishing the real-time pairing.

[0423] In some embodiments, the device and the first counterpart device satisfying the condition of cancelling the real-time pairing includes at least one of the following: completion of the surgery, abnormal interruption of the surgery, and agreement of the two devices to switch the pairing.

[0424] Regarding the completion of the surgery, at least one of (1)-(3) is included: (1) the necessary physical accessories for the surgery are detached from the slave robot or from the patient's body and the delay time is reached, or the slave robot is recovered to the preoperative preparation state and the delay time is reached; (2) the image platform is closed and the delay time is reached, or the field of view provided by the imaging device such as the endoscope in the slave robot is switched from the surgical field of view to the non-surgical field of view and the delay time is reached; (3) the doctor or the patient leaves the surgical environment and the delay time is reached.

[0425] Regarding (1), for each type of master-slave surgical robotic system, for example, for a laparoscopic surgical robotic system or an interventional surgical robotic system, the acquiring the surgical necessary physical accessories from the slave robot includes at least one of: acquiring all medical instruments from the slave robot (of manipulator), acquiring a sterile drape from the slave robot, acquiring a puncture device that guides the medical instrument to insert into the patient's body from the slave robot (of manipulator). The acquiring the surgical necessary physical accessories from the patient's body includes at least one of: acquiring all medical instruments from the patient's body, acquiring the puncture device from the patient's body, and acquiring an auxiliary device such as a monitor or a breathing machine from the patient's body. The acquiring the slave robot to recover to the preoperative preparation state includes acquiring the mechanical arm of the slave robot to recover to the retracted state, or resetting the support feet of the slave robot chassis.

[0426] Regarding (2), in the surgery, the provision of the image is crucial, and the image platform closing to the delay time often means the completion of the surgery. Among them, the acquiring the field of view from the surgical field of view to the non-surgical field of view to the delay time includes: acquiring the field of view from the surgical field of view to the non-surgical field of view includes acquiring the endoscope from the patient's body, or the image collected by the endoscope does not include human tissue or organs.

[0427] Regarding (3), the acquiring the doctor or the patient to evacuate the surgical environment to the delay time includes: acquiring the doctor to evacuate the surgical environment includes acquiring the doctor to leave the remote doctor console. Acquiring the patient to evacuate the surgical environment includes acquiring the patient to evacuate from the operating bed or the operating bed carrying the patient to be removed from the operating room.

[0428] Regarding the surgical abnormal interruption, it includes detecting the network state abnormality of the network channel, for example, including detecting the network connectivity abnormality or the network communication quality abnormality. Further, the confirmation of the network state abnormality can be superimposed with a judgment condition, which includes, for example, the cumulative time of the network state abnormality reaching a time threshold, or the cumulative number of the network state abnormality reaching a number threshold.

[0429] Regarding the two-end device agreeing to switch the pairing, it includes that either end initiates a switching pairing request to the opposite end, the opposite end accepts the switching pairing request, and the opposite end device can notify the end that initiates the switching pairing request.

[0430] In some embodiments, the device and the second opposite end device satisfy the real-time pairing establishment condition includes: the second opposite end device and the device have established a reservation pairing. The establishment of the reservation pairing between the two can accelerate the speed of the switching pairing.

[0431] In some embodiments, when the device pairs with any peer device in real time, the device further establishes a network channel with the peer device. The establishing of the network channel includes establishing all network channels and establishing only target network channels based on the configured control authority. When the device un-pairs with any peer device in real time, the device further destroys all network channels or target network channels established with the peer device.

[0432] In some embodiments, when the device pairs with any peer device in real time, the device further acquires the configured control authority with the peer device. At least one of the device and the peer device can create a target network channel based on the acquired configured control authority. When the device un-pairs with any peer device in real time, the device further releases the configured control authority with the peer device.

[0433] In some embodiments, in 802, the device establishes a real-time pairing with a first peer device, further including establishing a first master-slave mapping relationship between the device and the first peer device. In 803, the device un-pairs with the first peer device, further including the device disconnecting the first master-slave mapping relationship between the device and the peer device. When the first master-slave mapping relationship is disconnected, the master-slave control is interrupted, including master-slave motion control or master-slave energy control. Wherein, for safety considerations, when the master-slave motion control is interrupted, the manipulator assembly can be configured to maintain at least one of the current position and the current posture, and when the master-slave energy control is interrupted, the energy platform can be configured to reduce or turn off the energy output. In 803, the device establishes a real-time pairing with a second peer device, further including establishing a second master-slave mapping relationship between the device and the second peer device. Exemplarily, one of the device and the peer device includes a remote physician console, and the other includes a slave robot or an energy platform. When the device and the peer device are performing motion control or energy control, they are usually controlled according to the master-slave mapping relationship. Wherein, the motion control occurs between the remote physician console and the slave robot, and the remote physician console can control the slave robot to move, for example, in master-slave control; the slave robot can also control the remote physician console to move, for example, in master-slave alignment of a laparoscopic surgical robot system. The energy control occurs between the remote physician console and the energy platform, and the remote physician console controls the on-off, current adjustment, voltage condition, power adjustment, or frequency adjustment of the energy platform, etc.

[0434] When the device is paired with the first peer device in real time, the device and the first peer device establish a first master-slave mapping relationship, and control is performed through the first master-slave mapping relationship. When the device cancels the real-time pairing with the first peer device and establishes real-time pairing with a second peer device, because the device types of the first peer device and the second peer device can be different, different device types usually correspond to different mechanical arm configurations or energy output modes, etc., and control between the two through the first master-slave mapping relationship can be inappropriate, and it is necessary to reestablish a master-slave mapping relationship adapted to the device and the second peer device. Assuming that the reestablished master-slave mapping relationship is a second master-slave mapping relationship, in order to ensure the smooth implementation of the remote surgery, it is necessary to switch the first master-slave mapping relationship to the second master-slave mapping relationship when the device switches from real-time pairing with the first peer device to real-time pairing with the second peer device.

[0435] In some embodiments, the master-slave mapping relationship between the device and the peer device includes at least one of first master-slave mapping logic and second master-slave mapping logic. The first master-slave mapping logic includes master-slave mapping logic in which the device controls the peer device. The second master-slave mapping logic includes master-slave mapping logic in which the peer device controls the device.

[0436] In some embodiments, the master-slave mapping logic comprises master-slave motion mapping logic. Take an example of a device being a remote physician console and a peer device being a slave surgical robot. Different slave surgical robot types usually have different manipulator assembly structures, and their controlled structures such as manipulator assemblies for performing surgery are usually different. Different manipulator assemblies can involve different link parameters. In master-slave control, different kinematic models are determined according to different manipulator assemblies, and then the manipulator assemblies are controlled based on the corresponding kinematic models. Different slave surgical robot types can need to control the manipulator assemblies according to different motion scaling ratios when executing the same pose instruction. Different slave surgical robot types usually include different numbers or different arrangements of manipulator assemblies. In master-slave control, an operating part usually controls one manipulator assembly. When it is necessary to switch the manipulator assembly controlled by the operating part, the manipulator assembly can need to be switched according to specific instrument switching logic. Different slave surgical robot types can correspond to different instrument switching logic. In master-slave control, different slave surgical robot types can have different instrument operation logic for controlling different medical instruments to move. For example, in a single-hole laparoscopic slave surgical robot, controlling the movement of two operating parts simultaneously generates a control instruction for controlling the movement of an endoscope. In a multi-hole laparoscopic slave surgical robot, controlling the movement of one operating part generates a control instruction for controlling the movement of an endoscope. Thus, the master-slave motion mapping logic can include at least one of the kinematic models, motion scaling ratios, instrument switching logic, and instrument operation logic involved in master-slave control. The kinematic models involved in master-slave control at least include the kinematic models of the controlled structures such as manipulator assemblies in the slave surgical robot. They can also include the kinematic models of the mechanical arms in the operating parts of the remote physician console.

[0437] In some embodiments, the master-slave mapping logic comprises master-slave energy mapping logic. Take an example of a device being a remote physician console and a peer device being an energy platform. Different energy platform types can have different types of energy output forms. The remote physician console sends energy control instructions to the energy platform, and the energy platform outputs energy based on the energy control instructions, which includes specific conversion relationships. For example, some conversion relationships include current size conversion, some conversion relationships include power size conversion, and some conversion relationships include frequency size conversion. Thus, the master-slave energy mapping logic can include energy output conversion relationships in master-slave control.

[0438] In some embodiments, for example in a laparoscopic surgical robot system, the remote physician console includes a first type of remote physician console and a second type of remote physician console. The first type of remote physician console has an operating part in the form of a mechanical link, and the second type of remote physician console has an operating part in the form of magnetic navigation. For example, when the device refers to a remote physician console and the peer device refers to a slave surgical robot:

[0439] If the remote physician console is a first type remote physician console, the remote physician console can control the slave robot in master-slave (follow) mode, and the slave robot can also control the remote physician console in master-slave alignment. The former requires first master-slave mapping logic, and the latter requires second master-slave mapping logic, i.e., the master-slave mapping relationship includes the first master-slave mapping logic and the second master-slave mapping logic.

[0440] If the remote physician console is a second type remote physician console, the remote physician console can control the slave robot in master-slave (follow) mode, but the slave robot cannot control the remote physician console in master-slave alignment, and only the former requires first master-slave mapping logic, i.e., the master-slave mapping relationship can only include the first master-slave mapping logic.

[0441] In some embodiments, the device information of the remote physician console and the patient surgery platform can include the information included in the master-slave mapping logic described above. When registering with the server, these information can be stored in the server, and when pairing, these information can be sent to the opposite device. Further, the master-slave mapping relationship is established based on the master-slave mapping logic of at least one of the two ends.

[0442] In some embodiments, in the scenario where the device is a remote physician console and the opposite device is a patient surgery platform, when the device is paired with any opposite device in real time, the device further includes updating the UI interface in the display screen of the device. Specifically, when the device is paired with a first opposite device in real time, the display screen of the device displays a first UI interface; when the device switches from real-time pairing with the first opposite device to real-time pairing with a second opposite device, the display screen of the device displays a second UI interface. The first UI interface is associated with the information of the first opposite device, and the second UI interface is associated with the information of the second opposite device. Different opposite devices can have at least one of different device quantities, types, and states, resulting in different information of different opposite devices, which need to be accurately displayed accordingly. For example, the first opposite device includes a single-hole endoscopic slave robot, and the second opposite device includes a multi-hole endoscopic slave robot. At least the change in the model and state of the slave robot can cause the UI interfaces of the two to be different.

[0443] In some embodiments, in the case where the remote physician console has an operating part of a mechanical arm, when the device is paired with any opposite device in real time, in order to facilitate and quickly complete preoperative preparation, one or more remote physician consoles that have obtained operator permissions can be automatically aligned in master-slave mode. Referring to Figure 39 The method of master-slave alignment includes:

[0444] 8031, the patient surgery platform sends the current pose of the medical instrument at the end of a manipulator assembly in the slave robot in the endoscope coordinate system to the remote physician console that has operator permissions for the manipulator assembly.

[0445] The operator authority mainly refers to a motion control authority. The transmission of the current posture can be transmitted based on a motion control network channel created by the remote physician console and the patient surgery platform according to the operator authority.

[0446] 8032, the remote physician console acquires the current posture of the medical instrument tip in the endoscope coordinate system sent by the patient surgery platform.

[0447] 8033, based on the current posture of the medical instrument tip in the endoscope coordinate system, the remote physician console converts the current posture of the medical instrument tip in the endoscope coordinate system into a target posture of the operating part in the display screen coordinate system of the remote physician console.

[0448] 8034, the remote physician console analyzes the target posture based on inverse kinematics to obtain target joint parameters of each joint of the mechanical arm in the operating part.

[0449] 8035, the remote physician console controls the motion of the corresponding joint in the mechanical arm based on the target joint parameters of each joint, so that the tip of the operating part reaches the target posture in the display screen coordinate system, so that the posture of the operating part is aligned with the posture of the medical instrument tip, and the master-slave alignment is realized.

[0450] Through the above 8031-8035, the operating part and the medical instrument can be aligned in posture, which is beneficial to quickly complete the preoperative preparation, and is beneficial to the doctor to realize the hand-eye intuition when controlling the associated medical instrument through the operating part.

[0451] <Exchange of control authority>

[0452] In some embodiments, in some surgery scenarios such as teaching scenarios, multiple remote physician consoles can be paired with one patient surgery platform in real time at the same time, wherein different remote physician consoles are configured to have different control authorities. In the teaching scenario, it is assumed that a first remote physician console and a second remote physician console are paired with the same patient surgery platform in real time, the first remote physician console is configured to have one of the operator authority and the observer authority, and the second remote physician console is configured to have the other one of the operator authority and the observer authority. The remote physician console with the operator authority focuses on “teaching”, and the remote physician console with the observer authority focuses on “learning”; or, the remote physician console with the operator authority focuses on “practice”, and the remote physician console with the observer authority focuses on “guidance”. In order to quickly switch between teaching and learning, or between practice and guidance, the utility model further provides a method for quickly switching control authorities, which is described below with reference to Figure 40 The method comprises the following steps:

[0453] 901, the first remote physician console sends a control authority exchange request to the patient surgery platform.

[0454] The control authority exchange request is used to request to exchange control authority with a target second remote physician console in the at least one second remote physician console. The first remote physician console has a first control authority, and the at least one second remote physician console has a second control authority. The plurality of second remote physician consoles can have different second control authorities.

[0455] 902, the patient surgery platform receives the control authority exchange request sent by the first remote physician console and forwards it to the target second remote physician console.

[0456] Since the first remote physician console and the second remote physician console do not communicate directly, the patient surgery platform can act as a server to relay relevant information.

[0457] 903, the target second remote physician console receives the control authority exchange request and obtains a response instruction generated by the control authority exchange request.

[0458] The second remote physician console can generate and play a notification based on the received control authority exchange request to facilitate the physician to respond to the control authority exchange request based on the notification.

[0459] The content of the notification can be whether the second remote physician console accepts the control authority exchange request initiated by the first remote physician console. The notification can be played in the form of a UI interface or in the form of a sound. The physician can respond to the control authority exchange request through the operable control of the UI interface or through voice recognition. The response includes acceptance or rejection.

[0460] 904, the target second remote physician console sends the response instruction to the patient surgery platform.

[0461] 905, the patient surgery platform receives the response instruction, and when the response instruction is an affirmative response instruction, configures the first control authority of the first remote physician console as the second control authority of the target second physician console, and configures the second control authority of the target second remote physician console as the second control authority of the first physician console.

[0462] The response instruction includes an affirmative response instruction or a negative response instruction. The affirmative response instruction indicates that the target second remote physician console obtains the response instruction generated by accepting the control authority exchange request. The negative response instruction indicates that the target second remote physician console obtains the response instruction generated by rejecting the control authority exchange request.

[0463] Further, in response to the instruction being a negative response instruction, the patient surgery platform does not re-allocate the control authority of the first remote physician console and the target second remote physician console, both of which maintain the original control authority.

[0464] In S905, the patient surgery platform generally needs to receive a positive response instruction within a preset time range to reconfigure the control authority of the first remote physician console and the target second remote physician console. The preset time range generally refers to the timing of the patient surgery platform from forwarding the exchange control authority request to the target second remote physician console, for example, the preset time range is within 3 minutes.

[0465] 906, the patient surgery platform sends the second control authority to the first remote physician console and sends the first control authority to the target second remote physician console.

[0466] 907, the first remote physician console acquires the second control authority, and the target second remote physician console acquires the first control authority.

[0467] In some embodiments, the first remote physician console includes a first display screen, and the second remote physician console includes a second display screen. The method can include:

[0468] The first remote physician console can generate a control on the display interface of the first display screen in response to acquiring a phase change of the surgical process, and the control is used to trigger sending an exchange control authority request to the patient surgery platform. Wherein, the phase change of the surgical process can involve the demand for control authority exchange.

[0469] Further, in S901, the first remote physician console can send an exchange control authority request to the patient surgery platform in response to acquiring that the control is triggered. Of course, when the first remote physician console does not acquire that the control is triggered, it will not send an exchange control authority request to the patient surgery platform, and still operate under the original control authority.

[0470] In some embodiments, the phase change of the surgical process includes at least one of the following: the type of the target surgical object in the surgical process changes; and the type of the target surgical action in the surgical process changes.

[0471] Exemplarily, the target surgery object types include important target surgery objects and non-important target surgery objects. For example, the important target surgery objects include at least one of an artery, a heart, a liver, a spleen, a kidney, a stomach, a lung, and a gallbladder. For example, the non-important target surgery objects include at least one of a vein, fat, a large intestine, and a small intestine. Generally, different types of target surgery objects can be associated with different experienced doctors to perform surgery. The type of the target surgery object can be determined by recognizing the surgery image transmitted by the first remote doctor console to the patient surgery platform.

[0472] Exemplarily, the target surgery action types include important target surgery actions and non-important target surgery actions. For example, the important target surgery actions include at least one of resection and hemostasis. For example, the non-important target surgery actions include at least one of suturing and drainage. Different types of target surgery actions can be associated with different experienced doctors to perform surgery. Generally, different types of medical instruments are usually used to perform different surgery actions. The type of the target surgery action can be determined by recognizing the type of the currently used medical instrument transmitted by the first remote doctor console to the patient surgery platform.

[0473] Generally, the important target surgery objects or the important target surgery actions require more experienced doctors to perform surgery, and the non-important target surgery objects or the non-important target surgery actions can be performed by relatively less experienced doctors. Therefore, the control authority exchange prompt can be given based on the stage change of the surgery process, so as to facilitate the doctor to determine whether to exchange the control authority according to the actual needs.

[0474] In other embodiments, the control can also be generated in the first display screen without being based on the stage change of the surgery process, and the control is used for the doctor to trigger the sending of the exchange control authority request to the patient surgery platform according to the actual needs.

[0475] In some embodiments, to ensure that the exchange control authority request sent by the first remote doctor console is reliable, the request can be encapsulated by using a custom protocol agreed between the first remote doctor console and the patient surgery platform, and the request is unpacked on the patient surgery platform side to realize authentication, which is not repeated here.

[0476] In some embodiments, the first remote doctor console or the target second remote doctor console can generate a prompt based on the control authority exchange situation. When the control authority exchange is successful, the successful exchange of the authority is prompted on both ends. When the control authority is maintained, it is prompted on both ends that the authority is not exchanged.

[0477] In some embodiments, the two remote physician consoles capable of exchanging control authority are usually two remote physician consoles of the same device type, or two remote physician consoles with the same device function. In some embodiments, the method can include enabling or disabling the function of any remote physician console to initiate an exchange of control authority request based on identifying the device type or device function of the two remote physician consoles that are real-time paired with the patient surgery platform. For example, when the device type and / or device function of the two remote physician consoles do not match, the function of any remote physician console to initiate an exchange of control authority request is disabled; when the device type and / or device function of the two remote physician consoles match, the function of any remote physician console to initiate an exchange of control authority request is enabled. Different configuration interfaces can be provided for the two remote physician consoles according to such results, when enabled, a configuration interface including a control that can trigger an exchange of control authority request is provided; when disabled, a configuration interface not including a control that can trigger an exchange of control authority request is provided.

[0478] When the control authority is released, the remote physician console with operator authority disconnects the master-slave control, including disconnecting the master-slave motion control or the master-slave energy control. When the master-slave motion control is disconnected, the manipulator assembly can be configured to maintain at least one of the current position and attitude. When the master-slave energy control is disconnected, the energy platform can be configured to reduce or shut down energy output to ensure safety.

[0479] In this embodiment, the first remote physician console and the second remote physician console can maintain pairing with the patient surgery platform without being unpaired, and can seamlessly exchange control authority throughout the entire surgery process, improving the rapid switching of teaching and learning, or practice and guidance modes, and improving the efficiency of the surgery.

[0480] In some embodiments, a first remote physician console can also establish real-time pairing with multiple patient surgery platforms to watch or guide the surgery process of different patient surgery platforms. However, at the same time, the first remote physician console can at most obtain operator authority for one of the patient surgery platforms, and the first remote physician console can simultaneously watch multiple videos sent by different patient surgery platforms on its display screen. Meanwhile, the patient surgery platforms can also establish real-time pairing with multiple second remote physician consoles, and for the same reason, a second remote physician console can at most obtain operator authority for one of the patient surgery platforms. When the first remote physician console and the second remote physician console have different control authorities for the same patient surgery platform, such as one having operator authority and one having observer authority, the two can switch control authority based on the foregoing embodiments, which will not be repeated here.

[0481] In the above embodiments, at least one of the following can be performed based on the change of the pairing relationship or the change of the control authority: establishing a network channel between the two ends of the pairing, establishing a master-slave mapping relationship, performing pairing success detection, and updating a UI interface.

[0482] The various devices described above, including the first device, the second device, the server, or the portable mobile terminal, can include a memory and at least one processor that, when at least one program stored in the memory is processed by the at least one processor, causes the at least one processor to implement the method described in any one of the above embodiments.

[0483] The utility model further provides a computer readable storage medium, computer readable storage medium has instruction, when the instruction runs on at least one processor, can realize the step in each method embodiment described above. The memory 503 can include a high-speed RAM memory, and can also include a non-volatile memory, for example, at least one disk memory. The processor can be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the utility model, or a graphics processor GPU (Graphics Processing Unit). The one or more processors included in the control device can be the same type of processor, such as one or more CPUs, or alternatively, one or more GPUs. They can also be different types of processors, such as one or more CPUs and one or more GPUs.

[0484] The utility model further provides a computer program product, when the computer program product runs on the device, makes the device execute and realizes the step in each method embodiment described above.

[0485] The technical features of the above-described embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described, but as long as the combinations of the technical features do not contradict, they should be considered within the scope of the present disclosure.

[0486] The above-described embodiments only express several implementation manners of the utility model, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the invention patent. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the utility model, a number of modifications and improvements can be made, which are within the protection scope of the utility model. Therefore, the protection scope of the utility model patent should be subject to the appended claims.

Claims

1. A remote surgical robot system, characterized in that, The system includes a server, and a first device and a second device capable of communicating with the server. The first device includes one of a remote doctor console and a patient surgical platform operable by the remote doctor console. The second device includes the other of the remote doctor console and the patient surgical platform. The first device includes: The first receiving unit is configured to receive second device information that can be paired with a second device, which is matched and sent by the server based on a first request. The first request is configured to request the server to send the second device information that can be paired with the second device. The second device that can be paired with the second device includes a second device that is registered to the server and has agreed on a matching custom protocol with the first device. The display unit is used to display a configuration interface with a first control, and is also used to display the second device information of the pairable second device received by the first receiving unit in the configuration interface. The first control is used to trigger the first device to send the first request to the server. The determining unit is configured to determine the target second device information of the target second device to be paired based on the second device information displayed in the configuration interface of the display unit, and send a pairing request to the server. The pairing request includes the first device information of the first device and the target second device information, and the pairing request is used to request to establish a pairing with the target second device. The second receiving unit is configured to receive the device network information of the target second device sent by the server when the server receives an affirmative response instruction generated by the target second device in response to the pairing request forwarded by the server; and The pairing unit is used to establish a pairing with the target second device based on the device network information received by the second receiving unit, so as to realize direct communication with the target second device.

2. The remote surgical robot system according to claim 1, characterized in that, The second device information of the pairable second device is displayed on the configuration interface via a list control, and the second device information of one pairable second device is associated with a list option of the list control; the determining unit is used for: In response to detecting that a target list option is selected in the list options, the second device information of the second device associated with the target list option is determined to be the target second device information of the target second device to be paired.

3. The remote surgical robot system according to claim 1, characterized in that, The second device information of the pairable second device is displayed on the configuration interface via a first drag control. The second device information of one pairable second device is associated with one of the first drag controls. The configuration interface also includes a window control. The determining unit is used for: In response to detecting that the target first drag control in the first drag control has moved into the window of the window control, the second device information of the second device associated with the target first drag control is determined to be the target second device information of the target second device to be paired.

4. The remote surgical robot system according to claim 1, characterized in that, The establishment unit is also used for: Establish a master-slave mapping relationship between the first device and the target second device. The master-slave mapping relationship includes at least one of the kinematic model, motion scaling ratio, device switching logic, and device operation logic involved in the master-slave control of the first device and the target second device.

Images