Surgical robot and audio system interconnection method thereof

Abstract

The embodiment of the invention provides a surgical robot and an audio system interconnection method thereof, the surgical robot comprises a server, and a first device and a second device capable of communicating with the server, and the first device and the second device comprise one of a main console and a patient side surgical platform capable of being controlled by the main console. The first device is configured with a first audio system, and the second device is configured with a second audio system. According to the surgical robot and the audio system interconnection method thereof, when the first audio system only needs to receive / pay attention to the audio signal of the specific audio system, the specific audio system is selected as the target audio system and performs private talkback, so that the target audio signal from the target audio system is prevented from being interfered by other audio signals; clear audio signals of a target audio system are transmitted and received, and potential safety hazards of an operation caused by influence on instruction or information transmission are reduced.

Description

Technical Field

[0001] The present application relates to the field of medical devices, and in particular to a surgical robot and a method for interconnecting its audio system. Background Art

[0002] With the innovation and development of science and technology and medical technology, surgical robotics has gradually matured. For doctors, surgical robots offer advantages such as ease of operation and high precision. For patients, surgeries performed with surgical robots have the advantages of minimal trauma, minimal pain, and rapid recovery, and are widely accepted by both doctors and patients.

[0003] Existing surgical robots typically consist of a main control console and a patient operating platform. The main control console controls the patient operating platform to perform surgery. With the increasing maturity of high-speed communication network technologies with low latency and high bandwidth, such as 5G and the internet, remote surgical robotics, which combines the advantages of both surgical robotics and high-speed communication networks, has emerged. Remote surgical robotics allows doctors to perform ultra-remote surgery on patients in remote locations, for example, in different countries or provinces, thereby maximizing the benefits of spatial constraints on surgical implementation and effectively alleviating the uneven distribution of medical resources.

[0004] The main control console and patient surgical platform in a telesurgery robotic system are typically deployed in different countries, provinces, or different jurisdictions within the same city. Before performing ultra-telesurgical surgery, communication between the remote main control console and the patient surgical platform needs to be established. At the same time, during telesurgery, effective communication can improve surgical quality and efficiency. For example, the doctor operating the main control console and the medical staff surrounding the patient may communicate to convey specific instructions or information. However, when the audio systems of multiple devices are interconnected, a single device may simultaneously receive audio signals from multiple connected devices. When the device only needs to focus on the audio signals emitted by a specific device, audio signals from other devices may interfere with the audio signals from that specific device, affecting the transmission of instructions or information between specific devices during telesurgery and potentially posing a safety hazard to the surgery. Summary of the Invention

[0005] Based on this, it is necessary to provide a method for interconnecting a surgical robot and its audio system that can transmit clear audio signals between specific devices.

[0006] In a first aspect, the present application provides a surgical robot, comprising a server and a first device and a second device capable of communicating with the server, wherein the first device and the second device comprise one of a main console and a patient-side surgical platform operable by the main console, the first device being configured with a first audio system, and the second device being configured with a second audio system; Wherein, the first audio system is configured as follows: acquiring, in real time, second site information of a target second audio system that can be privately paired with the first audio system; In response to receiving target site information of a target audio system determined based on the second site information, sending a private pairing request to the target audio system, the private pairing request including the first site information of the first audio system, the private pairing request being used to request establishment of a private pairing with the target audio system; In response to receiving an affirmative response instruction generated by the target audio system affirmatively responding to the private pairing request, establishing a private pairing with the target audio system based on the target site information; The second audio system is configured as follows: Receiving the private pairing request; Confirm that the response instruction generated to the private pairing request is an affirmative response instruction, establish a private pairing with the first audio system based on the first site information, and send the affirmative response instruction to the first audio system.

[0007] Furthermore, establishing a private pairing with the target audio system based on the target site information includes: Based on the target site information, disconnecting from other audio systems except the target audio system; confirming that a connection with the target audio system is not established, and establishing a connection with the target audio system based on the target site information; The establishing of a private pairing with the first audio system based on the first site information includes: Based on the first site information, disconnecting from other audio systems except the first audio system; It is confirmed that no connection is established with the first audio system, and a connection is established with the first audio system based on the first site information.

[0008] Furthermore, establishing a private pairing with the target audio system based on the target site information includes: confirming that a connection with the target audio system is not established, and establishing a connection with the target audio system based on the target site information; generating a first private intercom instruction based on the target site information, and sending the first private intercom instruction to the audio system with which the connection is established, so that only the target audio system among the connected audio systems responds to the first private intercom instruction and plays the first audio signal acquired by the first audio system; According to the first private intercom instruction, suppressing the playing of audio signals other than the target audio signal acquired by the target audio system; The establishing of a private pairing with the first audio system based on the first site information includes: confirming that a connection with the first audio system is not established, and establishing a connection with the first audio system based on the first site information; generating a fourth private intercom instruction based on the first site information, and sending the fourth private intercom instruction to the audio systems with which the connection is established, so that only the first audio system among the audio systems with which the connection is established responds to the fourth private intercom instruction and plays the target audio signal; According to the fourth private intercom instruction, playing of audio signals other than the first audio signal is suppressed.

[0009] Furthermore, establishing a private pairing with the target audio system based on the target site information includes: confirming that a connection with the target audio system is not established, and establishing a connection with the target audio system based on the target site information; generating a second private intercom instruction based on the target site information; sending the acquired first audio signal to the target audio system according to the second private intercom instruction, and suppressing the playing of audio signals other than the target audio signal acquired by the target audio system; The establishing of a private pairing with the first audio system based on the first site information includes: confirming that a connection with the first audio system is not established, and establishing a connection with the first audio system based on the first site information; generating a fifth private intercom instruction based on the first site information; According to the fifth private intercom instruction, the target audio signal is sent to the first audio system, and the playing of audio signals other than the first audio signal is suppressed.

[0010] Furthermore, establishing a private pairing with the target audio system based on the target site information includes: confirming that a connection with the target audio system is not established, and establishing a connection with the target audio system based on the target site information; generating a third private intercom instruction based on the target site information, and sending the third private intercom instruction to the audio systems with which the connection has been established, so that the audio systems other than the target audio system in the established audio systems suppress the playback of the first audio signal acquired by the first audio system in response to the third private intercom instruction; suppressing, according to the third private intercom instruction, playback of audio signals other than the target audio signal acquired by the target audio system; The establishing of a private pairing with the first audio system based on the first site information includes: confirming that a connection with the first audio system is not established, and establishing a connection with the first audio system based on the first site information; generating a sixth private intercom instruction based on the first site information, and sending the sixth private intercom instruction to the audio systems with which the connection is established, so that the audio systems other than the first audio system in the connected audio systems suppress the playback of the target audio signal in response to the sixth private intercom instruction; According to the sixth private intercom instruction, playing of audio signals other than the first audio signal is suppressed.

[0011] Furthermore, the target audio system is configured with a whitelist, the whitelist is associated with at least one first audio system that is allowed to pair with the target audio system, and the affirmative response instruction is automatically generated when the target audio system meets a preset condition.

[0012] Furthermore, the surgical robot further includes a plurality of master audio systems and slave audio systems cascaded with the master audio systems, the first audio system and the second audio system are the master audio systems or the slave audio systems, and both the master audio systems and the slave audio systems can be determined as the target audio systems; After establishing private pairing between the first audio system and the target audio system: The first audio system only plays the target audio signal acquired by the target audio system, and the target audio system only plays the first audio signal acquired by the first audio system. The audio system cascaded with the first audio system does not play the target audio signal, and the audio system cascaded with the target audio system does not play the first audio signal.

[0013] Furthermore, the first audio system is further configured to: Perform echo cancellation and noise reduction processing on the acquired first audio signal.

[0014] Furthermore, performing echo cancellation on the acquired first audio signal includes: Adaptive filtering is used to dynamically estimate a transfer function of the echo path, and an echo signal is estimated based on the transfer function, and the echo signal is subtracted from the first audio signal.

[0015] Furthermore, performing echo cancellation on the acquired first audio signal further includes: In response to receiving a target audio signal from the target audio system, the target audio signal is used as a reference signal to filter out a second audio signal from the first audio signal, where the second audio signal is an audio signal obtained by the target audio signal being played outwardly by the first audio system into the audio environment of the first audio system and then re-entering the first audio system.

[0016] Furthermore, the performing noise reduction processing on the acquired first audio signal includes: Active noise cancellation is performed on the first audio signal.

[0017] Furthermore, the surgical robot also includes a live broadcast platform, which can be privately paired with the first audio system or the second audio system.

[0018] In a second aspect, the present application further provides an audio system interconnection method, which is applied to a first audio system in a surgical robot. The surgical robot includes a server and a first device and a second device that can communicate with the server. The first device and the second device include one of a main console and a patient-side surgical platform that can be manipulated by the main console. The first audio system is configured in the first device, and the second device is configured with a second audio system. The method includes: acquiring, in real time, second site information of a target second audio system that can be privately paired with the first audio system; In response to receiving the target site information of the target audio system determined based on the second site information, sending a private pairing request to the target audio system, so that the target audio system generates a response instruction to the private pairing request, and establishes a private pairing with the first audio system based on the first site information when the response instruction is an affirmative response instruction, the private pairing request including the first site information of the first audio system, the private pairing request being used to request establishment of a private pairing with the target audio system, and the affirmative response instruction being generated when the target audio system responds affirmatively to the private pairing request; In response to receiving the affirmative response instruction, a private pairing with the target audio system is established based on the target site information to achieve private intercom with the target audio system.

[0019] In a third aspect, the present application further provides an audio system interconnection method, which is applied to a second audio system in a surgical robot. The remote surgical robot includes a server and a first device and a second device that can communicate with the server. The first device and the second device include one of a main console and a patient-side surgical platform that can be manipulated by the main console. The second audio system is configured in the second device, and the first device is configured with the first audio system. The method includes: receiving a private pairing request initiated by the first audio system, the private pairing request including first site information of the first audio system, the private pairing request being used to request private pairing with a target audio system, the target audio system being the second audio system associated with target site information determined by the first audio system based on second site information of a target second audio system that can be privately paired with the first audio system; Confirm that the response instruction generated to the private pairing request is an affirmative response instruction, establish a private pairing with the first audio system based on the first site information, and send the affirmative response instruction to the first audio system, so that the first audio system establishes a private pairing with the target audio system based on the target site information when receiving the affirmative response instruction, so as to achieve private intercom with the first audio system.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions are executed on at least one processor, the method described in any one of the above embodiments is implemented.

[0021] The surgical robot and the audio system interconnection method thereof of the present application have the following beneficial effects: The target site information of the target audio system is determined based on the second site information of the target second audio system that can be privately paired, and a private pairing request is sent to the target audio system, so that the target audio system generates a response instruction to the private pairing request and establishes a private pairing with the first audio system based on the first site information when the response instruction is an affirmative response instruction. Then, when the affirmative response instruction is received, a private pairing with the target audio system is established based on the target site information, so that both the first audio system and the target audio system establish a private pairing with the other end, thereby realizing private intercom between the first audio system and the target audio system. Therefore, when the first audio system only needs to receive / pay attention to the audio signal of a specific audio system, the embodiment of the present application can select the specific audio system as the target audio system and conduct private intercom, thereby avoiding interference of the target audio signal from the target audio system with other audio signals, realizing the transmission and reception of clear audio signals with the target audio system, and reducing the surgical safety risks caused by the influence of instruction or information transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic structural diagram of a remote surgical robot according to an embodiment of the present application; Figure 2 This is a schematic structural diagram of a surgical instrument according to one embodiment of the present application; Figure 3 This is a schematic structural diagram of a slave robot of a remote surgical robot according to an embodiment of the present application; Figure 4 This is a schematic structural diagram of an interventional surgical robot according to one embodiment of the present application; Figure 5 A network topology diagram of a remote surgical robot according to an embodiment of the present application; Figure 6 This is a module diagram of an audio system according to an embodiment of the present application; Figure 7 This is a flow chart of a method for interconnecting the audio system of a surgical robot according to one embodiment of the present application; Figure 8 A schematic diagram of a custom protocol according to an embodiment of the present application; Figure 9 A schematic diagram of the network topology of an audio system according to an embodiment of the present application; FIG10( a ) is a schematic diagram of the flow of audio signals for intercom in cascade mode according to an embodiment of the present application; FIG10( b ) is a schematic diagram of the flow of audio signals for private intercom in cascade mode according to an embodiment of the present application; Figure 11 A schematic diagram of a live broadcast platform according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] To facilitate understanding of the present application, a more comprehensive description of the present application will be provided below with reference to the accompanying drawings. The accompanying drawings illustrate preferred embodiments of the present application. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the disclosure of the present application.

[0024] It should be noted that when an element is referred to as being "disposed on" another element, it may be directly on the other element or there may also be a centered element. When an element is considered to be "connected" to another element, it may be directly connected to the other element or there may be a centered element at the same time. When an element is considered to be "coupled" to another element, it may be directly coupled to the other element or there may be a centered element at the same time. The terms "vertical", "horizontal", "left", "right", "above", "below" and similar expressions used herein are for illustrative purposes only and are not intended to be the only embodiment. It should be understood that these spatially related terms are intended to cover different orientations of the device in use or in operation in addition to the orientations depicted in the drawings. For example, if the device is flipped in the drawings, elements or features described as being "below" or "beneath" other elements or features will be oriented "above" other elements or features. Therefore, the example term "below" can include both above and below orientations.

[0025] The terms "distal end" and "proximal end" as used herein are directional terms commonly used in the field of interventional medical devices, where "distal end" refers to the end away from the operator during surgery, and "proximal end" refers to the end close to the operator during surgery. "Coupled" as used herein can be broadly understood as any event in which two or more objects are connected in a manner that allows the absolutely coupled objects to operate together, such that there is no relative movement between the objects in at least one direction, such as a coupling of a protrusion and a groove, which can move relative to each other in the radial direction but not in the axial direction.

[0026] The term "tool" is used herein to describe a medical device that is inserted into a patient's body and used to perform a surgical or diagnostic procedure, the tool comprising an end effector, which may be a surgical tool for performing a surgical procedure, such as an electrocautery device, a clamp, a stapler, a shear, an imaging device (such as an endoscope or ultrasound probe), and the like. Some tools used in embodiments of the present application further include providing an articulated component (such as a joint assembly) for the end effector so that the position and orientation of the end effector can be manipulated and moved with one or more mechanical degrees of freedom relative to the instrument axis. Furthermore, the end effector includes functional mechanical degrees of freedom, such as opening and closing the clamp. The tool may also include stored information that can be updated by the surgical system, whereby the storage system can provide one-way or two-way communication between the tool and one or more system components. Some tools used in some embodiments may also not include providing an articulated component for the end effector.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein in the specification of this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "and / or" and "and / or" as used herein include any and all combinations of one or more of the associated listed items.

[0028] The remote surgical robot of the embodiment of the present application includes a master-slave surgical robot suitable for performing ultra-remote surgery. The master-slave surgical robot includes a remote doctor's main console and a patient-side surgical platform. The patient-side surgical platform can include different types of slave robots, including but not limited to single-port laparoscopic slave robots, multi-port laparoscopic slave robots, bronchial interventional slave robots, vascular interventional slave robots, and orthopedic slave robots. Different types of slave robots have different structural characteristics and may be suitable for the same or different types of surgeries.

[0029] For example, in Figure 1 The master-slave surgical robot shown includes a remote doctor's main console 100 and a patient-side surgical platform 200, which includes a single-port laparoscopic slave robot. The remote doctor's main console 100 can send control commands to the patient-side surgical platform 200 based on the doctor's operations to control the patient-side surgical platform 200. The remote doctor's main console 100 is also used to display images captured by the patient-side surgical platform 200. The patient-side surgical platform 200 responds to control commands sent by the remote doctor's main console 100, performs corresponding operations, and captures images of the patient's body, such as images of the surgical environment within the patient's body.

[0030] The patient-side surgical platform 200 includes a robotic arm 210 and a drive assembly disposed on the robotic arm 210. The drive assembly includes a driver 220 disposed on the robotic arm 210 and a driver 220 disposed on the driver 220. Figure 2 The surgical tool 230 is shown. The patient-side surgical platform 200 also includes a puncture device 240 mounted on the long axis 231 of the surgical tool 230. When the patient-side surgical platform 200 responds to control commands from the remote doctor's main console 100, the robotic arm 210 is used to adjust the position of the surgical tool 230, the driver 220 is used to drive the surgical tool 230 to perform the corresponding operation, and the end effector 232 of the surgical tool 230 is used to extend into the patient's body and perform surgical operations and / or obtain images of the patient's body through its distal end instrument.

[0031] Figure 3 Another patient-side surgical platform 200' is shown. This patient-side surgical platform 200' includes a multi-port laparoscope slave robot. The patient-side surgical platform 200' includes a robotic arm and multiple manipulator assemblies 230' disposed on the robotic arm. The robotic arm includes a main arm 210' and multiple adjustment arms 220' disposed on an orientation platform 215' within the main arm 210'. Different manipulator assemblies 230' are disposed on different adjustment arms 220'. The main arm 210' can adjust the position of the adjustment arms 220' and the manipulator assemblies 230', and the adjustment arms 220' can adjust the position of the manipulator assemblies 230'. The manipulator assembly 230' includes a holding arm 240' and a medical device 250' detachably mounted on the manipulator 240'. The manipulator assembly 230' includes a parallelogram mechanism. Utilizing the parallelogram principle, the holding arm 240' can be limited to rotational motion around a remote center of motion (RC). Multiple medical devices 250' can be inserted into the patient's body through different puncture devices 500'. It is understandable that Figure 1 The remote doctor main console 100 shown can also be used to operate Figure 3 Movement of manipulator assembly 230 ′ is shown in patient-side surgical platform 200 ′.

[0032] Figure 4An interventional surgical robot 300 is shown. The interventional surgical robot 300 is a natural cavity surgical robot, comprising a remote doctor main console and a patient-side surgical platform 320. The remote doctor main console comprises a handle 310' and an imaging cart 330 that are interconnected. The patient-side surgical platform 320 may also comprise an imaging cart 330. The patient-side surgical platform 320 is connected to a catheter instrument 340, a sensor system 350, and a control system 360 for achieving control among the catheter instrument 340, the sensor system 350, and the imaging cart 330. When the doctor performs various procedures on the patient next to the patient-side surgical platform 320, he can trigger control instructions by operating the handle 310' and transmit them to the patient-side surgical platform 320 for driving, thereby controlling the catheter instrument 340 to advance, retract, bend, and turn.

[0033] It is understood that the patient-side surgical platform 320 can generally be moved to the side of the surgical bed to engage the catheter instrument 340 and, under control commands, control the catheter instrument 340 to move vertically, horizontally, or in non-vertical and non-horizontal directions, thereby providing a better preoperative preparation angle for the operation of the catheter instrument 340. The control commands can be triggered by the doctor operating the patient-side surgical platform 320, or by the doctor directly clicking or pressing a button on the patient-side surgical platform 320. Of course, in other embodiments, the control commands can also be voice control or commands triggered by a force feedback mechanism.

[0034] like Figure 4 As shown, the patient-side surgical platform 320 may further include a base 321, a sliding base 322 that can be raised and lowered along the base 321, and two robotic arms 323 fixedly connected to the sliding base 322. The robotic arms 323 may include multiple arm segments connected at joints. The multiple arm segments provide the robotic arms 323 with multiple degrees of freedom, for example, seven degrees of freedom corresponding to the seven arm segments. A manipulator ( Figure 4 (Not shown) The manipulator of robotic arm 323 is used to engage catheter instrument 340 and, driven by the manipulator, control the distal end of catheter instrument 340 to bend and turn accordingly. The two robotic arms 323 can be identical in structure or partially identical in structure, with one robotic arm 323 engaging the inner catheter instrument 341 and the other engaging the outer catheter instrument 342. During installation, the outer catheter instrument 342 can be installed first. After the outer catheter instrument 342 is installed, the catheter of the inner catheter instrument 341 is inserted into the catheter of the outer catheter instrument 342.

[0035] The sensor system 350 has one or more subsystems for receiving information about the catheter device 340. The subsystems may include: a position sensor system; a shape sensor system for determining the position, orientation, speed, velocity, pose, and / or shape of the tip of the catheter device 340 and / or one or more segments along the catheter that may comprise the catheter device 340; and / or a visualization system for capturing images from the tip of the catheter device 340.

[0036] The imaging vehicle 330 can be provided with a display system 331 and a washing system ( Figure 4 Display system 331 is used to display images or representations of the surgical site and catheter device 340 generated by the subsystems of sensor system 350. It can also display real-time images of the surgical site and catheter device 340 captured by the visualization system. It can also use image data from imaging technologies to present images of the surgical site recorded before or during surgery. Imaging technologies include computed tomography (CT), magnetic resonance imaging (MRI), optical coherence tomography (OCT), and ultrasound.

[0037] The preoperative or intraoperative image data can be presented as a two-dimensional, three-dimensional, or four-dimensional image (e.g., time-based or rate-based information) and / or as an image from a model created based on the preoperative or intraoperative image dataset, or as a virtual navigation image. In the virtual navigation image, the actual position of the catheter device 340 is registered with the preoperative image to present a virtual image of the catheter device 340 within the surgical site to the operator from the outside.

[0038] Control system 360 includes at least one memory and at least one processor. It will be appreciated that control system 360 can be integrated into patient-side surgical platform 320 or imaging cart 330, or can be independently configured. Control system 360 can support wireless communication protocols such as IEEE 802.11, IrDA, Bluetooth, HomeRF, DECT, and wireless telemetry. Control system 360 can transmit one or more signals instructing the manipulator to move catheter device 340. Catheter device 340 can be extended to a surgical site within the body through a natural orifice or surgical incision in the patient.

[0039] Furthermore, the control system 360 may include a mechanical control system ( Figure 4 not shown) and an image processing system ( Figure 4 (not shown). The mechanical control system is used to control the movement of the catheter instrument 340 and can therefore be integrated into the patient surgical platform 320. The image processing system is used to plan the virtual navigation path and can therefore be integrated into the imaging cart 330. Of course, the various subsystems of the control system 360 are not limited to the specific ones listed above and can be reasonably configured according to actual circumstances.

[0040] Among them, the image processing system can use the above-mentioned imaging technology to image the surgical site based on the image of the surgical site recorded before or during the operation, and can also use software combined with manual input to convert the recorded image into a two-dimensional or three-dimensional synthetic image of part or the entire anatomical organ or segment. During virtual navigation, the sensor system 350 can be used to calculate the position of the catheter instrument 340 relative to the patient's anatomical structure, which can be used to generate an external tracking image and an internal virtual image of the patient's anatomical structure, so as to achieve the actual position of the catheter instrument 340 and the preoperative image. The virtual image of the catheter instrument 340 in the surgical site can be presented to the operator from the outside.

[0041] The internal catheter device 341 and the external catheter device 342 have substantially the same structural composition, and each comprises a slender and flexible inner catheter 41 and an outer catheter 42, wherein the diameter of the outer catheter 42 is slightly larger than that of the inner catheter 41, so that the inner catheter 41 can pass through the outer catheter 42 and provide a certain degree of support for the inner catheter 41, thereby enabling the inner catheter 41 to reach the target location in the patient's body, so as to facilitate operations such as tissue or cell sampling from the target location.

[0042] Certain movements of handle 310 can cause corresponding movements of catheter device 340. For example, when a doctor operates handle 310 to move the directional lever upward or downward, the movement of the directional lever can be mapped to a corresponding pitch movement of the distal end of catheter device 340. When a doctor operates handle 310 to move the directional lever left or right, the movement of the directional lever can be mapped to a corresponding yaw movement of the distal end of catheter device 340. In this embodiment, handle 310 can control the movement of the distal end of catheter device 340 within a 360-degree spatial range.

[0043] The remote surgical robot of this application also includes a server. Figure 5 As shown, remote doctor master consoles A1-A3 and patient-side surgical platforms B1-B3 of the same or different master-slave surgical robots are each connected to a server S for communication. Server S can be used to manage information sent by remote doctor master consoles A1-A3 and patient-side surgical platforms B1-B3 and can be used to relay information transmitted between remote doctor master consoles A1-A3 and patient-side surgical platforms B1-B3. Server S can also be used to achieve device interconnection, i.e., pairing, between remote doctor master consoles A1-A3 and patient-side surgical platforms B1-B3. Pairing involves connecting two devices to enable communication.

[0044] In ultra-teleoperative surgery, the remote doctor's main console and the patient-side surgical platform are deployed in different locations. For example, they can be deployed in different countries, different provinces and cities, or different jurisdictions in the same city.

[0045] The server may also be deployed at a different location than either the remote physician console or the patient-side surgical platform. For example, the server, the remote physician console, and the patient-side surgical platform may be deployed in different cities. The server may also be deployed at the same location as either the remote physician console or the patient-side surgical platform. For example, the server may be deployed independently of the remote physician console or the patient-side surgical platform located at the same location. For another example, the server may be integrated with the remote physician console or the patient-side surgical platform located at the same location. It is understood that the server may also be deployed in the cloud.

[0046] In the remote surgical robot of the present application, at least one remote doctor main console, patient-side surgical platform and server are deployed respectively.

[0047] There may be at least one remote doctor main console, at least one patient-side surgical platform, and at least one server.

[0048] When there is a single server, each remote physician main console and each patient-side surgical platform are connected to the single server. When there are multiple servers, the servers are connected via a network topology, and different remote physician main consoles and different patient-side surgical platforms can be connected to the same or different servers. The network topology between the multiple servers includes at least one of a star topology, a ring topology, a bus topology, a tree topology, a mesh topology, a virtual local area network, and a wireless topology.

[0049] The remote doctor's main console and the server, the patient's surgical platform and the server, and the servers themselves can be connected via the same or different types of high-speed communication networks. For example, these types of high-speed communication networks may include at least one of the following: broadband internet, a dedicated internet network, a 5G network, and a dedicated 5G network.

[0050] It's understandable that audio transmission via the audio system is a crucial means of conveying information between the patient-side surgical platform and the remote surgeon's main console during telesurgery. Using the audio system, the telesurgery robot can detect audio signals from the patient-side surgical platform and its vicinity, or from the remote surgeon's main console and its vicinity, in real time and reproduce them for those near the device on the other end. For example, communication might occur between the surgeon operating the main console and medical staff surrounding the patient to convey specific instructions or information.

[0051] In a telesurgery robot, both the remote doctor's main console and the patient-side surgical platform will deploy at least one audio system. The audio system can be deployed independently of the remote doctor's main console or the patient-side surgical platform located at the same location, or it can be integrated into the remote doctor's main console or the patient-side surgical platform located at the same location. An audio system generally includes a sound acquisition module, a sound processing module, a sound playback module, or may also include a human-computer interaction module, such as Figure 6 shown.

[0052] The sound collection module is used to collect the sound in the audio environment where the near-end device is currently located, and transmit the collected sound to the sound processing module.

[0053] The sound processing module receives the sound from the sound acquisition module and sends it to the remote device after processing. In addition, the sound processing module also receives and processes the sound collected, processed, and emitted by the audio system deployed by the remote device through the server, and finally plays it through the sound playback module.

[0054] It is understood that the proximal device refers to the device where the audio system is deployed, that is, the device located at the same location as the audio system, and can be either the remote physician's main console or the patient-side surgical platform. The remote device refers to the device that is interconnected and communicates with the proximal device, and can be the other of the remote physician's main console and the patient-side surgical platform.

[0055] It is understandable that the interconnection between the near-end device and the far-end device includes the interconnection between the audio system deployed in the near-end device and the audio system deployed in the far-end system, so as to realize the audio signal transmission between the near-end device and the far-end device.

[0056] However, when the audio systems of multiple devices are interconnected, a single device may receive audio signals from multiple connected devices simultaneously. If the device only needs to focus on the audio signal from a specific device, audio signals from other devices may interfere with the audio signal from that specific device, affecting the transmission of commands or information between specific devices during remote surgery and potentially posing a safety risk to the surgery.

[0057] Based on this, the present application proposes a surgical robot and an audio system interconnection method thereof. The surgical robot includes a server and a first device and a second device that can communicate with the server. The first device and the second device include a main console and a patient-side surgical platform that can be manipulated by the main console. The audio system interconnection method is applied to the first audio system of the surgical robot. The first audio system is configured on the first device, and the second device is configured with the second audio system. That is, the first audio system is deployed on the first device, and the second audio system is deployed on the second device. The first audio system and the second audio system can also communicate with the server. In an embodiment of the present application, the surgical robot may include a remote surgical robot.

[0058] In some embodiments, reference Figure 7 , the present application provides an audio system interconnection method comprising: S110: Acquire in real time second site information of a target second audio system that can be privately paired with the first audio system.

[0059] S120. In response to receiving the target site information of the target audio system determined based on the second site information, send a private pairing request to the target audio system, so that the target audio system generates a response instruction to the private pairing request, and establishes a private pairing with the first audio system based on the first site information when the response instruction is an affirmative response instruction.

[0060] S130 : In response to receiving an affirmative response instruction generated by the target audio system making an affirmative response to the private pairing request, establish a private pairing with the target audio system based on the target site information to implement private intercom with the target audio system.

[0061] In step S110, as described above, the first device is one of the main control console and the patient-side surgical platform, and the first audio system is deployed at the same location as the first device. For example, the first device is the patient-side surgical platform, which is deployed in the operating room, and the first audio system belongs to the first device and is deployed in the same operating room as the first device. The first audio system can be deployed independently of the first device or integrated into the first device, and can collect audio signals emitted by sound sources in the audio environment where the first device is located, and process the audio signals and send them to the interconnected remote devices and / or the audio systems deployed in the interconnected remote devices, and can receive audio signals sent from the interconnected remote devices and / or the audio systems deployed in the interconnected remote devices, and process the audio signals and broadcast them.

[0062] The audio environment in which the first device is located refers to an environment in which the first device is located and has at least one sound source distribution.

[0063] For example, if the first device is a patient-side surgical platform, the audio environment in which the first device resides is the environment in which the patient-side surgical platform is located, with at least one sound source distributed. In this audio environment, sound sources generally include various personnel, instruments, monitoring equipment, and the audio environment itself. These personnel include doctors, nurses, and patients, whose conversations generate audio signals, as do the patient's physiological activities (such as heartbeat and breathing). Instruments inserted into the patient's body and used to perform surgical or diagnostic procedures also generate audio signals. Monitoring equipment (such as a monitor) monitors the patient's condition in real time and issues an alarm if the patient's condition deviates. The audio environment itself refers to the ambient noise generated by the audio environment, which may originate from the reflection of sound waves by the audio environment.

[0064] In addition, the target second audio system that can be privately paired with the first audio system is a second audio system that is interconnected with the first audio system, or the target second audio system is a second audio system that is registered with the server and has agreed on a matching custom protocol with the first audio system. Among them, the target second audio system that can be privately paired with the first audio system is a second audio system that is interconnected with the first audio system, corresponding to the scenario where all audio systems in the remote surgical robot are in an interconnected state (including the default interconnected state and the established interconnection), or all audio systems in the remote surgical robot that can be paired with the first audio system are in an interconnected state (including the default interconnected state and the established interconnection) with the first audio system. In these scenarios, any second audio system in the remote surgical robot that is interconnected with the first audio system can be privately paired with the first audio system; the target second audio system is a second audio system that is registered with the server and has agreed on a matching custom protocol with the first audio system, corresponding to the scenario where not all audio systems in the remote surgical robot are in an interconnected state, or not all second audio systems that can be paired with the first audio system are in an interconnected state with the first audio system. In these scenarios, any second audio system in the remote surgical robot that is registered with the server and has agreed on a matching custom protocol with the first audio system can be privately paired with the first audio system.

[0065] It is understood that when the target second audio system is a second audio system registered with the server and has agreed to a matching custom protocol with the first audio system, the target second audio system and the first audio system may or may not be connected. If the target audio system determined in the target second audio system is not connected to the first audio system, then when establishing a private pairing between the first audio system and the target audio system, the server will need to forward the corresponding pairing request and audio system network information to establish a connection between the first audio system and the target audio system.

[0066] Custom protocol matching means that different audio systems support the same custom protocol for pairing, primarily including the same format specified by the custom protocol. For example, if a first audio system and a second audio system have agreed on matching custom protocols, this means that the custom protocol supported by the first audio system for pairing is the same as the custom protocol supported by the second audio system for pairing. A custom protocol generally refers to a communication protocol. Pairing between a first audio system and a second audio system that are not registered with the server or that support different custom protocols (i.e., lack matching custom protocols) is prohibited.

[0067] For example, a custom protocol includes a protocol header and a protocol body. The protocol header is the message header, and the protocol body is the message data portion. Different custom protocols typically have the same format for the protocol header, but different formats for the protocol body. Differences in the protocol body can be reflected in at least one of the number of fields, field types, field order, and data exchange format. Figure 8 It illustrates a custom protocol based on TCP protocol.

[0068] It should be noted that, in the embodiments of the present application, the interconnection / connection between audio systems is equivalent to the pairing between audio systems, both of which refer to the establishment of a connection between the audio systems (such as the establishment of a network channel) and the ability to transmit audio signals to each other.

[0069] It is understandable that a custom protocol is also agreed upon between the audio system and the server, so that the audio system can register with the server and communicate with the server.

[0070] It is understandable that the matching custom protocol agreed upon between the audio systems is used for communication between the audio systems, which is different from the custom protocol agreed upon between the audio systems and the server.

[0071] It should be noted that in the embodiments of the present application, the target second audio system is a second audio system interconnected with the first audio system, indicating that the target second audio system and the first audio system have agreed on a matching custom protocol and are capable of transmitting audio signals to each other. In some embodiments, audio systems that have agreed on matching custom protocols are automatically paired. In this embodiment, the second audio system interconnected with the first audio system can be directly determined as the target second audio system.

[0072] In some embodiments, the remote surgical robot includes several master audio systems and slave audio systems cascaded with the master audio systems. The first audio system is the master audio system or the slave audio system. Both the master audio system and the slave audio system can be determined as target audio systems.

[0073] Among them, the master audio system in the remote surgical robot and the slave audio system cascaded with the master audio system can be obtained by configuring the master-slave relationship. In some embodiments, the master-slave relationship of all the audio systems on the network can be configured in the human-computer interaction module of the audio system to configure the network topology of the audio system, such as Figure 9 As shown. Among them, matching custom protocols are agreed upon between the main audio systems, and matching custom protocols are also agreed upon between the main audio system and the slave audio system with a master-slave relationship. The main audio system is cascaded with the slave audio system with a master-slave relationship. Different from the interconnection between the main audio systems, the interconnection between the main audio system and its cascaded slave audio system is a cascade mode with a master-slave relationship. In cascade mode, the main audio system and its cascaded slave audio system play sound synchronously, and the audio signal obtained by the slave audio system will be sent to the main audio system cascaded with it. The audio signals obtained by the main audio system and the slave audio system are mixed by the main audio system and sent to other interconnected main audio systems. The main audio system and its cascaded slave audio system are equivalent to one audio system.

[0074] It is understood that in some embodiments, the custom protocol supported by the slave audio system for pairing may be different from the custom protocol supported by the first audio system, but the custom protocol supported by the master audio system of the slave audio system for pairing is the same as the custom protocol supported by the first audio system. In this case, since the slave audio system can establish an interconnection with the first audio system through the interconnection between the master audio system and the first audio system, it can play the audio signal from the first audio system synchronously with the master audio system. At the same time, the audio signal obtained by the slave audio system can also be mixed by the master audio system and sent to the first audio system for playback. In this embodiment of the application, it is considered that the slave audio system and the first audio system also have agreed on a matching custom protocol. When the target second audio system is a second audio system registered with the server and has agreed on a matching custom protocol with the first audio system, the slave audio system can be determined as the target second audio system that can be privately paired with the first audio system. It is understood that based on the above, when the target second audio system is the second audio system interconnected with the first audio system, the slave audio system can also be determined as the target second audio system that can be privately paired with the first audio system.

[0075] In some embodiments, after the network topology of the audio system is configured, the master audio systems will automatically pair up and be in an interconnected state. The master audio system and its slave audio systems in a master-slave relationship will also automatically pair up and be in an interconnected state, which is a cascade with a master-slave relationship. At the same time, any slave audio system cascaded with a master audio system is also in an interconnected state with other master audio systems interconnected with the master audio system and the slave audio systems cascaded with these master audio systems. That is, the audio signals obtained by the slave audio system can be sent to other master audio systems and the slave audio systems cascaded with these master audio systems for playback after being mixed by the master audio system cascaded with it. The audio signals sent by other master audio systems received by the master audio system will be processed and played synchronously by the master audio system and the slave audio systems cascaded with it. It is understandable that the audio signals sent by other master audio systems received by the master audio system may include audio signals obtained by other master audio systems and audio signals obtained by the slave audio systems cascaded with these master audio systems.

[0076] In some embodiments, after completing the configuration of the network topology of the audio system, the main audio systems will not be automatically paired with each other, and some of the main audio systems that are actively paired are in an interconnected state. The main audio system and its slave audio systems with a master-slave relationship will automatically pair and be in an interconnected state, which is a cascade with a master-slave relationship. At the same time, any slave audio system cascaded with a main audio system is also in an interconnected state with other main audio systems connected to the main audio system and slave audio systems cascaded with these main audio systems, that is, the audio signal obtained by the slave audio system can be sent to other main audio systems connected to the main audio system and slave audio systems cascaded with these main audio systems for playback after being mixed by the main audio system cascaded with it, and the audio signal received by the main audio system from other main audio systems connected to it will be played synchronously in the main audio system and the slave audio system cascaded with it after being processed.

[0077] Taking the second site information of the target second audio system as an example, the site information of the audio system in the embodiment of the present application is described. The second site information of the target second audio system includes at least one of multiple audio system information of the target second audio system, and the at least one audio system information can uniquely represent the target second audio system.

[0078] For security and simplicity, in some embodiments, the second site information may only include the unique identifier of the target second audio system, without providing the audio system network information of the target second audio system. This unique identifier can be named in conjunction with the location information, for example, "XX Hospital, Patient XX, Surgical Platform Audio System." This provides more useful information without compromising communication security.

[0079] To enhance pairing convenience and efficiency, in some embodiments, the second site information of the target second audio system may also include audio system network information associated with a network channel of the target second audio system. This network channel is a network channel established between the audio systems for transmitting corresponding network information (e.g., audio signals). It will be understood that the establishment of a network channel between the first audio system and the target second audio system indicates a connection between the first audio system and the target second audio system, i.e., the first audio system and the target second audio system are interconnected.

[0080] Exemplarily, the audio system network information may include at least one of an IP address, a port number, and a custom protocol of the audio system.

[0081] From this, it can be seen that the network channel is established based on the audio system network information or site information (the case where the site information includes the audio system network information), that is, the connection between audio systems can be established based on the audio system network information or site information (the case where the site information includes the audio system network information).

[0082] It can be understood that if the first audio system and the target second audio system are master audio systems, the first audio system establishes a connection with the target second audio system through the network channel between the first audio system and the target second audio system, and is in an interconnected state with the target second audio system; if the first audio system is a slave audio system and the target second audio system is a master audio system interconnected with the master audio system cascaded with the first audio system, the first audio system establishes a connection with the target second audio system through the network channel between the master audio system cascaded with it and the network channel between the master audio system cascaded with it and the target second audio system, and is in an interconnected state with the target second audio system; if the first audio system is a slave audio system and the target second audio system is a master audio system interconnected with the master audio system cascaded with the first audio system If the first audio system is a slave audio system cascaded with the master audio system, the first audio system establishes a connection with the target second audio system through the network channel between the cascaded master audio system, the network channel between the cascaded master audio system and the interconnected master audio system, and the network channel between the interconnected master audio system and the cascaded target second audio system, and is in an interconnected state with the target second audio system; if the first audio system is the master audio system and the target second audio system is a slave audio system cascaded with the master audio system interconnected with the first audio system, the first audio system establishes a connection with the target second audio system through the network channel between the interconnected master audio system and the network channel between the interconnected master audio system and the cascaded target second audio system, and is in an interconnected state with the target second audio system.

[0083] The second site information of the target second audio system may be stored in the server when the target second audio system is registered with the server. The way for the first audio system to obtain the second site information may be that the first audio system sends an acquisition request to the server, and the server decapsulates the acquisition request based on the custom protocol agreed upon with the first audio system to obtain the custom protocol supported for pairing by the first audio system; alternatively, the second site information of the target second audio system may be stored in the target second audio system, and the first audio system sends an acquisition request, which is forwarded by the server or sent to the target second audio system through the network channel between the first audio system and the target second audio system, and then obtained from the target second audio system; alternatively, when the target second audio system is a second audio system interconnected with the first audio system, the first audio system may directly obtain the second site information of the target second audio system based on the information of the currently interconnected audio system.

[0084] Therefore, the embodiment of the present application first obtains the second site information of the target second audio system that can be privately paired with the first audio system in real time, and can screen out the target second audio system that can be privately paired with the first audio system from the second audio system before subsequently selecting the target audio system for private pairing, and provide the second site information of the target second audio system, so that the target audio system can be conveniently determined based on the second site information in the subsequent private pairing, thereby improving the compatibility of the first audio system and the target audio system in the subsequent private pairing, and at the same time improving the efficiency of private pairing.

[0085] In step S120, the private pairing request includes the first site information of the first audio system. The private pairing request is used to request a private pairing with a target audio system. It is understood that the target audio system is the audio system in the target second audio system that the user (e.g., a doctor) wishes to privately pair with the first audio system.

[0086] In some embodiments, the second site information of the target second audio system acquired in real time in step S110 is displayed in the audio configuration interface, and the second site information of each target second audio system is associated with a control in the audio configuration interface. In this embodiment, determining the target site information of the target audio system based on the second site information may include: In response to a target control among the controls receiving a first preset operation, the second site information associated with the target control is determined as the target site information, so that the audio system corresponding to the target site information is the target audio system.

[0087] The embodiments of the present application do not limit the display style of the control in the audio configuration interface. Optionally, if the control is a button, receiving a first preset operation (such as a single-click or double-click on the target control) on the target control can trigger the first audio system to determine the second site information associated with the target control as the target site information. Alternatively, if the control is a slider, receiving a first preset operation (such as a move or drag operation on the target control) on the target control can trigger the first audio system to determine the second site information associated with the target control as the target site information. Alternatively, if the control is a text box, receiving a first preset operation (such as inputting preset content into the target control) on the target control can trigger the first audio system to determine the second site information associated with the target control as the target site information.

[0088] In order to intuitively display the second site information associated with each control in the audio configuration interface, in some embodiments, the audio configuration interface also displays a virtual module associated with each second site information, and the virtual module displays at least one audio system information in the associated second site information that can uniquely represent the target second audio system.

[0089] It can be understood that in the audio configuration interface, each control is associated with a virtual module.

[0090] The control and the associated virtual module of the embodiment of the present application can be displayed as a whole or separately (the control can be displayed at any position close to the display area of ​​the associated virtual module).

[0091] Therefore, the embodiment of the present application can intuitively determine the target site information of the target audio system based on the second site information by operating the controls in the audio configuration interface. At the same time, this process is simple and easy to understand, reduces learning costs, and improves user experience.

[0092] In some embodiments, the audio configuration interface also displays controls associated with the first audio system, and the network topology of each control. The network topology of each control is associated with the network topology of the audio system described above, that is, the network topology of the audio system can be learned through the network topology of the controls, and the network topology of the audio system can be configured by configuring the network topology of the controls.

[0093] In some embodiments, the audio system interconnection method may further include configuring the network topology of the audio system by configuring the network topology of the control, which may specifically include: In response to a first target control in the controls receiving a second preset operation, the first target control is set as a slave control of the second target control, and a cascade control between the first target control and the second target control is generated and displayed.

[0094] It can be understood that when the first target control is set as a slave control of the second target control, the second target control is the master control of the first target control.

[0095] The second preset operation at least includes an operation of associating the first target control with the second target control. Optionally, the second preset operation includes dragging the first target control to a preset area associated with the second target control; or the second preset operation includes entering preset content in the first target control that has a unique correspondence with the second target control.

[0096] When the first target control receives the second preset operation, it can trigger the first audio system to set the first target control as a slave control of the second target control, and the second target control as the master control of the first target control, and generate and display cascade controls between the first target control and the second target control, so that the audio system corresponding to the site information associated with the first target control is configured as a slave audio system cascaded with the audio system corresponding to the site information associated with the second target control, and the audio system corresponding to the site information associated with the second target control is a master audio system cascaded with the audio system corresponding to the site information associated with the first target control.

[0097] The embodiments of the present application do not limit the display style of cascading controls between controls. For example, a cascading control between controls can be a line between two controls associated with the cascading control; or a cascading control between controls can be an arrow pointing from a slave control to a master control, or from a master control to a slave control, between two controls associated with the cascading control; or a cascading control between controls can be a double arrow pointing from two controls associated with the cascading control to both the master control and the slave control.

[0098] In some embodiments, the main controls are automatically paired and interconnected, and the interconnected controls between the main controls are displayed in the audio configuration interface.

[0099] In other embodiments, the main controls are not automatically paired. When the first main control among the main controls receives the third preset operation, the first main control is paired with the second main control, and interconnected controls between the first main control and the second main control are generated and displayed.

[0100] The third preset operation at least includes an operation of associating the first main control with the second main control. Optionally, the third preset operation includes dragging the first main control to a preset area associated with the second main control; or the third preset operation includes entering preset content in the first main control that has a unique correspondence with the second main control.

[0101] When the first main control receives the third preset operation, it can trigger the first audio system to pair the first main control with the second main control, and generate and display the interconnection control between the first main control and the second main control, thereby interconnecting the audio system corresponding to the site information associated with the first main control with the audio system corresponding to the site information associated with the second main control.

[0102] The embodiments of this application do not limit the display style of interconnected controls between main controls. For example, an interconnected control between main controls can be a line connecting two main controls associated with the interconnected control; or an interconnected control between main controls can be a double arrow pointing between two main controls associated with the interconnected control.

[0103] In some embodiments, the first audio system includes a human-computer interaction module, the human-computer interaction module includes a display screen, and the audio configuration interface is displayed on the display screen.

[0104] Therefore, the embodiment of the present application can intuitively configure the network topology of the audio system by configuring the network topology of the control. At the same time, this process is simple and easy to understand, which reduces the learning cost and improves the user experience.

[0105] As can be seen from the foregoing, in some embodiments, when sending a private pairing request to the target audio system in step S120: If the first audio system and the target audio system are not connected, the private pairing request is encapsulated into a data packet in the format specified by the custom protocol agreed upon between the first audio system and the server and sent to the server. The server then forwards the private pairing request to the target audio system. It will be understood that the private pairing request includes audio system information uniquely identifying the target audio system within the target site information, instructing the server to forward the private pairing request to the target audio system.

[0106] If the first audio system and the target audio system are interconnected, the private pairing request is encapsulated into a data message in a format specified by a matching custom protocol agreed upon by the first audio system and the target audio system and sent to the target audio system.

[0107] In some embodiments, when step S120 sends a private pairing request to the target audio system: If the first audio system is the primary audio system, the target audio system is the primary audio system, and the first audio system and the target audio system are interconnected, the private pairing request is encapsulated into a data message in the format specified by the matching custom protocol agreed upon by the first audio system and the target audio system and sent to the target audio system.

[0108] If the first audio system is the primary audio system, the target audio system is the primary audio system, and the first audio system and the target audio system are not interconnected, the private pairing request is encapsulated into a data message in the format specified by the custom protocol agreed upon by the first audio system and the server and sent to the server. The server then forwards the private pairing request to the target audio system.

[0109] If the first audio system is the master audio system, the target audio system is the slave audio system, and the first audio system and the target audio system are interconnected, the private pairing request is encapsulated into a data message in the format specified by the matching custom protocol agreed upon by the first audio system and the target audio system and sent to the target audio system. Alternatively, the private pairing request is encapsulated into a data message in the format specified by the matching custom protocol agreed upon by the master audio system cascaded with the first audio system and the target audio system and sent to the master audio system, which then forwards the data message to the target audio system.

[0110] If the first audio system is the master audio system and the target audio system is the slave audio system, and the first audio system and the target audio system are not connected, the private pairing request is encapsulated into a data message in the format specified by the custom protocol agreed upon by the first audio system and the server and sent to the server. The server then forwards the private pairing request to the target audio system.

[0111] If the first audio system is a slave audio system, the target audio system is a master audio system, and the first audio system and the target audio system are interconnected, the private pairing request is encapsulated into a data message in the format specified by the matching custom protocol agreed upon by the first audio system and the target audio system and sent to the target audio system. Alternatively, the private pairing request is encapsulated into a data message in the format specified by the matching custom protocol agreed upon by the master audio system cascaded with the first audio system and the target audio system and sent to the master audio system, which then forwards the data message to the target audio system.

[0112] If the first audio system is a slave audio system, the target audio system is a master audio system, and the first audio system and the target audio system are not interconnected, the private pairing request is encapsulated into a data message in the format specified by the custom protocol agreed upon by the first audio system and the server and sent to the server. The server then forwards the private pairing request to the target audio system.

[0113] If the first audio system is a slave audio system, the target audio system is a slave audio system, and the first audio system and the target audio system are interconnected, the private pairing request is encapsulated into a data message in the format specified by the matching custom protocol agreed upon by the first audio system and the target audio system and sent to the target audio system, or the private pairing request is encapsulated into a data message in the format specified by the matching custom protocol agreed upon by the main audio system cascaded by the first audio system and the main audio system cascaded by the target audio system and sent to the main audio system cascaded by the first audio system, which is then forwarded by the main audio system cascaded by the target audio system to the target audio system, and then forwarded by the main audio system cascaded by the target audio system to the target audio system.

[0114] If the first audio system is a slave audio system, the target audio system is a slave audio system, and the first audio system and the target audio system are not interconnected, the private pairing request is encapsulated into a data message in the format specified by the custom protocol agreed upon by the first audio system and the server and sent to the server. The server then forwards the private pairing request to the target audio system.

[0115] It can be understood that in any of the above situations, the embodiments of the present application can directly encapsulate the private pairing request into a data message in the format specified by the custom protocol agreed upon by the first audio system and the server and send it to the server, and the server forwards the private pairing request to the target audio system, without considering the interconnection status between the first audio system and the target audio system, and without considering whether the first audio system and the target audio system are master audio systems or slave audio systems.

[0116] When the target audio system receives the private pairing request from the first audio system, it obtains a response instruction generated in real time to the private pairing request, wherein the response instruction includes an affirmative response instruction and a negative response instruction.

[0117] In some embodiments, the response instruction is generated by a user (such as a doctor) who controls the target audio system in response to the private pairing request.

[0118] In some embodiments, when the target audio system receives a private pairing request from the first audio system, it generates and plays a notification based on the private pairing request. The content of the notification is whether to accept the private pairing initiated by the first audio system, so as to facilitate the user to respond to the private pairing request based on the notification.

[0119] For example, the notification can be played in the form of a UI interface or a sound. The user can respond to the private pairing request through the operable controls of the UI interface or through voice recognition. The response includes acceptance or rejection.

[0120] For example, when the notification is played in the form of a UI interface, operable controls such as "Accept private pairing of the first audio system" and "Reject private pairing of the first audio system" may be displayed on the display screen of the human-computer interaction module of the target audio system. When the user operates the operable control "Accept private pairing of the first audio system", it indicates acceptance of the private pairing of the first audio system. Alternatively, when the user operates the operable control "Reject private pairing of the first audio system", it indicates rejection of the private pairing of the first audio system. When the notification is played in the form of sound, the sound playback module of the target audio system may play the message "Do you accept the private pairing of the first audio system?". The user may use the voice recognition device set in the human-computer interaction module of the target audio system to recognize the user's audio input of "Accept private pairing of the first audio system" and generate an affirmative response instruction to accept the private pairing of the first audio system. Alternatively, the user may recognize the user's audio input of "Reject private pairing of the first audio system" and generate a negative response instruction to reject the private pairing of the first audio system.

[0121] In some embodiments, the target audio system can also make the decision to accept or reject the private pairing with the first audio system, that is, the decision is made by the target audio system itself. For example, when the target audio system receives the private pairing request from the first audio system, it can start a timer. If the response waiting time range expires and the target audio system does not receive a response from the user to the notification, the target audio system will generate a response instruction rejecting the private pairing with the first audio system by default, that is, a negative response instruction. The target audio system sends this response instruction to the first audio system to notify it that the private pairing failed.

[0122] In some embodiments, the target audio system can be configured to store a whitelist, which is associated with one or more first audio systems. The target audio system allows the first audio systems associated with the whitelist to pair with it without the user having to respond to the notification. When the target audio system is configured with a whitelist and the first audio system is on the target audio system's whitelist, the user can actively respond to the notification within the response waiting time range. The response includes the target audio system accepting or rejecting the private pairing of the first audio system. When preset conditions are met, such as when the response waiting time range expires and the target audio system does not receive a response from the user to the notification, the target audio system automatically generates a response instruction to accept the private pairing of the first audio system, that is, an affirmative response instruction. The target audio system sends the response instruction to the first audio system to notify that the private pairing request has been accepted.

[0123] In some embodiments, the first audio system may also be configured with a whitelist associated with one or more target audio systems. The first audio system allows the target audio systems associated with the whitelist to pair with it without the user having to respond to a notification. The specific details are the same as those for configuring a target audio system to store a whitelist, and are not further described here.

[0124] It can be understood that when the user responds to the private pairing request by accepting (i.e., an affirmative response), the response instruction is an affirmative response instruction; when the user responds to the private pairing request by rejecting (i.e., a negative response), the response instruction is a negative response instruction.

[0125] The transmission path and method of the affirmative response instruction (or negative response instruction) sent by the target audio system to the first audio system are similar to the transmission path and method of the private pairing request sent by the first audio system to the target audio system, and are not repeated here.

[0126] In step S130, upon receiving the positive response instruction generated by the target audio system in response to the private pairing request, the first audio system establishes a private pairing with the target audio system based on the target site information. Thus, both the first audio system and the target audio system establish a private pairing with their peers, enabling private intercom communication between the first audio system and the target audio system. The peer of the audio system refers to the audio system connected to the audio system.

[0127] It should be noted that in the embodiments of the present application, private intercom achieved after both the first audio system and the target audio system have established private pairing with the other end means that the first audio system can only play the target audio signal obtained by the target audio system, and the first audio signal obtained by the first audio system can only be played by the target audio system. Similarly, the target audio system can only play the first audio signal, and the target audio signal can only be played by the first audio system.

[0128] It can be understood that the target audio system can only play the first audio signal, and the target audio signal can only be played by the first audio system. It is necessary to perform an operation on the target audio system side as in the embodiment of the present application, in which the first audio system establishes a private pairing with the target audio system based on the target site information, that is, as described in step S120: when the response instruction is an affirmative response instruction, the target audio system establishes a private pairing with the first audio system based on the first site information. The specific steps are similar to the specific steps of the first audio system establishing a private pairing with the target audio system based on the target site information.

[0129] The following describes an embodiment in which both the first audio system and the target audio system establish private pairing with the other end and implement private intercom.

[0130] In some embodiments, establishing a private pairing between the first audio system and the target audio system based on the target site information may include: Based on the target site information, disconnecting from other audio systems except the target audio system; It is confirmed that no connection is established with the target audio system, and a connection is established with the target audio system based on the target site information.

[0131] Likewise, the target audio system establishing a private pairing with the first audio system based on the first site information may include: based on the first site information, disconnecting from other audio systems except the first audio system; It is confirmed that no connection is established with the first audio system, and a connection is established with the first audio system based on the first site information.

[0132] Specifically, in this embodiment, the first audio system can, based on the target site information, sever the connection (e.g., network channel) between the first audio system and any audio systems other than the target audio system, retaining / establishing only the connection (e.g., network channel) between the first audio system and the target audio system, thereby achieving private pairing with the target audio system. Similarly, when the response instruction is a positive response instruction, the target audio system will, based on the first site information, sever the connection (e.g., network channel) between the target audio system and any audio systems other than the first audio system, retaining / establishing only the connection (e.g., network channel) between the target audio system and the first audio system, thereby achieving private pairing with the first audio system. Consequently, both the first audio system and the target audio system establish private pairing with their respective counterparts, enabling private intercom.

[0133] Alternatively, in some embodiments, establishing a private pairing between the first audio system and the target audio system based on the target site information may include: Confirm that no connection is established with the target audio system, and establish a connection with the target audio system based on the target site information; generating a first private intercom instruction based on the target site information, and sending the first private intercom instruction to the audio systems with which the connection is established, so that only the target audio system among the connected audio systems responds to the first private intercom instruction and plays the first audio signal acquired by the first audio system; According to the first private intercom instruction, playing of audio signals other than the target audio signal acquired by the target audio system is suppressed.

[0134] Likewise, the target audio system establishing a private pairing with the first audio system based on the first site information may include: confirming that a connection with the first audio system is not established, and establishing a connection with the first audio system based on the first site information; generating a fourth private intercom instruction based on the first site information, and sending the fourth private intercom instruction to the audio systems with which the connection is established, so that only the first audio system among the connected audio systems responds to the fourth private intercom instruction and plays the target audio signal acquired by the target audio system; According to the fourth private intercom instruction, playing of audio signals other than the first audio signal acquired by the first audio system is suppressed.

[0135] Specifically, in this embodiment, the first audio system can generate a first private intercom command based on the target site information. Only the target audio system can respond to the first private intercom command and play the first audio signal. Furthermore, based on the first private intercom command, the first audio system will suppress the playback of audio signals other than the target audio signal, thereby achieving private pairing with the target audio system. Similarly, when the response command is a positive response command, the target audio system will generate a fourth private intercom command based on the first site information. Only the first audio system can respond to the fourth private intercom command and play the target audio signal. Furthermore, based on the fourth private intercom command, the target audio system will suppress the playback of audio signals other than the first audio signal, thereby achieving private pairing with the first audio system. As a result, both the first audio system and the target audio system establish private pairing with their counterparts, achieving private intercom.

[0136] Alternatively, in some embodiments, establishing a private pairing between the first audio system and the target audio system based on the target site information may include: Confirm that no connection is established with the target audio system, and establish a connection with the target audio system based on the target site information; generating a second private intercom instruction based on the target site information; According to the second private intercom instruction, the acquired first audio signal is sent to the target audio system, and the playing of audio signals other than the target audio signal acquired by the target audio system is suppressed.

[0137] Likewise, the target audio system establishing a private pairing with the first audio system based on the first site information may include: confirming that a connection with the first audio system is not established, and establishing a connection with the first audio system based on the first site information; generating a fifth private intercom instruction based on the first site information; According to the fifth private intercom instruction, the acquired target audio signal is sent to the first audio system, and the playing of audio signals other than the first audio signal acquired by the first audio system is suppressed.

[0138] Specifically, in this embodiment, the first audio system can generate a second private intercom instruction based on the target site information. Based on the second private intercom instruction, the first audio signal can only be sent by the first audio system to the target audio system, and the first audio system will suppress the playback of audio signals other than the target audio signal based on the second private intercom instruction, thereby achieving private pairing with the target audio system. Similarly, when the response instruction is a positive response instruction, the target audio system will generate a fifth private intercom instruction based on the first site information. Based on the fifth private intercom instruction, the target audio signal can only be sent by the target audio system to the first audio system, and the target audio system will suppress the playback of audio signals other than the first audio signal based on the fifth private intercom instruction, thereby achieving private pairing with the first audio system. As a result, both the first audio system and the target audio system establish private pairing with the other end, achieving private intercom.

[0139] Alternatively, in some embodiments, establishing a private pairing between the first audio system and the target audio system based on the target site information may include: Confirm that no connection is established with the target audio system, and establish a connection with the target audio system based on the target site information; generating a third private intercom instruction based on the target site information, and sending the third private intercom instruction to the audio systems with which the connection has been established, so that the audio systems other than the target audio system among the connected audio systems respond to the third private intercom instruction and suppress the playback of the first audio signal acquired by the first audio system; According to the third private intercom instruction, playing of audio signals other than the target audio signal acquired by the target audio system is suppressed.

[0140] Likewise, the target audio system establishing a private pairing with the first audio system based on the first site information may include: confirming that a connection with the first audio system is not established, and establishing a connection with the first audio system based on the first site information; generating a sixth private intercom instruction based on the first site information, and sending the sixth private intercom instruction to the audio system with which the connection has been established, so that the audio systems other than the first audio system in the established audio system respond to the sixth private intercom instruction and suppress the playback of the target audio signal acquired by the target audio system; According to the sixth private intercom instruction, playing of audio signals other than the first audio signal acquired by the first audio system is suppressed.

[0141] Specifically, in this embodiment, the first audio system can generate a third private intercom instruction based on the target site information. Upon receiving the third private intercom instruction, audio systems other than the target audio system will suppress the playback of the first audio signal in accordance with the third private intercom instruction. Furthermore, the first audio system will suppress the playback of audio signals other than the target audio signal based on the third private intercom instruction, thereby achieving private pairing with the target audio system. Similarly, when the response instruction is a positive response instruction, the target audio system will generate a sixth private intercom instruction based on the first site information. Upon receiving the sixth private intercom instruction, audio systems other than the first audio system will suppress the playback of the target audio signal in accordance with the sixth private intercom instruction. Furthermore, the target audio system will suppress the playback of audio signals other than the first audio signal based on the sixth private intercom instruction, thereby achieving private pairing with the first audio system. As a result, both the first audio system and the target audio system establish private pairing with their counterparts, achieving private intercom.

[0142] It can be understood that, by cutting off other connections (such as network channels) and retaining / establishing only the connection (such as the network channel) between the first audio system and the target audio system, a "private intercom channel" is constructed between the first audio system and the target audio system. Based on the "private intercom channel", the first audio system receives only the target audio signal as the audio signal, and only sends the acquired first audio signal to the target audio system. At the same time, the target audio system receives only the first audio signal as the audio signal, and only sends the acquired target audio signal to the first audio system. As a result, the first audio system and the target audio system do not need to suppress audio signals other than the private intercom, thereby reducing the amount of algorithm calculation (such as the amount of calculation of the algorithm for identifying audio signals and performing audio signal suppression), and avoiding audio interference problems caused by poor audio suppression effect. At the same time, the amount of data transmission between the first audio system / target audio system and other audio systems is greatly reduced, bandwidth pressure is alleviated, audio processing efficiency is improved, and the quality of the private intercom is improved.

[0143] It can be understood that, compared with the above-mentioned embodiment of constructing a "private intercom channel", the embodiment of private pairing based on private intercom instructions does not need to cut off the connection between the first audio system / target audio and other audio systems (such as the network channel), so that after the private intercom ends, there is no need to re-establish the connection with other audio systems, and the switching of the intercom mode is faster and more flexible.

[0144] It is understandable that when the target site information does not include the audio system network information of the target audio system associated with the network channel, the embodiment of the present application can obtain the audio system network information based on the target site information to facilitate establishing a connection with the target audio system. In some embodiments, the audio system network information of the target audio system associated with the network channel can be forwarded to the first audio system by the server according to an affirmative response instruction, or the audio system network information can be included in the affirmative response instruction and obtained by the first audio system based on the target site information after decapsulation based on a custom protocol matching the target audio system. It is understandable that the specific process of establishing a connection with the corresponding audio system based on other site information is the same as that of establishing a connection with the target audio system based on the target site information, and will not be repeated here.

[0145] Based on the aforementioned embodiments of establishing a "private intercom channel" and performing private pairing based on private intercom commands, some embodiments of the present application consider a situation where a remote surgical robot includes multiple master audio systems and slave audio systems cascaded with the master audio system, and the first audio system is either the master audio system or the slave audio system, and both the master audio system and the slave audio system can be determined as the target audio system. This situation is described. In this embodiment, after private pairing is established between the first audio system and the target audio system: The first audio system only plays the target audio signal acquired by the target audio system, and the target audio system only plays the first audio signal acquired by the first audio system. The audio system cascaded with the first audio system does not play the target audio signal, and the audio system cascaded with the target audio system does not play the first audio signal.

[0146] It is understood that, based on the aforementioned cascade mode, in this embodiment of the present application, after a private pairing is established between a first audio system and a target audio system, the first audio system will only play the target audio signal acquired by the target audio system and will not play the audio signal acquired by the audio system cascaded with the target audio system; the target audio system will only play the first audio signal acquired by the first audio system and will not play the audio signal acquired by the audio system cascaded with the first audio system; the audio system cascaded with the first audio system will only play audio signals other than the target audio signal acquired by the target audio system (e.g., the audio signal acquired by the audio system cascaded with the target audio system); and the audio system cascaded with the target audio system will only play audio signals other than the first audio signal acquired by the first audio system (e.g., the audio signal acquired by the audio system cascaded with the first audio system). In other words, the first audio system and the target audio system are engaged in a private intercom, and the audio systems cascaded with the first audio system and the target audio system, since they are not selected, will not play the private intercom audio. However, the audio systems cascaded with the first audio system and the target audio system can normally engage in non-private intercom.

[0147] It is understandable that the private intercom based on the cascade mode described in the previous paragraph can be implemented through the aforementioned embodiment of private pairing based on the private intercom instruction, or it can also be implemented by establishing a "private intercom channel" between the first audio system and the target audio system and through an audio suppression instruction, as described in detail below.

[0148] For example, referring to Figures 10(a) and 10(b), the first audio system is slave audio system 1 in Figures 10(a) and 10(b), and the target audio system is master audio system 2 in Figures 10(a) and 10(b). Slave audio system 1 is cascaded with master audio system 1, slave audio system 2 is cascaded with master audio system 2, and master audio system 1 and master audio system 2 are interconnected. W1, W2, W3, and W4 are the audio signals obtained from slave audio system 1, master audio system 1, slave audio system 2, and master audio system 2, respectively. 10( a ), in cascade mode, the slave audio system 1 synchronously plays the audio signal from the master audio system 2 (including the audio signal W3 obtained by the master audio system 2 and the audio signal W4 obtained by the slave audio system 2 mixed by the master audio system 2). The audio signal W1 obtained by the slave audio system 1 and the audio signal W2 obtained by the master audio system 1 are mixed by the master audio system 1 and then sent to the master audio system 2, and are synchronously played by the master audio system 2 and the slave audio system 2. 10( b ), after a private pairing is established between the first audio system and the target audio system, that is, after a private pairing is established between the slave audio system 1 and the main audio system 2, the slave audio system 1 and the main audio system 2 have a private conversation. At this time, the main audio system 1 receives the audio signal W3 obtained by the main audio system 2 and the audio signal W4 obtained by the slave audio system 2 mixed by the main audio system 2. The slave audio system 1 only plays the audio signal W3 obtained by the main audio system 2 and does not play the audio signal W4 obtained by the slave audio system 2, that is, "W4=0" as shown in FIG. 10( b ). At the same time, the main audio system 2 does not play the audio signal W3 obtained by the main audio system 2. The audio signal W3 is obtained from the slave audio system 1, which is "W3=0" as shown in Figure 10(b). The audio signal W1 obtained from the slave audio system 1 and the audio signal W2 obtained from the master audio system 1 are mixed by the master audio system 1 and sent to the master audio system 2. However, only the master audio system 2 can play the audio signal W1 obtained from the slave audio system 1, and the slave audio system 2 cannot play the audio signal W1 obtained from the slave audio system 1, which is "W1=0" as shown in Figure 10(b). Moreover, the master audio system 2 can only play the audio signal W1 obtained from the slave audio system 1 and cannot play the audio signal W2 obtained from the master audio system 2, which is "W2=0" as shown in Figure 10(b). In addition, as shown in Figure 10(b), the master audio system 1 can normally play the audio signal W4 obtained from the slave audio system 2, and the slave audio system 2 can normally play the audio signal W2 obtained from the master audio system 1. In other words, the private intercom between the slave audio system 1 and the master audio system 2 does not affect the non-private intercom between the master audio system 1 and the slave audio system 2.

[0149] It should be noted that in FIG. 10( a ) and FIG. 10 ( b ), the slave audio system 1 , the master audio system 1 , the slave audio system 2 , and the master audio system 2 obtain audio signals through microphones and play the audio signals through speakers.

[0150] Specifically, when the target second audio system is a second audio system interconnected with the first audio system, in the embodiment of the present application, in establishing a "private intercom channel" between the first audio system and the target audio system, the first audio system may establish a private pairing with the target audio system based on the target site information, including: Determine third site information of a third audio system in the second site information based on the target site information, where the third audio system is an audio system other than the target audio system in the target second audio system; and cut off the connection with the third audio system based on the third site information.

[0151] Alternatively, the first audio system establishing a private pairing with the target audio system based on the target site information may include: The connection with the target second audio system is cut off based on the second site information; and the connection with the target audio system is established based on the target site information.

[0152] Specifically, when the target second audio system is a second audio system registered with the server and has agreed on a matching custom protocol with the first audio system, in the embodiment of the present application, when establishing a "private intercom channel" between the first audio system and the target audio system, the first audio system may establish a private pairing with the target audio system based on the target site information, including: Based on the target site information, it is determined whether the target audio system is connected.

[0153] If so, obtain the fourth site information of the fourth audio system interconnected with the first audio system, and determine the fifth site information of the fifth audio system in the fourth site information based on the target site information, where the fifth audio system is the audio system in the fourth audio system other than the target audio system; and cut off the connection with the fifth audio system based on the fifth site information.

[0154] If not, obtain the fourth site information, and cut off the connection with the fourth audio system based on the fourth site information; and establish a connection with the target audio system based on the target site information.

[0155] Alternatively, the first audio system establishing a private pairing with the target audio system based on the target site information may include: Acquire fourth site information, and cut off the connection with the fourth audio system based on the fourth site information; and establish a connection with the target audio system based on the target site information.

[0156] Some embodiments of the present application, based on the aforementioned specific embodiments, consider the situation where the remote surgical robot includes several main audio systems and slave audio systems cascaded with the main audio system, and the first audio system is the main audio system or the slave audio system, and both the main audio system and the slave audio system can be determined as the target audio system.

[0157] In this embodiment, when the target second audio system is a second audio system interconnected with the first audio system, in establishing a "private intercom channel" between the first audio system and the target audio system, the first audio system may establish a private pairing with the target audio system based on the target site information, including: Fifth site information of a fifth audio system is determined in the second site information based on the target site information, the fifth audio system being an audio system in the target second audio system excluding the audio system cascaded with the first audio system and the audio system cascaded with the target audio system.

[0158] The connection with the fifth audio system is severed based on the fifth site information.

[0159] Suppresses playback of audio signals acquired by an audio system cascaded with the target audio system.

[0160] An audio suppression instruction is generated and sent to an audio system cascaded with the first audio system, so that the audio system cascaded with the first audio system suppresses the playback of the target audio signal acquired by the target audio system in response to the audio suppression instruction.

[0161] In addition, in this embodiment, when the target second audio system is a second audio system registered with the server and has agreed on a matching custom protocol with the first audio system, in establishing a "private intercom channel" between the first audio system and the target audio system, the first audio system may establish a private pairing with the target audio system based on the target site information, which may include: Based on the target site information, it is determined whether the target audio system is connected.

[0162] If so, obtain the sixth site information of the sixth audio system interconnected with the first audio system, and determine the seventh site information of the seventh audio system in the sixth site information based on the target site information, the seventh audio system being the audio system in the sixth audio system excluding the audio system cascaded with the first audio system and the audio system cascaded with the target audio system, and cut off the connection with the seventh audio system based on the seventh site information; suppress the playback of the audio signal obtained by the audio system cascaded with the target audio system; generate an audio suppression instruction, and send the audio suppression instruction to the audio system cascaded with the first audio system, so that the audio system cascaded with the first audio system responds to the audio suppression instruction and suppresses the playback of the target audio signal obtained by the target audio system.

[0163] If not, obtain the sixth site information, and determine the eighth site information of the eighth audio system from the sixth site information, the eighth audio system being the audio system in the sixth audio system excluding the audio system cascaded with the first audio system; cut off the connection with the eighth audio system based on the eighth site information; establish a connection with the target audio system based on the target site information (when the target audio system is a master audio system, establish a connection with the target audio system; when the target audio system is a slave audio system, establish a connection with the master audio system cascaded with the target audio system); suppress the playback of the audio signal obtained by the audio system cascaded with the target audio system; generate an audio suppression instruction, and send the audio suppression instruction to the audio system cascaded with the first audio system, so that the audio system cascaded with the first audio system responds to the audio suppression instruction and suppresses the playback of the target audio signal.

[0164] Alternatively, the first audio system establishing a private pairing with the target audio system based on the target site information may include: Acquire sixth site information, and determine eighth site information from the sixth site information; and disconnect the eighth audio system based on the eighth site information.

[0165] A connection is established with the target audio system based on the target site information (when the target audio system is a master audio system, a connection is established with the target audio system; when the target audio system is a slave audio system, a connection is established with the master audio system cascaded to the target audio system).

[0166] Suppresses playback of audio signals acquired by an audio system cascaded with the target audio system.

[0167] An audio suppression instruction is generated and sent to an audio system cascaded with the first audio system, so that the audio system cascaded with the first audio system suppresses the playing of the target audio signal in response to the audio suppression instruction.

[0168] In the embodiment of the present application, in constructing a "private intercom channel" between the first audio system and the target audio system, the specific steps for the target audio system to establish a private pairing with the first audio system based on the first site information are similar to the specific steps for the first audio system to establish a private pairing with the target audio system based on the target site information. The only difference is between the local audio system and the remote audio system, which is not repeated here.

[0169] It should be noted that, in the embodiments of the present application, the audio signal obtained by any audio system refers to the audio signal collected by the audio system itself from the audio environment, and does not include the audio signal collected from the audio environment by the audio system cascaded with the audio system.

[0170] It is understandable that the audio environment in which the first device is located typically has smooth, rigid surfaces, such as tiled floors and stainless steel countertops. These surfaces reflect sound waves in the audio environment, causing the first audio signal acquired by the first audio system to be mixed with background noise such as echoes and reverberations. This background noise may distract the attention of relevant personnel (such as remote doctors) listening to the first audio signal and may mask other sounds or make other sounds unclear. Based on this, in some embodiments, the audio system interconnection method may further include: Perform acoustic echo cancellation (AEC) and noise reduction processing on the first audio signal.

[0171] As mentioned above, the first audio signal is a mixed signal, which can be simply expressed as: , in, The embodiments of this application require the collection and transmission of sounds from specific sound sources, such as human voices, instrument sounds, alarm sounds, etc. is the echo signal, It is environmental noise.

[0172] By performing echo cancellation and noise reduction processing on the first audio signal, that is, Filter out echo signals and ambient noise , get the pure sound of a specific sound source , the embodiment of the present application can make the sound of a specific sound source Clearer.

[0173] In some embodiments, performing echo cancellation on the first audio signal includes: Adaptive filtering is used to dynamically estimate a transfer function of the echo path, and an echo signal is estimated based on the transfer function, and the echo signal is subtracted from the first audio signal.

[0174] Specifically, adaptive filtering: Adaptive filter parameters are calculated using the normalized least mean square algorithm , and estimate the transfer function of the echo path based on the adaptive filter parameters : , in, , is the error signal. is the first audio signal. is the step size factor, ; is a small constant to avoid the denominator being 0.

[0175] The calculated echo signal: , Subtract the echo signal from the first audio signal to obtain the first audio signal after echo removal : , It is understandable that when the first audio system receives the target audio signal from the target audio system and plays it externally, the direct sound of the externally played target audio signal directly re-enters the first audio system (e.g., is collected by the microphone array) and the reverberation that enters the first audio system after being reflected by the audio environment (e.g., is collected by the microphone array) is also the main background noise in the first audio signal. This background noise may form an acoustic feedback closed loop, causing the audio system to howl, which seriously interferes with the operation of the audio system and the execution of remote surgery. Based on this, further, in some embodiments, performing echo cancellation on the first audio signal also includes: In response to receiving a target audio signal from a target audio system, the target audio signal is used as a reference signal to filter out a second audio signal from the first audio signal. The second audio signal is an audio signal obtained by the target audio signal being played outwardly by the first audio system into the audio environment of the first audio system and then re-entering the first audio system.

[0176] The target audio signal is the remote audio signal, and the second audio signal is the audio signal generated when the target audio signal is played back into the audio environment by the first audio system and then re-enters the first audio system (for example, by being collected by a microphone array). This includes the direct sound of the target audio signal directly re-entering the first audio system and the reverberation that enters the first audio system after being reflected by the audio environment. The audio environment of the first audio system refers to the audio environment in which the first device is located.

[0177] Specifically, this embodiment of the present application uses the target audio signal as a reference signal and employs adaptive filtering to dynamically estimate the transfer function of the echo path. The echo signal is estimated based on the transfer function and then subtracted from the first audio signal. The detailed echo estimation and cancellation process is the same as previously described and will not be further elaborated here.

[0178] In some embodiments, performing noise reduction processing on the first audio signal includes: Active noise cancellation (ANC) is performed on the first audio signal.

[0179] Specifically, the embodiment of the present application uses fast Fourier transform to decompose the noise in the first audio signal into frequency domain components, identify the main noise frequency, and dynamically track noise changes based on adaptive filtering, and then delay the noise signal and invert it (180° phase difference) to form an anti-phase sound wave, which is used to destructively interfere with the noise in the first audio signal.

[0180] Furthermore, the embodiment of the present application can adjust the amplitude of the anti-phase sound wave so that the amplitude of the anti-phase sound wave is consistent with the amplitude of the noise, thereby maximizing the cancellation effect.

[0181] It is understandable that the embodiment of the present application can also perform echo cancellation and noise reduction processing on the target audio signal, such as the echo cancellation and noise reduction processing performed on the first audio signal. The specific content is the same as the specific content of the echo cancellation and noise reduction processing performed on the first audio signal, and is not repeated here.

[0182] In some embodiments, the surgical robot further includes a live broadcast platform, which can be used to broadcast the surgical process of the surgical robot on the Internet, such as Figure 11 As shown. Among them, Figure 11 The two smaller split-screen images in the upper left corner are, from top to bottom, a real-time panoramic image of the doctor operating the main console and a real-time panoramic image of the patient-side surgical platform performing surgical operations under the control of the main console. The other larger split-screen image is the surgical environment image acquired in real time by the endoscope of the patient-side surgical platform. The surgical environment image includes surgical instruments in operation and human tissue being operated on by these surgical instruments. The embodiment of the present application is based on this live broadcast platform and can broadcast live the real-time remote surgical process including the above three split-screen images on the Internet, which is helpful for remote surgical operations and surgical robot operation teaching. At the same time, it can also meet the needs of expert collaboration and consultation during remote surgery. Experts in different fields can provide decision-making support during the operation, such as the determination of tumor boundaries. In addition, the live broadcast based on this live broadcast platform helps to promote surgical robot technology and promote the popularization of minimally invasive surgery, and promote the standardization of surgical robot surgical operations, reducing the medical risks caused by differences in surgical operations.

[0183] exist Figure 11In the example, one of the main console and the patient-side surgical platform is configured with a first audio system, and the other is configured with a second audio system, and the live broadcast platform can be privately paired with the first audio system or the second audio system, so that only the audio from the first audio system or the second audio system is played during the live broadcast, and the information pushed by the live broadcast platform is only pushed to the first audio system or the second audio system, thereby meeting the needs of specific scenarios. For example, when only the audio signal of the main console needs to be paid attention to, or when information needs to be pushed to the main console separately for surgical collaboration, a private pairing can be established between the live broadcast platform and the audio system of the main console to achieve private intercom between the live broadcast platform and the audio system of the main console, so that the live broadcast platform can clearly play the audio from the audio system of the main console without being affected by other audio signals, and at the same time enable the audio system of the main console to clearly receive the surgical collaboration instructions from the live broadcast platform. For example, when only the audio signal of the surgical platform on the patient side needs to be paid attention to, or when information needs to be pushed to the surgical platform on the patient side for surgical collaboration, a private pairing can be established between the live broadcast platform and the audio system of the surgical platform on the patient side to realize private intercom between the live broadcast platform and the audio system of the main console, so that the live broadcast platform can clearly play the audio from the audio system of the surgical platform on the patient side without being affected by other audio signals, and at the same time, the audio system of the surgical platform on the patient side can clearly receive the surgical collaboration instruction information from the live broadcast platform, which is also not affected by other audio signals, and the audio system of the main console will not receive the surgical collaboration instruction information, avoiding interference with the surgical operation of the doctor operating the main console, thereby realizing accurate communication of surgical collaboration instructions and improving surgical safety.

[0184] It is understandable that the private pairing process between the live broadcast platform and the first audio system or the second audio system is similar to the aforementioned private pairing process between the first audio system and the second audio system (the target audio system in the second audio system), and will not be repeated here.

[0185] In summary, the surgical robot and its audio system interconnection method proposed in the embodiment of the present application determines the target site information of the target audio system based on the second site information of the target second audio system that can be privately paired, and sends a private pairing request to the target audio system, so that the target audio system generates a response instruction to the private pairing request and establishes a private pairing with the first audio system based on the first site information when the response instruction is an affirmative response instruction. Then, when the affirmative response instruction is received, a private pairing with the target audio system is established based on the target site information, so that both the first audio system and the target audio system establish a private pairing with the other end, thereby realizing private intercom between the first audio system and the target audio system. Therefore, when the first audio system only needs to receive / pay attention to the audio signal of a specific audio system, the embodiment of the present application can select the specific audio system as the target audio system and conduct private intercom, thereby avoiding interference of the target audio signal from the target audio system by other audio signals, realizing the transmission and reception of clear audio signals with the target audio system, and reducing the surgical safety risks caused by the influence of instruction or information transmission.

[0186] An embodiment of the present application further provides a computer-readable storage medium having instructions stored therein. When the instructions are executed on at least one processor, Figure 7The method shown. The storage medium can be volatile memory or non-volatile memory, or can include both volatile and non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface storage, optical disk, or compact disc read-only memory (CD-ROM); magnetic surface storage can be magnetic disk storage or magnetic tape storage. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM). The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0187] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0188] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A surgical robot, characterized in that: The surgical robot includes a server and a first device and a second device capable of communicating with the server, the first device and the second device comprising one of a main console and a patient-side surgical platform operable by the main console, the first device being configured with a first audio system, and the second device being configured with a second audio system; Wherein, the first audio system is configured as follows: acquiring, in real time, second site information of a target second audio system that can be privately paired with the first audio system; In response to receiving target site information of a target audio system determined based on the second site information, sending a private pairing request to the target audio system, the private pairing request including the first site information of the first audio system, the private pairing request being used to request establishment of a private pairing with the target audio system; In response to receiving an affirmative response instruction generated by the target audio system affirmatively responding to the private pairing request, establishing a private pairing with the target audio system based on the target site information; The second audio system is configured as follows: Receiving the private pairing request; Confirm that the response instruction generated to the private pairing request is an affirmative response instruction, establish a private pairing with the first audio system based on the first site information, and send the affirmative response instruction to the first audio system.

2. The surgical robot according to claim 1, wherein: The establishing of a private pairing with the target audio system based on the target site information includes: Based on the target site information, disconnecting from other audio systems except the target audio system; confirming that a connection with the target audio system is not established, and establishing a connection with the target audio system based on the target site information; The establishing of a private pairing with the first audio system based on the first site information includes: Based on the first site information, disconnecting from other audio systems except the first audio system; It is confirmed that no connection is established with the first audio system, and a connection is established with the first audio system based on the first site information.

3. The surgical robot according to claim 1, wherein: The establishing of a private pairing with the target audio system based on the target site information includes: confirming that a connection with the target audio system is not established, and establishing a connection with the target audio system based on the target site information; generating a first private intercom instruction based on the target site information, and sending the first private intercom instruction to the audio system with which the connection is established, so that only the target audio system among the connected audio systems responds to the first private intercom instruction and plays the first audio signal acquired by the first audio system; According to the first private intercom instruction, suppressing the playing of audio signals other than the target audio signal acquired by the target audio system; The establishing of a private pairing with the first audio system based on the first site information includes: confirming that a connection with the first audio system is not established, and establishing a connection with the first audio system based on the first site information; generating a fourth private intercom instruction based on the first site information, and sending the fourth private intercom instruction to the audio systems with which the connection is established, so that only the first audio system among the audio systems with which the connection is established responds to the fourth private intercom instruction and plays the target audio signal; According to the fourth private intercom instruction, playing of audio signals other than the first audio signal is suppressed.

4. The surgical robot according to claim 1, wherein: The establishing of a private pairing with the target audio system based on the target site information includes: confirming that a connection with the target audio system is not established, and establishing a connection with the target audio system based on the target site information; generating a second private intercom instruction based on the target site information; sending the acquired first audio signal to the target audio system according to the second private intercom instruction, and suppressing the playing of audio signals other than the target audio signal acquired by the target audio system; The establishing of a private pairing with the first audio system based on the first site information includes: confirming that a connection with the first audio system is not established, and establishing a connection with the first audio system based on the first site information; generating a fifth private intercom instruction based on the first site information; According to the fifth private intercom instruction, the target audio signal is sent to the first audio system, and the playing of audio signals other than the first audio signal is suppressed.

5. The surgical robot according to claim 1, wherein: The establishing of a private pairing with the target audio system based on the target site information includes: confirming that a connection with the target audio system is not established, and establishing a connection with the target audio system based on the target site information; generating a third private intercom instruction based on the target site information, and sending the third private intercom instruction to the audio systems with which the connection has been established, so that the audio systems other than the target audio system in the established audio systems suppress the playback of the first audio signal acquired by the first audio system in response to the third private intercom instruction; suppressing, according to the third private intercom instruction, playback of audio signals other than the target audio signal acquired by the target audio system; The establishing of a private pairing with the first audio system based on the first site information includes: confirming that a connection with the first audio system is not established, and establishing a connection with the first audio system based on the first site information; generating a sixth private intercom instruction based on the first site information, and sending the sixth private intercom instruction to the audio systems with which the connection is established, so that the audio systems other than the first audio system in the connected audio systems suppress the playback of the target audio signal in response to the sixth private intercom instruction; According to the sixth private intercom instruction, playing of audio signals other than the first audio signal is suppressed.

6. The surgical robot according to claim 1, wherein: The target audio system is configured with a whitelist, wherein the whitelist is associated with at least one first audio system that is allowed to pair with the target audio system, and the affirmative response instruction is automatically generated when the target audio system meets a preset condition.

7. The surgical robot according to any one of claims 1 to 6, wherein: The surgical robot further includes a plurality of master audio systems and slave audio systems cascaded with the master audio systems, the first audio system and the second audio system are the master audio systems or the slave audio systems, and both the master audio system and the slave audio system can be determined as the target audio system; After establishing private pairing between the first audio system and the target audio system: The first audio system only plays the target audio signal acquired by the target audio system, and the target audio system only plays the first audio signal acquired by the first audio system. The audio system cascaded with the first audio system does not play the target audio signal, and the audio system cascaded with the target audio system does not play the first audio signal.

8. The surgical robot according to claim 1, wherein: The first audio system is further configured to: Perform echo cancellation and noise reduction processing on the acquired first audio signal.

9. The surgical robot according to claim 8, wherein: The performing echo cancellation on the acquired first audio signal includes: Adaptive filtering is used to dynamically estimate a transfer function of the echo path, and an echo signal is estimated based on the transfer function, and the echo signal is subtracted from the first audio signal.

10. The surgical robot according to claim 9, wherein: The performing echo cancellation on the acquired first audio signal further includes: In response to receiving a target audio signal from the target audio system, the target audio signal is used as a reference signal to filter out a second audio signal from the first audio signal, where the second audio signal is an audio signal obtained by the target audio signal being played outwardly by the first audio system into the audio environment of the first audio system and then re-entering the first audio system.

11. The surgical robot according to claim 8, wherein: The performing noise reduction processing on the acquired first audio signal includes: Active noise cancellation is performed on the first audio signal.

12. The surgical robot according to any one of claims 1 to 6, wherein: The surgical robot also includes a live broadcast platform, which can be privately paired with the first audio system or the second audio system.

13. A method for interconnecting an audio system of a surgical robot, characterized in that: The method is applied to a first audio system in a surgical robot. The surgical robot includes a server and a first device and a second device capable of communicating with the server. The first device and the second device include one of a main console and a patient-side surgical platform operable by the main console. The first audio system is configured in the first device, and the second device is configured with a second audio system. The method includes: acquiring, in real time, second site information of a target second audio system that can be privately paired with the first audio system; In response to receiving the target site information of the target audio system determined based on the second site information, sending a private pairing request to the target audio system, so that the target audio system generates a response instruction to the private pairing request, and establishes a private pairing with the first audio system based on the first site information when the response instruction is an affirmative response instruction, the private pairing request including the first site information of the first audio system, the private pairing request being used to request establishment of a private pairing with the target audio system, and the affirmative response instruction being generated when the target audio system responds affirmatively to the private pairing request; In response to receiving the affirmative response instruction, a private pairing with the target audio system is established based on the target site information to achieve private intercom with the target audio system.

14. A method for interconnecting an audio system of a surgical robot, characterized in that: The method is applied to a second audio system in a surgical robot. The surgical robot includes a server and a first device and a second device capable of communicating with the server. The first device and the second device include one of a main console and a patient-side surgical platform operable by the main console. The second audio system is configured in the second device, and the first device is configured with the first audio system. The method includes: receiving a private pairing request initiated by the first audio system, the private pairing request including first site information of the first audio system, the private pairing request being used to request private pairing with a target audio system, the target audio system being the second audio system associated with target site information determined by the first audio system based on second site information of a target second audio system that can be privately paired with the first audio system; Confirm that the response instruction generated to the private pairing request is an affirmative response instruction, establish a private pairing with the first audio system based on the first site information, and send the affirmative response instruction to the first audio system, so that the first audio system establishes a private pairing with the target audio system based on the target site information when receiving the affirmative response instruction, so as to achieve private intercom with the first audio system.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on at least one processor, the method according to any one of claims 13 to 14 is implemented.

Images