Data transmission method for a tele-surgical robot and tele-surgical robot

Abstract

This application provides a data transmission method for a remote surgical robot system and a remote surgical robot. The remote surgical robot includes a first main console and a patient-side surgical system, which are connected via a long-distance network. The patient-side surgical system includes surgical instruments for performing surgery and an endoscope for acquiring the surgical environment. The first main console is used to control the movement of the surgical instruments and the endoscope. The data transmission method uses different transmission strategies for transmitting images of regions of interest and non-regions of interest to adapt to changes in the long-distance network.

Description

Technical Field

[0001] This application relates to the field of medical devices, and in particular to a data transmission method for a remote surgical robot and a remote surgical robot. Background Technology

[0002] Minimally invasive surgery refers to a surgical procedure performed inside the human body using modern medical instruments and equipment such as laparoscopes and thoracoscopes. Compared to traditional surgical methods, minimally invasive surgery has advantages such as less trauma, less pain, and faster recovery.

[0003] With advancements in technology, surgical robot technology has matured and become widely used. Compared to traditional minimally invasive surgery, surgical robots offer numerous advantages, including more precise control, a better field of vision, and reduced surgeon fatigue. Current surgical robots typically consist of a main control console and a patient operating platform. The main control console controls the patient operating platform to perform surgery, and both are located in the same room. However, the emergence of long-distance networks such as 5G and Starlink has made remote robotic surgery a reality. But current long-distance networks are still not very stable. How to apply long-distance networks to surgical robots to achieve remote surgery while ensuring surgical safety remains a challenge without a satisfactory solution. Summary of the Invention

[0004] Based on this, to solve the above problems, in one aspect, this application provides a data transmission method for a remote surgical robot. The remote surgical robot includes a first main console and a patient-side surgical system, which are connected via a long-distance network. The patient-side surgical system includes surgical instruments for performing surgery and an endoscope for acquiring the surgical environment. The first main console is used to control the movement of the surgical instruments and the endoscope. The method includes:

[0005] Obtain the region of interest and non-region of interest in the environmental image output by the endoscope;

[0006] The region of interest is encoded at a higher bit rate than the region of non-interest, and the encoded image data is sent to the first main console via the long-distance network.

[0007] In one specific embodiment, obtaining the region of interest and non-region of interest in the environmental image output by the endoscope includes obtaining the human tissue image in the environmental image as the region of interest based on a preset machine learning model, and obtaining the image of the surgical instrument in the environmental image as the non-region of interest.

[0008] In one specific embodiment, the method further includes determining the distance between the human tissue and the surgical instrument based on the position and / or pose of the surgical instrument, determining the human tissue image closest to the surgical instrument as a first region of interest based on the machine learning model, determining human tissue images outside the first region of interest as a second region of interest, and encoding the first region of interest image at a higher bit rate than the second region of interest image.

[0009] In one specific embodiment, the position and / or pose of the surgical instrument in an operating state and a standby state are acquired, and the distance between the surgical instrument and human tissue is determined. Based on the machine learning model, the human tissue image closest to the surgical instrument in the operating state is determined as a first region of interest (ROI), the human tissue image closest to the surgical instrument in the standby state is determined as a second region of interest (ROI), and human tissue images outside the first and second ROIs are determined as a third region of interest (ROI). The first ROI image is encoded at a higher bitrate than the second ROI image, and the second ROI image is encoded at a higher bitrate than the third ROI image.

[0010] In one specific embodiment, the method further includes reconstructing a virtual surgical instrument based on the image data of the non-region of interest and a preset surgical instrument model, wherein the virtual surgical instrument is displayed together with the image of interest.

[0011] In a second aspect, this application also provides an image transmission method for a remote surgical robot system, the method comprising:

[0012] Obtain the region of interest and non-region of interest in the environmental image output by the endoscope;

[0013] The region of interest is sent to the first main console via the long-distance network, but the region of non-interest is not sent.

[0014] In one specific embodiment, the first main console further includes an input device for controlling the movement of the surgical instruments and the endoscope, and the method further includes:

[0015] Based on the position and orientation of the input device, and the mapping relationship between the first main console and the patient-side surgical system, a virtual surgical instrument is reconstructed, and the virtual surgical instrument is displayed together with the region of interest image.

[0016] In one specific embodiment, the virtual surgical instrument has a 1:1 size with the surgical instrument in the environmental image.

[0017] In one specific embodiment, the position and orientation of the end effector of the surgical instrument located within the endoscopic field of view are obtained, and the region of interest is obtained based on the position and / or orientation and a volume model with preset values.

[0018] In one specific embodiment, the type of the volume model can be configured.

[0019] In one specific embodiment, the size of the volume model can be adjusted.

[0020] In one specific embodiment, acquiring the region of interest and non-region of interest in the environmental image output by the endoscope includes acquiring fluorescent and non-fluorescent images in the environmental image, using the fluorescent image as the region of interest and the non-fluorescent image as the region of non-interest.

[0021] In one specific embodiment, the main first console includes an eye tracker, and the method includes: determining the region of interest based on data from the eye tracker transmitted via the long-distance network.

[0022] In one specific embodiment, the remote surgical robot system further includes a second main console, which is connected to the patient surgical system via a local network. When the remote network status is below a preset threshold, control of the patient surgical platform is transferred from the first main console to the second main console.

[0023] In a third aspect, this application also provides a remote surgical robot, which includes a first main console and a patient-side surgical system. The first main console and the patient-side surgical system are connected via a long-distance network. The patient-side surgical system includes a patient surgical platform and a processing device. The patient surgical platform has surgical instruments for performing surgery and an endoscope for acquiring the surgical environment. The first main console is used to control the movement of the surgical instruments and the endoscope. The processing device is configured to:

[0024] Obtain the region of interest and non-region of interest in the environmental image output by the endoscope;

[0025] The region of interest is encoded at a higher bit rate than the region of non-interest, and the encoded image data is sent to the first main console via the long-distance network. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of a remote surgical robot system according to an embodiment of this application;

[0027] Figure 2 This is a schematic diagram of a patient surgical platform according to one embodiment of this application;

[0028] Figure 3 This is a schematic diagram of a surgical instrument according to one embodiment of this application;

[0029] Figure 4A This is a schematic diagram of the main console according to one embodiment of this application;

[0030] Figure 4B This is a schematic diagram of the observation device of the main control console according to an embodiment of this application;

[0031] Figure 5A This is a schematic diagram of an input device according to an embodiment of this application;

[0032] Figure 5B This is a schematic diagram of an input device according to another embodiment of this application;

[0033] Figure 5C This is a schematic diagram of an input device according to another embodiment of this application;

[0034] Figure 6 A flowchart illustrating a method for adjusting the data transmission strategy between a first main control console and a patient-side surgical system based on system delay, according to an embodiment of this application;

[0035] Figure 7A This is a schematic diagram of the region of interest and non-regions of interest in an environmental image acquired by an endoscope according to an embodiment of this application.

[0036] Figures 7B-7C These are schematic diagrams of volume models for two embodiments of this application;

[0037] Figure 8 This is a schematic diagram of a processing apparatus according to an embodiment of this application;

[0038] Figure 9A This is a schematic outline of a surgical instrument according to one embodiment of this application;

[0039] Figure 9B This is a schematic diagram showing the reconstructed virtual surgical instruments and region of interest together according to one embodiment of this application;

[0040] Figure 9C This is a schematic diagram illustrating a virtual surgical instrument reconstruction error according to an embodiment of this application; Detailed Implementation

[0041] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings. Preferred embodiments of this application are shown in the drawings. However, this application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application.

[0042] It should be noted that when an element is referred to as being "set on" another element, it can be directly on the other element or there may be an intervening element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there may be an intervening element. When an element is considered to be "coupled" to another element, it can be directly coupled to the other element or there may be an intervening element. The terms "vertical," "horizontal," "left," "right," "above," "below," and similar expressions used herein are for illustrative purposes only and do not represent the only possible implementations. It should be understood that these spatially related terms are intended to cover different orientations of the device in use or operation, in addition to those depicted in the drawings. For example, if the device is flipped in the drawings, an element or feature described as "below" or "under" other elements or features would be oriented "above" other elements or features. Therefore, the example term "below" can include both above and below orientations.

[0043] The terms "distal" and "proximal" used in this article are directional terms commonly used in the field of interventional medical devices. "Distal" refers to the end furthest from the operator during the procedure, while "proximal" refers to the end closest to the operator. The term "coupling" as used in this article can be broadly understood as two or more objects being connected to any event in a manner that allows absolutely coupled objects to operate together such that there is no relative movement between the objects in at least one direction. For example, the coupling of a protrusion and a groove, where they can move radially relative to each other but not axially.

[0044] The term "instrument" is used herein to describe a medical device for insertion into a patient's body and for performing surgical or diagnostic procedures. This instrument includes an end effector, which can be a surgical tool used to perform surgical procedures, such as an electrocautery device, clamp, stapler, scissor, imaging device (e.g., an endoscope or ultrasound probe), and the like. Some instruments used in embodiments of this application further include a hinged component (e.g., a joint assembly) for the end effector, allowing the position and orientation of the end effector to be manipulated with one or more mechanical degrees of freedom relative to an instrument axis. Further, the end effector includes functional mechanical degrees of freedom, such as opening and closing clamps. The instrument may also include stored information that can be updated by a surgical system, whereby the storage system can provide one-way or two-way communication between the instrument and one or more system components. Some instruments used in some embodiments may also not include a hinged component for the end effector.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The terms “and / or” and “and / or” as used herein include any and all combinations of one or more of the associated listed items.

[0046] One embodiment of the remote surgical robot system of this application is as follows: Figure 1 As shown, the remote surgical robot system includes a patient-side surgical system 10 and a first main control console 20. The first main control console 20 and the patient-side surgical system 10 are connected via a long-distance network 30, which can be a satellite network (e.g., Starlink), a 5G network, the Internet, a private network, etc. The remote surgeon SR operates the first main control console 20 to control the patient-side surgical system 10 to perform surgery on the patient P via the long-distance network 30. The remote surgeon SR can perform surgery on patients thousands or even tens of thousands of kilometers away using the remote surgical robot system. Given my country's vast territory, remote surgical robots have broad application prospects in telemedicine, remote teaching and training, and are of great significance in improving medical efficiency and quality, and alleviating the uneven distribution of medical resources. Remote robotic surgery can connect medical resources around the world, bringing more convenient and efficient medical services to patients globally.

[0047] The patient-side surgical system 10 typically also includes an imaging system that allows the operator to view the surgical site from outside the patient's body. This imaging system typically includes an imaging device (e.g., an endoscope) with video image acquisition capabilities and one or more video display devices for displaying the acquired images. Generally, the imaging device includes optics of one or more imaging sensors (e.g., CCD or CMOS sensors) that acquire images of the patient's body. These one or more imaging sensors may be placed at a distal end of the imaging device, and the signals generated by these sensors may be transmitted via cable or wirelessly for processing and display on the video display devices.

[0048] In some embodiments, the patient-side surgical system 10 includes a patient surgical platform 120, which is connected to a first main console 20 via a long-distance network 30. The patient surgical platform 120 receives instructions sent by the first main console 20 via the long-distance network 30 to perform related actions.

[0049] In some embodiments, such as Figure 1 and Figure 2As shown, the patient surgical platform 120 includes multiple robotic arms 121a, 121b, 121c, and 121d. Multiple surgical instruments 122 and / or imaging devices 126 are detachably mounted on different robotic arms 121a, 121b, 121c, and 121d. One or more cannulas 123 are connected to the distal ends of the holding arms 125 of the robotic arms 121a, 121b, 121c, and 121d. The cannulas 123 are inserted into the body of the patient P lying on the operating table T. Assistant A, depending on the surgical situation, mounts the surgical instruments 122 and imaging devices 126 onto or replaces / reloads the surgical instruments 122 and imaging devices 126 from the robotic arm 121. After the surgical instruments 122 and imaging devices 126 are mounted onto the robotic arm 21, they are inserted into the body of the patient P through the cannulas. The surgeon SR, assistant A, and anesthesiologist B form a basic remote surgical team.

[0050] In some embodiments, the patient surgical platform 120 may have only one robotic arm, with multiple surgical instruments and imaging devices mounted on the same robotic arm and entering the patient P through the same tube. For examples of multiple surgical instruments and imaging devices mounted on the same robotic arm, see the applicant's Chinese patents CN201810649286.4 and CN201811228482.0.

[0051] like Figure 3 As shown, the surgical instrument 122 includes a transmission box 122a, a long shaft 122b, and an end effector 122c. The transmission box 122a contains multiple transmission units. These transmission units are coupled to the drive unit 125 of the holding arms 124 of the robotic arms 121a, 121b, 121c, and 121d and can be driven by the drive unit 125. The drive unit 125 drives the end effector 122c to move via the transmission units according to control commands from the first main control console 20. The transmission units can be multiple flexible cables and a winch. The proximal ends of the cables are wound around the winch, and the distal ends are connected to the end effector. The transmission box 122a drives the winch to rotate, thereby pulling / retracting the flexible cables to control the end effector. The end effector 122c has a joint assembly, which enables it to perform movements with multiple Cartesian degrees of freedom, such as pitch and yaw. The end effector 122c is used to perform surgical procedures. Depending on the needs of the surgical procedure, the end effector 122c can be an electrocautery device, a clamp, a stapler, scissors, a camera, etc.

[0052] One embodiment of this application shows a first main console 20 as follows: Figure 4AAs shown, the first main control console 20 includes a display device 21, an armrest 22, first and second input devices 23L and 23R, an observation device 24, and multiple pedals. The first and second input devices 23L and 23R are used to control surgical instruments 122 or imaging devices 123. The display device 21 is used to display images acquired by the imaging system; for example, the display device 21 is a three-dimensional imaging display device. The surgeon SR observes the images displayed on the display device 21 through the observation device 24. The armrest 22 is used to place the surgeon's arm and / or hand. In some embodiments, the armrest or the observation device 24 may be omitted depending on actual needs, in which case direct observation is possible.

[0053] In one embodiment, such as Figure 4B As shown, the observation device 24 of the first main control console 20 is equipped with eye trackers 24L and 24R. The eye trackers 24L and 24R are used to track the eye movements of the doctor SR, thereby determining the focus of the doctor SR's gaze. For example, the eye trackers 24L and 24R can emit infrared light to the left and right eyes of the doctor SR respectively, and determine the orientation of the eyes based on the infrared light reflected from the received eyes. It is understood that the eye trackers 24L and 24R can also be located in other places within the first main control console 20, such as within the display device 21.

[0054] In one embodiment, such as Figure 5A As shown, the input device 23 includes a handle 111, a clamp 210, multiple L-shaped links 113, 115, 117, and multiple rotary joints 112, 114, 116. The clamp 210 is movably mounted on the handle 111 and can move relative to the handle 111. The handle 111 is rotatably connected to the first L-shaped link 113 via the first rotary joint 112, so that the handle 111 and the clamp 210 can rotate around the rotation axis Y of the rotary joint 112. The first L-shaped link 113 is rotatably connected to the second L-shaped link 115 via the second rotary joint 114, so that the first L-shaped link 113 can rotate around the rotation axis Z1 of the second rotary joint 114. The second L-shaped link 115 is rotatably connected to the third L-shaped link 117 via the third rotary joint 116, so that the second L-shaped link 115 can rotate around the rotation axis X of the third rotary joint 116. The third L-shaped link 117 is rotatably connected to the fourth rotary joint (…). Figure 5A (Not shown) is rotatably connected to the main control console 21, so that the third L-shaped link 117 can rotate about the rotation axis of the fourth rotary joint.

[0055] The surgeon SR controls the movement of tools on the patient surgical platform 120 by operating the first and second input devices 23L and 23R. The control signal processing system of the first master console 20 processes the input signals from the input devices 23 and sends control signals to the patient surgical platform 120 through the long-distance network 30. The patient surgical platform 120 responds to the control signals from the master first console 20 and performs corresponding operations, i.e., master-slave control.

[0056] In some embodiments, the input device may also be other types of input devices, such as Figure 5B Input devices 423L and 423LR are magnetic navigation type. Input devices 423L and 423LR work with a magnetic field generator (not shown in the figure) to move within the magnetic field. The control system of the first main control console 20 can obtain the position and attitude of input device 423 within the magnetic field.

[0057] In some embodiments, such as Figure 5C As shown, the position and posture of the surgeon's left and right hands 201, 202 can be acquired by the camera 424 set on the first main control console 20, and the movement of surgical instruments 221, 222, 223 can be controlled by the position and posture.

[0058] In some embodiments, the input device may be VR glasses or a display device with pitch and / or yaw motion degrees of freedom.

[0059] In some embodiments, the imaging device 126 is an endoscope, and the first main console 20 displays surgical environment images acquired by the endoscope 126 of the patient surgical platform 120. The surgeon SR can observe the three-dimensional stereoscopic images of the patient's body provided by the imaging system through the first main console 20. By observing the three-dimensional images of the patient's body, the surgeon SR can control the patient surgical platform 120 to perform related operations (such as performing surgery or acquiring images of the patient's body) with an immersive feeling.

[0060] The images acquired by the patient surgical platform 120 need to be sent to the first main console 20 via the long-distance network 30. Since the amount of surgical image data is relatively large, and the long-distance network 30 may experience delays or network instability during the operation, the image data transmitted from the patient-side surgical system 10 to the first main console 20 may be lost or distorted, making it impossible for the surgeon SR to fully understand the patient's internal condition during the operation, resulting in safety risks in remote surgery.

[0061] In some embodiments, such as Figure 6As shown, this application also provides a data transmission method 100 based on the delay of a remote surgical robot system. In steps S120-S130, if it is determined that the system delay TDS is greater than a preset first threshold TR1 but less than a second threshold TR2, the long-distance network 30 is determined to be in a first weak network state, and the process proceeds to step S131. In step S131, the remote surgical robot system uses a first transmission strategy to transmit data between the first main console 20 and the patient-side surgical system 10. The data includes image data transmitted from the patient-side surgical system 10 to the first main console 20.

[0062] If the system delay TDS is determined to be less than the preset first threshold TR1 in step S110, the process proceeds to step S121. The remote surgical robot system transmits data between the first main console 20 and the patient-side surgical system 10 using a conventional transmission method. The conventional transmission method may not employ the following special transmission strategy.

[0063] In some embodiments, the patient-side surgical system 10 stores a pre-trained machine learning model, for example, the machine learning model is stored in the storage device of the patient surgical platform. The preset machine learning model is pre-trained to identify human tissue and / or surgical instruments.

[0064] Figure 7A The illustration shows an environmental image 220 of the patient's body during surgery, acquired by an endoscope 126 on the patient surgical platform 120. The environmental image 220 includes human tissues P1, P2, P3 and surgical instruments 221, 222, 223. Generally, the surgeon focuses more on the human tissues than the surgical instruments. In some embodiments, the processing device 40 of the patient-side surgical system 10 (e.g., equipped with a GPU, CPU, etc.) uses a preset machine learning model to identify objects in the environmental image 220, namely, the human tissues P1, P2, P3 and / or the surgical instruments 221, 222, 223, and divides the environmental image 220 into regions of interest Z1 and non-regions of interest Z2. In some embodiments, the human tissues P1, P2, P3 in the environmental image 220 are divided into regions of interest Z1, and the surgical instruments 221, 222, 223 are divided into non-regions of interest Z2. It is understood that the first main control console 20 may also be equipped with a processing device 40.

[0065] In some embodiments, a preset machine learning model can classify specific human tissues or organs, such as blood vessels, kidneys, and fat. The processing device 40 acquires the distances between the surgical instruments 221, 222, and 223 and the human tissues based on their positions and / or postures. Based on the machine learning model, the image of the specific tissue P1 closest to the surgical instruments 221, 222, and 223 is determined as a first region of interest (ROI) Z11, for example, P1 being a kidney undergoing surgery. Images of tissues P2 and P3 other than P1 are determined as a second region of interest (ROI) Z12, for example, other human tissues or organs that do not require immediate surgery. Generally, doctors pay more attention to tissues in the first ROI than those in the second ROI. The distances between the surgical instruments 221, 222, and 223 and the human tissues can be obtained using a three-dimensional volume measurement method. In some embodiments, the distances between the surgical instruments 221, 222, and 223 and the human tissues can also be obtained solely based on their positions using a three-dimensional volume measurement method. However, the orientation of surgical instruments affects the distance between the instruments and human tissue in the direction they are facing. When further subdividing the region of interest, the influence of the surgical instrument orientation on this distance needs to be considered.

[0066] In one embodiment, surgical instruments 221 and 222 are surgical instruments currently being operated by the physician, while surgical instrument 223 is a surgical instrument not currently being operated by the physician and is in a standby state. The processing device determines the image of a specific tissue closest to the operating surgical instruments 221 and 222 as a first region of interest (ROI) Z11, the image of a specific tissue P2 closest to the standby surgical instrument 223 as a second ROI Z12, and the image of tissue P2 outside the first and second ROIs Z11 as a third ROI Z13. It is understood that the physician may be operating one or more surgical instruments, and may be in a standby state with one or more surgical instruments.

[0067] Compared to all human tissues, surgeons often focus more on the tissue in the area where surgical instruments are being used. In some embodiments, the processing device determines regions of interest and non-regions of interest based on a preset volumetric model and the position and / or pose of the surgical instruments, where the position and / or pose of the surgical instruments can be the position and / or pose of the end effector of the surgical instruments. The volumetric model can be one or more. Figure 7BAs shown, the volume model 231 is a cube centered on the end effector of the surgical instrument 221. The image inside the cube is the region of interest Z1, and the image outside the cube is the region of non-interest Z2. In some embodiments, there can be multiple volume models 231, with multiple volume models 231 centered on the end effector of one or two of the surgical instruments 222 and 223.

[0068] In some embodiments, the volume model can be other types, such as spheres, cones, etc. Figure 7C As shown, the cone-shaped volume model 232 is centered on the end effector of the surgical instrument 222, with its apex facing the proximal end of the surgical instrument 222. Since the surgical instrument is often not the focus of the surgeon, the cone-shaped volume model includes more tissue and less of the surgical instrument. In some embodiments, the volume model may not be centered on the end effector of the surgical instrument; for example, the end effector of the surgical instrument may be a vertex of the volume model, or the end effector of the surgical instrument may be an edge of the volume model that contacts the volume model. Considering that the posture of the surgical instrument will affect the position of the volume model associated with the posture of the surgical instrument, the influence of the posture of the surgical instrument on the volume model needs to be considered when further subdividing the region of interest.

[0069] In some embodiments, the type of volumetric model can be configured, for example, by the doctor selecting an appropriate volumetric model based on the surgical situation or the condition of human tissue.

[0070] In some embodiments, the volume of the volume model can be adjusted. For example, when suturing, if the doctor is more focused on the suture site, the doctor can adjust the volume model to cover the volume of the suture site.

[0071] In some embodiments, a pre-defined machine learning model can identify fluorescent images in environmental images, i.e., images of tissue stained with fluorescent dyes, and divide fluorescent images into regions of interest and non-fluorescent images into regions of non-interest.

[0072] In some embodiments, the first main console 20 transmits the eye position data of the doctor acquired by the eye trackers 24L and 24R to the patient-side surgical system 10 via a long-distance network 30. The processing device of the patient-side surgical system 10 determines the regions of interest and non-regions of interest in the environmental image 220 based on the received data from the eye trackers 24L and 24R. For example, the focus of the surgeon SR's gaze is determined by the data from the eye trackers 24L and 24R. The human tissue image in the focus area is the region of interest Z1, and the others are the non-regions of interest Z2.

[0073] In some embodiments, the processing device of the patient-side surgical system determines an image of a specific volume of human tissue as a first region of interest Z11 based on data from eye trackers 24L and 24R, an image of human tissue outside the first region of interest as a second region of interest Z12, and surgical instruments as a region of non-interest Z2.

[0074] In some embodiments, the first transmission strategy is to prioritize the transmission of image data in the region of interest Z1, and randomly discard image data in the region of non-interest Z2.

[0075] In some embodiments, such as Figure 8 The processing unit 40 of the patient-side surgical system 10 shown includes a processor 401, an encoder 402, and a receiving / transmitting unit 403. It is understood that the processor 401, encoder 402, and receiving / transmitting unit 403 can be located within the patient surgical platform 120, or they can be located separately or all on different devices, such as on an electronic equipment cart 13. The electronic equipment cart 13 may include electronic devices such as an energy generator and an image signal processing device. The first transmission strategy involves the encoder 402 encoding the region of interest Z1 and the non-region of interest Z2 using different bit rates. Specifically, the region of interest Z1 is compressed using a higher bit rate than the non-region of interest Z2. This ensures that, even when the long-distance network is in a weak network condition, the image data consumes less bandwidth while maintaining the image quality of the region of interest, thus ensuring surgical safety.

[0076] See again Figure 6 In some embodiments, method 100 further includes step S140. In steps S130-S140, if it is determined that the system delay TDS is greater than the second threshold TR2 but less than the third threshold TR3, the long-distance network 30 is determined to be in a second weak network state. The second weak network state is worse than the first weak network state. At this time, the process proceeds to step S141. In step S141, the remote surgical robot system 10 uses a second transmission strategy to transmit data between the first main console 20 and the patient-side surgical system 10.

[0077] In some embodiments, the second transmission strategy includes encoding the image of the first region of interest Z11 at a higher bit rate than the second region of interest Z12, while the bit rate of the second region of interest Z12 remains higher than the bit rate of the non-region of interest Z2. The determination of the first region of interest Z11 and the second region of interest Z12 can be referred to the embodiments above, and will not be repeated here. Since doctors often pay more attention to the first region of interest than the second region of interest, when the long-distance network is in a weaker second network condition than the first network, the image quality of the first region of interest Z11, which is of greater interest to doctors, can be ensured while reducing the bandwidth occupied by image data.

[0078] In some embodiments, method 100 further includes step S150. In steps S140-S150, if it is determined that the system delay TDS is greater than the third threshold TR3 but less than the fourth threshold TR4, then the long-distance network 30 is determined to be in the third weak network state. The third weak network state is worse than the second weak network state. Then, proceed to step S151. In step S151, the remote surgical robot system 10 uses the third transmission strategy to transmit data between the first main console 20 and the patient-side surgical system 10.

[0079] In the physician embodiment, the third transmission strategy includes encoding the image of the first region of interest Z11 at a higher bit rate than the second and third regions of interest Z12 and Z13, and encoding the second region of interest Z12 at a higher bit rate than the third region of interest Z13. The determination of the first region of interest Z11, the second region of interest Z12, and the third region of interest Z13 can refer to the above embodiment, and will not be repeated here.

[0080] In some embodiments, such as Figure 9A As shown, the third transmission strategy includes the processing device acquiring information on the contours 221a, 222a, 223a of one or more surgical instruments 221, 222, 223 in the non-interest region Z2, and transmitting it to the first main console 20 via the long-distance network 30. The transmission method can be to compress and transmit the data at the same compression rate as the region of interest Z1, or to compress and transmit it at a lower compression rate than the region of interest Z2. Since the non-interest region Z2 only transmits the data of the surgical instrument contours 221a, 222a, 223a, the image data transmitted via the long-distance network occupies less bandwidth and is more suitable for network conditions under weak network conditions. The information on the contours 221a, 222a, 223a can be the feature vectors of the contours of the surgical instruments 221, 222, 223.

[0081] The first main control console 20 receives the outline information 221a, 222a, 223a of the surgical instruments via a long-distance network 30, and reconstructs the virtual surgical instruments based on a preset surgical instrument model, such as... Figure 9BThe reconstructed virtual surgical instruments 221b, 222b, and 223b are displayed together with the region of interest image. By reconstructing the virtual surgical instruments on the first main control console 20 side, the pressure on the long-distance network 30 can be reduced without affecting image quality and surgical results. Since the virtual surgical instruments 221b, 222b, and 223b are reconstructed and displayed based on the contour information 221a, 222a, and 223a of the surgical instruments, the poses of the virtual surgical instruments 221b, 222b, and 223b on the environmental image 220 are the same as the poses of the real surgical instruments 221, 222, and 223 on the environmental image 220. The virtual surgical instruments 221b, 222b, and 223b provide the surgeon (SR) with the feeling of operating the real surgical instruments 221, 222, and 223, ensuring surgical safety while reducing data transmission bandwidth.

[0082] In some embodiments, the virtual surgical instruments 221b, 222b, 223b have a 1:1 size relationship with the surgical instruments 221, 222, 223 in the environmental image.

[0083] In some embodiments, this application also provides a method for obtaining contour 221a, 222a, 223A information of surgical instruments 221, 222, 223, including:

[0084] The contours 221a, 222a, 223a of surgical instruments 221, 222, 223 are obtained based on a preset machine learning model.

[0085] Based on the contours 221a, 222a, 223 of surgical instruments 221, 222, 223, the contour information of the contours 221a, 222a, 223a of surgical instruments 221, 222, 223 is obtained, wherein the contours 221a, 222a, 223a can be the point set of the bounding box of the surgical instruments, and the contour information is the feature vector of the contour.

[0086] Predict the position of surgical instruments 221, 222, 223 in the next frame based on a preset motion model;

[0087] Based on the predicted position of the next frame and the feature vector, the contours 221a, 222a, 223a of the instruments 221, 222, 223 in the next frame are obtained;

[0088] The motion model predicts the position of surgical instruments 221, 222, 223 in the next frame based on the position of surgical instruments 221, 222, 223 in the current frame.

[0089] Because the outline information of surgical instruments 221a, 222a, 223a may be erroneous during long-distance transmission, such as... Figure 9CAs shown, surgical instrument 221 should appear at position A1, but due to an error in the transmission of contour information 221a, the reconstructed virtual surgical instrument 221b appears at position A2. To correct this error, in some embodiments, the first main console 20 pre-sets a kinematic model of the surgical instrument. Based on the contour information 221a, 222a, 223a of the surgical instrument and the pre-set kinematic model, it determines whether the contour information 221a, 222a, 223a has errors, thereby determining the validity of the reconstructed virtual surgical instrument. The kinematic model can be a pre-set model that conforms to the physical motion laws of the surgical instrument. For example, if surgical instrument 221 is located near position A1 in the previous frame, its position in the next frame should also be near position A1 according to the pre-set kinematic model. However, if position A2 is more than the error threshold of the kinematic model than position A1, the contour information at A2 is determined to be incorrect. In this case, if the virtual surgical instrument 221b is reconstructed, the reconstructed virtual surgical instrument 221b will be invalid.

[0090] In one embodiment, this application provides a method for determining whether the contour information 221a, 222a, 223a of surgical instruments received by the first main console 20 is incorrect, including:

[0091] Obtain the outline information of surgical instruments 221a, 222a, and 223a;

[0092] Based on the contour information 221a, 222a, 223a of the surgical instruments and the preset kinematic model, determine whether the contour information 221a, 222a, 223a is incorrect;

[0093] If the contour information 221a, 222a, 223a of the device does not conform to the kinematic model, then the contour information 221a, 222a, 223a is determined to be incorrect.

[0094] In some embodiments, errors in the contour information 221a, 222a, 223a of surgical instruments can be corrected using the pose information of the input devices 23, 423 of the first main console 20. Specifically, the poses of the input devices 23, 423 and the poses of the surgical instruments they control have a mapping relationship. The processor of the first main console 20 acquires the poses of the input devices 23, 423, determines the poses of the surgical instruments 221, 222, 223 using the contour information 221a, 222a, 223a, compares the poses of the input devices 23, 423 at the same control moment with the poses of the corresponding surgical instruments 221, 222, 223, and if the deviation is greater than a preset threshold, it is determined that the contour information 221a, 222a, 223a transmitted through the long-distance network 30 has an error. In some embodiments, the aforementioned same control moment can be determined by the timestamp of the first main console 20 issuing the control command to the patient surgical platform 120 and the timestamp of the patient surgical platform 120 receiving or executing the control command.

[0095] In one embodiment, this application provides a method for determining whether the image information of surgical instruments 221, 222, 223 received by the first main console 20 is erroneous, including:

[0096] Acquire image information of surgical instruments 221, 222, and 223;

[0097] The actual position and / or orientation of surgical instruments 221, 222, and 223 are obtained based on the image information of surgical instruments 221, 222, and 223.

[0098] Acquire the position and / or attitude information of input devices 23,423;

[0099] Based on the position and / or posture information of the input devices 23, 423 and the preset mapping relationship between the first main console 20 and the patient surgical platform 120, the position and / or posture information of the input devices 23, 423 is converted into the expected position and / or posture of the surgical instruments.

[0100] If the actual position is compared with the expected position, or the actual posture is compared with the expected posture, and the comparison result is greater than a preset threshold, it is determined that the surgical instrument 221, 222, 223 image information received by the first main console 20 has errors.

[0101] In one embodiment, this application provides a method for determining whether the image information of surgical instruments 221, 222, 223 received by the first main console 20 is erroneous, including:

[0102] Acquire image information of surgical instruments 221, 222, and 223;

[0103] The actual position and / or orientation of surgical instruments 221, 222, and 223 are obtained based on the image information of surgical instruments 221, 222, and 223.

[0104] Based on the actual position and / or posture of the surgical instruments 221, 222, 223, and the preset mapping relationship between the first main console 20 and the patient surgical platform 120, the actual position and / or posture of the surgical instruments 221, 222, 223 are converted into the expected position and / or posture of the input device.

[0105] Obtain the actual pose information of input device 23,423;

[0106] If the actual position is compared with the expected position, or the actual posture is compared with the expected posture, and the comparison result is greater than a preset threshold, it is determined that the surgical instrument 221, 222, 223 image information received by the first main console 20 has errors.

[0107] It can be understood that the image information of the surgical instruments 221, 222, 223 can be either the complete image information of the surgical instruments 221, 222, 223, or the outline information 221a, 222a, 223a of the surgical instruments 221, 222, 223.

[0108] It is understandable that the pose information of input devices 23 and 423 includes Figure 5C In the embodiment shown, the pose information of the surgeon's left and right hands 201, 202 is acquired by camera 424.

[0109] In some embodiments, in order to correct the errors in the transmission of surgical instrument image information, when an error is detected in the surgical instrument image information by the above method, the erroneous image information is discarded, and the discarded image information is filled with the correct image information from the previous or next frame, so that the reconstructed virtual surgical instrument conforms to kinematic constraints.

[0110] In some embodiments, method 100 further includes step S160. In steps S150-S160, if it is determined that the system delay TDS is greater than a preset fourth threshold TR4 but less than a fifth threshold TR5, then the long-distance network 30 is determined to be in a fourth weak network state. The fourth weak network state is worse than the third weak network state. At this time, the process proceeds to step S161. In step S161, the remote surgical robot system 10 uses a fourth transmission strategy to transmit data between the first main console 20 and the patient-side surgical system 10.

[0111] In some embodiments, the fourth transmission strategy includes the patient surgical platform 120 sending image data of the region of interest but not sending image data of the non-region of interest. The patient surgical platform 120 sends motion data of robotic arms 121a, 121b, 121c, 121d to the first main console. The motion data includes joint rotation information of robotic arms 121a, 121b, 121c, 121d and rotation information of the drive motor in the drive device 125. The first main console 20 reconstructs virtual surgical instruments 221b, 222b, 223b based on the motion data through forward kinematics calculations and a preset surgical instrument model. Since the amount of motion data is much less than that of image data, the fourth transmission strategy can occupy less bandwidth.

[0112] In some embodiments, method 100 further includes step S170. In steps S160-S170, if it is determined that the system delay TDS is greater than the preset fifth threshold TR5 but less than the sixth threshold TR6, the remote network 30 is determined to be a fifth weak network state that is worse than the fourth network state, and the process proceeds to step S171. In step S171, the remote surgical robot system 10 uses the fifth transmission strategy to transmit data between the first main console 20 and the patient-side surgical system 10.

[0113] In some embodiments, the fifth transmission strategy includes the patient surgical platform 120 transmitting only image data of the region of interest Z1, and not transmitting image data of the non-region of interest Z2. It also excludes the transmission of the contour information 221a, 222a, 223a of the surgical instruments, as well as the motion data of the robotic arms 121a, 121b, 121c, 121d and the surgical instrument drive device 125 transmitted by the patient surgical platform 120. At this time, the first main console 20 calculates the expected positions and orientations of the surgical instruments 221, 222, 223 based on the pose information of the input device 23, and reconstructs virtual surgical instruments 221b, 222b, 223b based on a preset surgical instrument model. For example, when the input device is a linkage-type input device as shown in Figures 4 and 5, the position and orientation of the input device's end can be obtained through the joint information of the linkage, and the expected positions and orientations of the surgical instruments 221, 222, 223 can be determined based on the mapping relationship between the first main console 20 and the patient surgical platform 120. Since the fifth transmission strategy only transmits image data of the region of interest Z1 from the patient surgical platform 120 to the first main console 20, the fifth transmission strategy will consume less bandwidth compared to the fourth transmission strategy.

[0114] In some embodiments, the surgical instrument model may be optionally configured to be in a transparent, semi-transparent, or highlighted mode.

[0115] Understandably, input devices can also be Figure 5BThe magnetic navigation input device 423 can also input the position and attitude information of the device. Figure 5C The embodiment shown is acquired by camera 424, and will not be described in detail here.

[0116] In some embodiments, the surgical instrument type is also obtained according to the machine learning model. For example, the instrument type includes electric hooks, staplers, needle holders, etc. Based on the obtained surgical instrument type information, the first main console 20 selects the corresponding surgical instrument model and reconstructs the corresponding virtual surgical instrument.

[0117] In some embodiments, the drive unit 125 of the patient surgical platform 120 reads the type of surgical instruments mounted on it. The type information of the surgical instruments obtained by the patient surgical platform 120 is sent to the first main console 20. Based on the obtained surgical instrument type information, the first main console 20 selects the corresponding surgical instrument model and reconstructs the corresponding virtual surgical instrument. Compared to obtaining the surgical instrument type through a machine learning model, sending the type of the surgical instruments read by the drive unit 125 to the first main console 20 consumes less bandwidth, making it suitable for use when the condition of the long-distance network 30 is worse.

[0118] In some embodiments, the first main console 20 acquires the position of the endoscope 126 and reconstructs virtual surgical instruments 221b, 222b, 223b based on the contour information 221a, 222a, 223a of the surgical instruments, the position of the endoscope 126 and the surgical instrument model, or based on the pose of the input device 23, the position of the endoscope 126 and the surgical instrument model. During the reconstruction of virtual surgical instruments 221b, 222b, and 223b, the position of the endoscope 126 is taken into account. This allows the reconstructed virtual surgical instruments 221b, 222b, and 223b to conform to the size of the environmental image. For example, when the endoscope 126 and the surgical instruments 221, 222, and 223 are relatively close, the size of the virtual surgical instruments 221b, 222b, and 223b becomes larger; when the endoscope 126 and the surgical instruments 221, 222, and 223 are relatively far apart, the size of the virtual surgical instruments 221b, 222b, and 223b becomes smaller, making the reconstructed virtual surgical instruments 221b, 222b, and 223b more realistic.

[0119] In some embodiments, the position of the endoscope 126 may be derived from joint space information sent by the patient surgical platform 120. The patient surgical platform 120 sends the position information of the drive device 125 of the robotic arm 121b on which the endoscope 126 is mounted to the first main console 20. The first main console 20 determines the position of the endoscope 126 based on the position information of the drive device 125. The position information of the drive device 125 may be information on the feed motion of the drive device 125 along the holding arm 124 between the proximal and distal ends.

[0120] In some embodiments, the patient surgical platform 120 also sends the joint information of the robotic arm 121b, on which the endoscope 126 is mounted, to the first main console 20. The first main console 20 determines the position of the endoscope 126 based on the position information of the drive device 124 and the joint information of the robotic arm 121b. Since the position and orientation of the endoscope 126 change simultaneously when the joints of the robotic arm 121b control the rotation of the endoscope 126 around the remote motion center, although the change in the position of the endoscope 126 does not significantly affect the depth of field, considering this positional change when reconstructing the virtual surgical instruments 221b, 222b, and 223b can provide the doctor with a more realistic experience.

[0121] In some embodiments, when the remote network 30 is in poor condition, such as a fourth or fifth weak network, the patient surgical platform 120 no longer sends the position and orientation information of the endoscope 126 to the first main console 20 through the remote network 30. The first main console 20 obtains the position and orientation of the input device 23 in the endoscope operation mode and determines the position and orientation of the endoscope 126 based on the position and orientation of the input device 23. This reduces the bandwidth usage of data sent from the patient surgical platform 120 to the first main console 120. The endoscope operation mode can be referenced in Chinese Patent CN202311146422.5 filed by the applicant.

[0122] In some embodiments, method 100 further includes step S180. In steps S170-S180, if it is determined that the system delay TDS is greater than the sixth threshold TR6 but less than the seventh threshold TR7, then the long-distance network 30 is determined to be in a sixth weak network state, which is worse than the fifth weak network state, and proceeds to step S181. Step S181 disables the excitation function of the energy device from the first main console. This is because the system delay is relatively large, and if the energy excitation function is delayed, it may cause damage to other tissues. Disabling the excitation function of the energy device can mean that the patient-side surgical system 10 no longer receives the excitation command sent by the first main console 20.

[0123] In some embodiments, see again Figure 1 The patient-side surgical system also includes a second main control console 11, which is connected to the patient surgical platform 120 via a local network. This local network differs from the aforementioned long-distance network 30, ensuring that the physical distance between the second main control console 11 and the patient surgical platform 120 cannot be too great; for example, they must be located within the same building. The local network can be an Ethernet, Wi-Fi, Bluetooth, or other network connection. The electronic device cart 13 is also connected to the second main control console 11 and the patient surgical platform 120 via a local network.

[0124] In some embodiments, the environmental image 220 from the endoscope 126 of the patient surgical platform 120 is sent directly to the second main console 11 via the local network, that is, the environmental image 220 is sent directly without going through the region of interest and non-region of interest processing.

[0125] In some embodiments, in step S180, if the system delay TDS is determined to be greater than the seventh threshold TR7, then the remote network 30 is determined to be in the seventh weak network state, which is worse than the sixth weak network state. At this time, the process proceeds to step S182. In step S182, the remote surgical robot system transfers the control of the patient surgical platform 120 and the electronic trolley 13 from the first main console 20 to the local second main console 11. At this time, there would be a significant risk if surgery were performed on the remote network 30.

[0126] It is understandable that there are other methods to determine the network status of the long-distance network 30. For example, the first main console 20 can send a delay flag to the patient surgical platform 120 and receive feedback signals from the patient surgical platform 120 regarding the delay flag. Based on this, the latency of the long-distance network 30 can be determined. Regardless of the method used to determine the latency of the long-distance network 30, the corresponding strategies described above can be adopted, such as the first to fifth data transmission strategies, and the second main console 11 taking over the control strategy of the first main console 20.

[0127] In some embodiments, step S110 may involve determining the network latency of the remote surgical robot system and adjusting the transmission strategy by comparing the network latency with first to seventh thresholds.

[0128] One embodiment of this application also provides a computer-readable storage medium storing instructions that, when executed on at least one processor, implement... Figure 6The method is illustrated. The storage medium can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM). The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A data transmission method for a remote surgical robot, characterized in that, The remote surgical robot includes a first main console and a patient-side surgical system, which are connected via a long-distance network. The patient-side surgical system includes surgical instruments for performing surgery and an endoscope for acquiring the surgical environment. The first main console is used to control the movement of the surgical instruments and the endoscope. The method includes: Obtain the region of interest and non-region of interest in the environmental image output by the endoscope; The region of interest is encoded at a higher bit rate than the region of non-interest, and the encoded image data is sent to the first main console via the long-distance network.

2. The method as described in claim 1, characterized in that, The step of acquiring regions of interest and non-regions of interest in the environmental image output by the endoscope includes acquiring human tissue images in the environmental image as regions of interest based on a preset machine learning model, and acquiring images of surgical instruments in the environmental image as non-regions of interest.

3. The method as described in claim 2, characterized in that, It also includes determining the distance between human tissue and the surgical instrument based on the position and / or pose of the surgical instrument, determining the human tissue image closest to the surgical instrument as a first region of interest based on the machine learning model, determining human tissue images outside the first region of interest as a second region of interest, and encoding the first region of interest image at a higher bit rate than the second region of interest image.

4. The method as described in claim 2, characterized in that, The system acquires the position and / or pose of the surgical instrument in both operational and standby states and determines the distance between the surgical instrument and human tissue. Based on the machine learning model, it identifies the human tissue image closest to the surgical instrument in the operational state as a first region of interest (ROI), the human tissue image closest to the surgical instrument in the standby state as a second ROI, and human tissue images outside the first and second ROIs as a third ROI. The first ROI image is encoded at a higher bitrate than the second ROI image, and the second ROI image is encoded at a higher bitrate than the third ROI image.

5. The method as described in claim 1, characterized in that, It also includes reconstructing virtual surgical instruments based on the image data of the non-region of interest and a preset surgical instrument model, and the virtual surgical instruments are displayed together with the image of interest.

6. A data transmission method for a remote surgical robot, characterized in that, The remote surgical robot includes a first main console and a patient-side surgical system, which are connected via a long-distance network. The patient-side surgical system includes surgical instruments for performing surgery and an endoscope for acquiring the surgical environment. The first main console is used to control the movement of the surgical instruments and the endoscope. The method includes: Obtain the region of interest and non-region of interest in the environmental image output by the endoscope; The region of interest is sent to the first main console via the long-distance network, but the region of non-interest is not sent.

7. The method as described in claim 6, characterized in that, The first main console further includes an input device for controlling the movement of the surgical instruments and the endoscope, and the method further includes: Based on the position and orientation of the input device, and the mapping relationship between the first main console and the patient-side surgical system, a virtual surgical instrument is reconstructed, and the virtual surgical instrument is displayed together with the region of interest image.

8. The method as described in claim 5 or 7, characterized in that, The virtual surgical instruments have a 1:1 size with the surgical instruments in the environmental image.

9. The method as described in claim 1 or 6, characterized in that, The position and orientation of the end effector of the surgical instrument located within the endoscopic field of view are obtained, and the region of interest is obtained based on the position and / or orientation and a volume model with preset values.

10. The method as described in claim 9, characterized in that, The type of the volume model can be configured.

11. The method as described in claim 9, characterized in that, The size of the volumetric model can be adjusted.

12. The method as described in claim 1 or 6, characterized in that, The step of acquiring the region of interest and non-region of interest in the environmental image output by the endoscope includes acquiring fluorescent and non-fluorescent images in the environmental image, using the fluorescent image as the region of interest and the non-fluorescent image as the region of non-interest.

13. The method as described in claim 1 or 6, characterized in that, The main first console includes an eye tracker, and the method includes: determining the region of interest based on data from the eye tracker transmitted via the long-distance network.

14. The method as described in claim 1 or 6, characterized in that, The remote surgical robot also includes a second main console, which is connected to the patient's surgical system via a local network. When the remote network status is below a preset threshold, control of the patient's surgical platform is transferred from the first main console to the second main console.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on at least one processor, implement the method as described in any one of claims 1 to 14.

16. A remote surgical robot, characterized in that, The remote surgical robot includes a first main console and a patient-side surgical system, which are connected via a long-distance network. The patient-side surgical system includes a patient surgical platform and a processing device. The patient surgical platform has surgical instruments for performing surgery and an endoscope for acquiring the surgical environment. The first main console is used to control the movement of the surgical instruments and the endoscope. The processing device is configured to: Obtain the region of interest and non-region of interest in the environmental image output by the endoscope; The region of interest is encoded at a higher bit rate than the region of non-interest, and the encoded image data is sent to the first main console via the long-distance network.

Images