Remote operation surgical system with surgical instrument wear tracking

Abstract

To solve problems with the prior art.SOLUTION: A medical method is provided. The medical method comprises: providing an information formation structure in a computer readable storage device that associates an indication of a surgeon skill level in at least one surgical activity performed using the surgical instrument with a surgical instrument safety state of the surgical instrument for use during performance of the at least one surgical activity using the surgical instrument by a surgeon having the indicated skill level; tracking a surgical instrument actuator state of a surgical instrument during performance of a surgical procedure by a surgeon; and transitioning the surgical instrument actuator state of the surgical instrument to the surgical instrument safety state during performance of the at least one surgical activity by the surgeon using the surgical instrument.SELECTED DRAWING: Figure 13

Description

Technical Field

[0001] (Reference to Related Applications) This application claims the benefit of priority of U.S. Patent Application No. 62 / 421,083, filed on Nov. 11, 2016, the entire content of which is incorporated herein by reference.

[0002] (Copyright Notice) Part of the disclosure of this patent document contains materials subject to copyright protection. The copyright owner reserves all copyrights except that it does not object to the facsimile reproduction by any one of the patent documents or patent disclosures as they appear in the patent file or records of the Patent and Trademark Office.

[0003] The inventive aspects relate to medical devices used during surgery. More specifically, those aspects relate to surgical instrument wear tracking.

Background Art

[0004] Surgeons typically conduct extensive research before performing a surgical procedure. Conventionally, surgeons have been restricted to studying general anatomical models such as photographs or drawings. More recently, various preoperative diagnostic procedures (e.g., X-rays, CT, MRI, etc.) have made patient-specific anatomical information available.

[0005] In some cases, it may be desirable to make additional relevant anatomical and surgical procedure information available to the surgeon. In one aspect, it is desirable to provide the surgeon planning an operation on a particular patient with a video record of the surgical site of previous surgical procedures performed on that particular patient. In another aspect, it is desirable to provide the surgeon with one or more (one or more) surgical video records of surgical procedures on other patients similar to the surgical procedure planned for a particular patient. In one aspect, it is desirable to provide such information to the surgeon before performing a particular surgical procedure. And in another aspect, it may be desirable to provide this information to the surgeon during the operation.

[0006] In one aspect, it is desirable to construct a video database containing intraoperative surgical site video records of various procedures received by various patients. In one aspect, it is desirable to configure a medical device capable of performing video recording to further include an input that enables a surgeon using the medical device to highlight (highlight) and annotate the video record in real time when the video is being recorded. In one aspect, it is desirable to configure a computer-based pattern matching algorithm to search individual records in the video database, identify relevant video records, and provide this relevant information to the surgeon for a particular surgical procedure. SUMMARY OF THE INVENTION

[0007] The following summary introduces specific aspects of the inventive subject matter to provide a basic understanding. This summary is not an extensive overview of the inventive subject matter and is not intended to identify key or critical elements or to delineate the scope of the inventive subject matter. This summary includes information related to various aspects and embodiments of the inventive subject matter, but its sole purpose is to present some aspects and embodiments in a general form as a prelude to the more detailed description below.

[0008] A surgical method for use with a robotic surgery system including a surgical instrument and a surgical instrument actuator. A first information structure associating a display of the remaining useful life of a surgical instrument identified by a surgical instrument identifier is provided in a computer-readable storage device. A second information structure associating a surgical instrument wear operating state with a surgical instrument life reduction amount is provided in a computer-readable computer-readable storage device. The surgical instrument actuator state of the identified surgical instrument is tracked during the performance of a surgical procedure. The display in the first information structure of the remaining useful life of the identified surgical instrument is decreased by the surgical instrument life reduction amount in the second information structure in response to the tracked surgical instrument actuator state matching the surgical instrument wear operating state during the performance of the surgical procedure.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6A

Figure 6B

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12A

Figure 12B

Figure 12C

Figure 13

Figure 14

Figure 15

[0010] This description, and the accompanying drawings that illustrate inventive aspects, embodiments, implementations, or applications, should not be considered limiting. That is, the claims define the protected invention. Various mechanical, compositional, structural, electrical, and operational changes may be made without departing from this description and the claims. In some cases, well-known circuits, structures, or techniques are not shown or described in detail so as not to obscure the present invention. Equivalent numerals in two or more (two or more) figures represent the same or similar elements.

[0011] Elements described in detail with reference to one embodiment, implementation, or application may, whenever practical, be included in other embodiments, implementations, or applications in which they are not specifically shown or described. For example, if an element is described in detail with reference to one embodiment and not described with reference to a second embodiment, that element may nonetheless be claimed as being included in the second embodiment. Thus, to avoid unnecessary repetition in the following description, one or more elements shown and described in connection with one embodiment, implementation, or application may be incorporated into other embodiments, implementations, or aspects, provided there is no specific description to the contrary, provided the embodiments or implementations in which there are one or more elements are not rendered inoperative, or provided two or more of those elements do not provide conflicting functions.

[0012] Aspects of the present invention are described primarily from the perspective of implementations using the da Vinci® Surgical System, commercially available from Intuitive Surgical, Inc., Sunnyvale, California, and in particular, the da Vinci Xi TM HD TM Surgical System, Model IS4000, which is commercially available. However, those skilled in the art will understand that the inventive aspects disclosed herein may be embodied and implemented in a variety of ways, including robotic embodiments and implementations, and, where applicable, non-robotic embodiments and implementations. Implementations related to the da Vinci® Surgical System (e.g., Model IS4000 da Vinci® Xi TM Surgical System, Model IS3000 da Vinci® Surgical System) are for illustration only and should not be considered as limiting the scope of the inventive aspects disclosed herein.

[0013] In various aspects, the present disclosure describes a surgical planning tool that includes a medical device configured to video record the performance of a surgical procedure. The video recording can be embedded with various metadata, such as highlights created by medical personnel. Additionally, the video recording can be tagged with text annotations that describe various metadata, such as a particular subject of the video, identification of the patient to whom the video recording corresponds, historical or medical information about the patient, and the like. In one aspect, the tagged metadata is embedded in the video recording.

[0014] According to a further aspect, the present disclosure describes a teleoperated medical device that includes a surgical instrument used to perform at least one surgical activity during a surgical procedure. The surgical instrument typically has a limited useful life that can be reliably used to perform the surgical activity. In some embodiments, the lifespan is indicated as a count of the number of surgeries for which the surgical instrument is permitted to be used. For example, an instrument with a lifespan count of 10 is permitted to be used in 10 additional surgeries. Conversely, for example, an instrument with a lifespan count of 5 is permitted to be used in 5 additional surgeries.

[0015] Different surgical instruments are used to perform different surgical activities. For example, a scalpel is used for incisions, a needle is used for suturing, and a heat source is used for cauterization. The use of surgical instruments to perform those surgical activities during a surgical procedure subjects the surgical instruments to wear. The wear of the surgical instruments accumulates such that over the course of several surgical procedures, the usefulness of the surgical instruments for their activity is reduced to a point where they are no longer reliable. For example, wear on a scalpel can include dulling of the cutting edge of the scalpel. Surgical instruments that have worn to a point where they are no longer reliable are typically discarded or, if feasible, refurbished. According to some embodiments, a record of the remaining useful life of a surgical instrument is maintained. The incremental wear imparted to a surgical instrument during a surgical procedure is tracked. The record of the remaining useful life of the surgical instrument is updated based on the incremental wear imparted during the surgical procedure.

[0016] In a robotic surgery system, different instruments may be used at different stages of a surgical procedure. Moreover, the same instrument may be used in different actuator states at different stages of a surgical procedure. As used herein, the term "actuator state" means the mechanical disposition of a surgical instrument as determined by an actuator, such as a motor, in response to an input command received from a surgeon or other surgical team member.

[0017] A video record and information structure associating surgical instrument actuator states with surgical guidance or actuator safety status information can be archived on an electronic medical record database implemented locally or on a cloud data storage service. The video record can be made available to interested healthcare providers. The information structure can be made available for use with a robotic medical device to provide surgical guidance and to control the surgical instrument actuator state during the performance of at least one surgical activity during a surgical procedure.

[0018] Healthcare providers can use the metadata tags described above to search a medical device database based on the surgeon skill level for videos of interest and information structure relationships. Additionally, in one aspect, the surgical planning tool includes computer-based pattern matching and analysis algorithms. In one aspect, the pattern matching algorithm culls through videos stored in an electronic medical record database and identifies correlations between visual characteristics in the video recordings and associated metadata tags created by medical personnel. The surgical planning tool can assist medical personnel performing a procedure in making decisions about the patient's anatomical structure, suitable surgical approaches, pathologies, potential complications, etc. by applying these correlations to the anatomical structures newly faced.

[0019] In another aspect, the pattern matching algorithm culls through videos stored in an electronic medical record database, identifies correlations between visual characteristics in the video recordings, and identifies surgical activities that contribute to instrument degradation. Since some routine surgical activities result in a predictable rate of instrument degradation, instruments can be specified for effectiveness and safety as being suitable for a certain number of surgical uses, called "lifespan". For example, an instrument having a lifespan of "x" for its intended use is suitable for its intended use in "x" surgeries before it must be refurbished or discarded. Specific irregular surgical uses of an instrument can accelerate its degradation. The surgical use of individual instruments is tracked. The surgical planning tool can apply the correlation between surgical activities and the instrument wear rate for individual surgical instruments for inventory planning purposes to determine when to replace or refurbish individual surgical instruments based on their individual remaining lifespan.

[0020] (Minimally Invasive Remote Operating Surgery System) Referring now to the drawings, like reference numerals represent like parts throughout the several views, and FIG. 1 is a plan view of a minimally invasive remote surgical system 10 typically used to perform minimally invasive diagnostic or surgical procedures on a patient 12 lying on an operating table 14. The system includes a surgeon console 16 for use by a surgeon 18 during the procedure. One or more (one or more) assistants 20 may participate in the procedure. The minimally invasive remote surgical system 10 further includes a patient-side cart 22 and an electronics cart 24. The patient-side cart 22 can operate at least one removably coupled surgical instrument 26 through a minimally invasive incision in the body of the patient 12 while the surgeon 18 views the surgical site through the surgeon console 16. An image of the surgical site can be obtained by an endoscope 28, such as a stereoscopic endoscope, and the patient-side cart 22 can operate the endoscope 28 to orient the endoscope 28. A computer processor disposed in the electronics cart 24 can be used to process an image of the surgical site for subsequent display to the surgeon 18 through the surgeon console 16. The number of surgical instruments 26 used at one time generally depends on, among other factors, the diagnostic or surgical procedure and the space constraints within the operating room. If it is necessary to change one or more of the surgical instruments 26 used during the procedure, the assistant 20 can remove the surgical instrument 26 from the patient-side cart 22 and replace it with another surgical instrument 26 from a tray 30 in the operating room.

[0021] Figure 2 is a perspective view of the surgeon console 16. The surgeon console 16 includes a left-eye display 32 and a right-eye display 34 that present the surgeon 18 with a coordinated stereoscopic view of the surgical site that enables depth perception. The console 16 further includes one or more control inputs 36. One or more surgical instruments that are installed for use with the patient-side cart 22 (shown in FIG. 1) move in response to the operation of the one or more control inputs 36 of the surgeon 18. The control inputs 36 provide the same mechanical degrees of freedom as their associated surgical instruments 26 (shown in FIG. 1) and can provide the surgeon 18 with a perception that the control inputs 36 are integral with the instruments 26 such that the surgeon has a sense of telepresence, or a strong sense that the surgeon is directly controlling the surgical instruments 26. To achieve this purpose, position, force, and tactile feedback sensors (not shown) may be utilized to convey sensations of position, force, and touch from the surgical instruments 26 to the surgeon's hands through the control inputs 36.

[0022] The surgeon console 16 is typically located in the same room as the patient so that the surgeon can directly monitor the procedure, be physically present if necessary, and talk directly to the patient-side assistant rather than through a telephone or other communication medium. However, the surgeon can be located in a different room, a completely different building, or some other location remote from the patient that enables remote surgery.

[0023] FIG. 3 is a perspective view of the electronic equipment cart 24. The electronic equipment cart 24 can be connected to the endoscope 28 and includes a computer processor that processes captured images for subsequent display to the surgeon on the surgeon console or on other suitable displays located locally or remotely. For example, if a stereoscopic endoscope is used, the computer processor on the electronic equipment cart 24 can process the captured images to present the surgeon with a calibrated stereoscopic image of the surgical site. Such coordination can include alignment between opposing images and can include adjusting the stereoscopic working distance of the stereoscopic endoscope. As another example, the image processing can include compensating for imaging errors of the image capture device, such as optical aberrations, using previously determined camera calibration parameters. Optionally, the equipment within the electronic equipment cart can be integrated into the surgeon console or the patient-side cart, or it can be distributed to various other locations within the operating room.

[0024] FIG. 4 schematically illustrates a teleoperation surgical system 50 (such as the minimally invasive teleoperation surgical system 10 of FIG. 1). A surgeon console 52 (such as the surgeon console 16 of FIG. 1) can be used by a surgeon to control a patient cart 54 (such as the patient cart 22 of FIG. 1) during a minimally invasive procedure. The patient cart 54 can use an imaging device such as a stereoscopic endoscope to capture an image of the surgical site and output the captured image to a computer processor disposed in an electronic device cart 56 (such as the electronic device cart 24 of FIG. 1). The computer processor typically includes one or more data processing boards for the purpose of executing computer-readable code stored in a non-volatile memory device of the computer processor. In one aspect, the computer processor can process the captured image in various ways prior to any subsequent display. For example, the computer processor can overlay the captured image with a virtual control interface prior to displaying the combined image to the surgeon via the surgeon console 52.

[0025] Additionally or alternatively, the captured image can undergo image processing by a computer processor disposed outside the electronic equipment cart 56. In one aspect, the remote surgery system 50 includes an optional computer processor 58 (as indicated by the dashed line) similar to the computer processor disposed in the electronic equipment cart 56, and the patient-side cart 54 outputs the captured image taken for image processing to the computer processor 58 prior to display on the surgeon console 52. In another aspect, the captured image first undergoes image processing by the computer processor in the electronic equipment cart 56 and then additional image processing by the computer processor 58 prior to being displayed on the surgeon console 52. The remote surgery system 50 can include an optional display 60, as indicated by the dashed line. The display 60 is connected to the computer processor disposed in the electronic equipment cart 56 and the computer processor 58, and the captured images processed by these computer processors can be displayed on the display 60 in addition to being displayed on the display of the surgeon console 52.

[0026] FIG. 5 is a perspective view of a patient-side cart 500 of a minimally invasive remote surgery system according to an embodiment of the present invention. The patient-side cart 500 includes one or more support assemblies 510. A surgical instrument manipulator 512 is attached to the end of each support assembly 510. Additionally, each support assembly 510 can optionally include one or more non-powered lockable setup joints used to position the surgical instrument manipulator 512 attached with reference to the patient for surgery. As depicted, the patient-side cart 500 is positioned on the floor. In other embodiments, the operating portion of the patient-side cart can be attached to a wall, ceiling, operating table 526 that also supports the patient's body 522, or other operating room equipment. Further, although the patient-side cart 500 is shown as including four surgical instrument manipulators 512, more or fewer surgical instrument manipulators 512 may be used.

[0027] A functional minimally invasive remote operation surgical system generally includes a vision system part that enables a user of the remote operation surgical system to view the surgical site from outside the patient's body 522. The vision system typically includes a camera device 528 for capturing video images and one or more video displays for displaying the captured video images. In some surgical system configurations, the camera device 528 includes an optical system that transfers an image from the distal end of the camera device 528 to one or more imaging sensors (e.g., CCD or CMOS sensors) outside the patient's body 522. Alternatively, the (multiple) imaging sensors can be positioned at the distal end of the camera device 528, and the signals generated by the (multiple) sensors can be transmitted along a lead wire or wirelessly for processing and display on one or more video displays. An example of a video display is a stereoscopic display on a surgeon console within a surgical system commercialized by Intuitive Surgical, Inc. of Sunnyvale, California.

[0028] Referring to FIG. 5, attached to each surgical instrument manipulator 512 is a surgical instrument 520 that operates at the surgical site within the patient's body 522. Each surgical instrument manipulator 512 can be provided in various forms that allow the associated surgical instrument to move with one or more mechanical degrees of freedom (e.g., all six Cartesian degrees of freedom, less than five Cartesian degrees of freedom, etc.). Typically, mechanical or control constraints restrict each manipulator 512 such that the associated surgical instrument moves around the center of motion on the instrument that remains stationary with respect to the patient, and this center of motion is typically located at the position where the instrument enters the body.

[0029] In one aspect, the surgical instrument 520 is controlled through computer-assisted remote operation. The minimally invasive functional remote robotic surgery system includes control inputs that receive inputs from a user (e.g., a surgeon or other healthcare provider) of the remote operation surgery system. The control inputs communicate with one or more computer-controlled remote operation actuators, such as one or more motors to which the surgical instrument 520 is coupled. In this way, the surgical instrument 520 moves in response to the movement of the control input of the healthcare provider. In one aspect, one or more control inputs are included in a surgeon console, such as the surgeon console 16 shown in FIG. 2. The surgeon can operate the control input 36 of the surgeon console 16 to activate the remote operation actuator of the patient-side cart 500. The force generated by the remote operation actuator is transmitted through a drive train mechanism, which transmits the force from the remote operation actuator to the surgical instrument 520.

[0030] Referring to FIG. 5, in one aspect, the surgical instrument 520 and the cannula 524 are removably coupled to the manipulator 512 with the surgical instrument 520 inserted through the cannula 524. One or more remote operation actuators of the manipulator 512 move the surgical instrument 512 as a whole. The manipulator 512 further includes an instrument carriage 530. The surgical instrument 520 is removably connected to the instrument carriage 530. In one aspect, the instrument carriage 530 houses inside one or more remote operation actuators that provide a number of controller operations for converting the various movements of the end effector of the surgical instrument 520 into corresponding movements of the surgical instrument 520. Thus, the remote operation actuators within the instrument carriage 530 move only one or more components of the surgical instrument 520 rather than moving the instrument as a whole. The input for controlling either the entire instrument or a component of the instrument is such that the input provided by the surgeon or other healthcare provider to a control input ("master" command) is converted into a corresponding movement ("slave" response) by the surgical instrument.

[0031] In an alternative embodiment, the instrument carriage 530 does not house a remotely operated actuator. The remotely operated actuator that enables various movements of the end effector of the surgical instrument 520 is housed at a location remote from the instrument carriage 530, for example, at another location on the patient side cart 500. A cable-based force transmission mechanism or a similar mechanism is used to transmit the movement of each remotely operated actuator of the remotely located remotely operated actuator to a corresponding instrument interface connection actuator output disposed on the instrument carriage 530. In some embodiments, the surgical instrument 520 is mechanically coupled to a first actuator that controls a first movement of the surgical instrument, such as longitudinal (z-axis) rotation. The surgical instrument 520 is mechanically coupled to a second actuator that controls a second movement of the surgical instrument, such as two-dimensional (x,y) movement. The surgical instrument 520 is mechanically coupled to a third actuator that controls a third movement of the surgical instrument, such as opening and closing of the jaw end effector.

[0032] FIG. 6A is a side view of a surgical instrument 520 including a proximal control mechanism 640 and a distal portion 650 coupled by an elongate tube 610 having an elongate tube central axis 611. The surgical instrument 520 is configured to be inserted into a patient's body and is used to perform surgical or diagnostic procedures. The distal portion 650 of the surgical instrument 520 can provide any of a variety of end effectors 654, such as the illustrated forceps, needle driver, cautery device, cutting tool, imaging device (e.g., an endoscope or ultrasonic probe), or the like. The surgical end effector 654 can include functional mechanical degrees of freedom, such as jaws that open and close or a knife that translates along a path. In the illustrated embodiment, the end effector 654 is coupled to the elongate tube 610 by a wrist 652, which enables the end effector to be oriented with respect to the elongate tube central axis 611. The surgical instrument 520 may include information stored (e.g., in a semiconductor memory device 660 associated with the instrument), which may be permanent or may be updatable by a surgical system configured to operate the surgical instrument 520. In some embodiments, a semiconductor memory device 660 associated with an instrument tracking device 662 is associated with the instrument 520. In some embodiments, the instrument tracking device 662 includes a radio frequency identification (RFID) device. Accordingly, the surgical system may provide one-way or two-way information communication between the surgical instrument 520 and one or more components of the surgical system.

[0033] FIG. 6B is an exemplary diagram depicting an RFID tracking device 662 associated with an instrument 520 according to some embodiments. The RFID device 662 includes a storage device 660 including an information structure 664 that associates device identification information 666 with remaining device life information 668. The RFID device 662 includes a transceiver circuit 670 for wirelessly transmitting and receiving information. The RFID device 662 includes control logic circuitry 672 for controlling the storage device 660 and the transceiver 670.

[0034] FIG. 7 is a perspective view of the surgical instrument manipulator 512. The instrument manipulator 512 is shown in a state where no surgical instrument is attached. The instrument manipulator 512 includes an instrument carriage 530, and a surgical instrument (e.g., surgical instrument 520) can be removably connected to the instrument carriage 530. The instrument carriage 530 houses a plurality of remote operation actuators. In some embodiments, an RFID reader 535 is disposed on the instrument manipulator 512 at a location for reading the contents of a storage device 660 within an RFID device 662 associated with the instrument attached to the manipulator 512. Each remote operation actuator includes an actuator output 705. When a surgical instrument is mounted on the instrument manipulator 512, one or more instrument inputs (not shown) of an instrument proximal control mechanism (e.g., proximal control mechanism 640 in FIG. 6) are mechanically coupled to the corresponding actuator output 705. In one aspect, this mechanical coupling is direct, and the actuator output 705 contacts the corresponding instrument input directly. In another aspect, this mechanical coupling occurs through an intermediate interface such as a component of a drape configured to provide a sterile barrier between the instrument manipulator 512 and the associated surgical instrument.

[0035] In one aspect, the operation of one or more instrument inputs by corresponding remotely operated actuators results in movement of the mechanical degrees of freedom of the surgical instrument. For example, in one aspect, the surgical instrument attached to the instrument manipulator 512 is the surgical instrument 520 shown in FIG. 6. Referring to FIG. 6, in one aspect, the operation of one or more instrument inputs of the proximal control mechanism 640 by corresponding remotely operated actuators rotates the elongate tube 610 (along with the attached wrist 652 and end effector 654) relative to the proximal control mechanism 640 about the elongate tube central axis 611. In another aspect, the operation of one or more instrument inputs by corresponding remotely operated actuators results in the operation of the wrist 652 that orients the end effector 654 relative to the elongate tube central axis 611. In another aspect, the operation of one or more instrument inputs by corresponding remotely operated actuators results in the operation of one or more movable elements (e.g., jaw members, knife members, etc.) of the end effector 654. Thus, the operation of the remotely operated actuator of the instrument carriage 530 can move the various mechanical degrees of freedom of the surgical instrument mounted on the instrument manipulator 512.

[0036] (Annotating the recorded video) FIG. 8 shows a schematic view of an exemplary surgical planning tool 800. In one aspect, the surgical planning tool 800 includes a teleoperation surgical system 850 that communicates data with an electronic medical device record database 830 and an instrument inventory management system 860. The teleoperation surgical system 850 shown here is similar to the teleoperation surgical system 850 shown in FIG. 4. In one aspect, the electronic medical record database 830 includes the medical records of patients treated at a particular hospital. The database 830 and the instrument inventory management system 860 can be implemented on a server located on-site at the hospital. Medical record entries included in the database 830 and the management system 860 can be accessed from a hospital computer through an intranet network. Alternatively, for example, one of a number of cloud data storage services can be used to implement the database 830 and the management system 860 on a remote server located off-site from the hospital. In this case, the medical record entries of the database 830 and the management system 860 are stored on the cloud server and can be accessed by a computer having Internet access.

[0037] In one aspect, a surgical procedure is performed on a first patient using a remote surgery system 850. An imaging device associated with the remote surgery system 850 captures an image of the surgical site and displays the captured image as video frames on a display of a surgeon console 52. In one aspect, a healthcare provider at the surgeon console 52 uses an input device of the surgeon console 52 to highlight or annotate a particular patient anatomical structure shown in the displayed video. An example of such an input device is the control input 36 shown in FIG. 2, which is coupled to a cursor that operates with a graphical user interface overlaid on the displayed video. The graphical user interface can include a QWERTY keyboard, a pointing device such as a mouse and an interactive screen display, a touch screen display, or other means for data or character input. Thus, the healthcare provider can highlight a particular tissue of interest in the displayed image or enter a text annotation.

[0038] In one aspect, the surgical site video is additionally displayed on a display disposed in an equipment cart 56. In one aspect, the display of the equipment cart is a touch screen user interface that can be used by a healthcare provider to highlight or annotate a particular portion of a patient anatomical structure shown in the image displayed for viewing on the display of the equipment cart. The user can highlight a portion of the displayed image by touching a portion of the patient anatomical structure displayed on the touch screen user interface. Additionally, a graphical interface including a QWERTY keyboard can be overlaid on the displayed image. The user can enter a text annotation using the QWERTY keyboard.

[0039] In one aspect, the surgical site video captured by an imaging device associated with the remote surgery system 850 is stored in the database 830 in addition to being recorded by the remote surgery system 850 and displayed to the user in real time or substantially in real time. Highlights and / or annotations associated with the recorded video created by the user can also be stored in the database 830. In one aspect, the highlights created by the user are embedded in the recorded video prior to their storage in the database 830. Later, the recorded video can be searched for viewing. In one aspect, a viewer of the recorded video can select whether the highlights are to be displayed or suppressed from display. Similarly, annotations associated with the recorded video can also be stored in the database 830. In one aspect, the annotations created by the user can be used to tag the recorded video and can be used to provide a means of identifying the subject matter included in the recorded video. For example, one annotation may describe a particular pathological condition. This annotation is used to tag the recorded video. Later, a person who wishes to view the recorded procedures related to this pathological condition can search for the video using a keyword search.

[0040] (Search of Stored Videos) In some cases, it is desirable for a healthcare provider to be able to view video recordings of past surgical procedures performed on a given patient. In one aspect, a patient who has previously undergone a first surgical procedure to treat a medical condition may subsequently require a second surgical procedure to treat a recurrence of the same medical condition or to treat an anatomical structure located near the surgical site of the first surgical procedure. In one aspect, the surgical site events of the first surgical procedure are captured in a surgical site video recording, and the video recording is stored in database 830 as part of the patient's electronic medical record. Prior to performing the second surgical procedure on the patient, the healthcare provider can search database 830 to locate the video recording of the patient's initial surgical procedure.

[0041] In some cases, it is desirable for a healthcare provider planning to perform a surgical procedure on a patient to be able to view video recordings of similar surgical procedures performed on persons with certain characteristics similar to that patient. In one aspect, the surgical site video recordings of the surgical procedures can be tagged with metadata information such as the patient's age, gender, body mass index, genetic information, type of procedure the patient has undergone, etc. before each video recording is archived in database 830. In one aspect, the metadata information used to tag the video recordings is automatically retrieved from the patient's medical record existing at that time and then used to tag the video recordings before they are archived in database 830. Thus, prior to performing a medical procedure on a patient, the healthcare provider can search database 830 for video recordings of similar procedures performed on patients sharing certain characteristics in common with that patient. For example, if a healthcare provider is planning to use a robotic surgery system 850 to perform a prostatectomy on a 65-year-old male patient with an elevated body mass index, the healthcare provider can search database 830 for video recordings of prostatectomy surgical sites performed using robotic surgery system 850 on other male patients of a similar age with a similar elevated body mass index.

[0042] In one aspect, a video recording of a surgical procedure is communicated by a database 830 to an optional personal computer 820 (shown by the dashed line) and made available for viewing by a healthcare provider planning to perform the surgical procedure. Additionally or alternatively, a video recording of an initial surgical procedure can be communicated by the database 830 to a remote surgery system 850 and made available for preoperative or intraoperative viewing. In one aspect, the video recording is displayed on a display located at a surgeon console 52 by the remote surgery system 850. In another aspect, the video recording of the first surgical procedure is displayed on a display located on an electronic equipment cart 56.

[0043] (Cloud-based video database) In one aspect, the database 830 is implemented on a remote server using a cloud data storage service and is accessible by multiple healthcare providers. Referring to FIG. 8, as shown by the dashed lines, the surgical planning tool 800 optionally includes a remote operating surgery system 850 (as shown by the dashed lines) and a personal computer 840 (as shown by the dashed lines). In one aspect, except that the remote operating surgery system 850 and the personal computer 820 are located at a first healthcare provider and the remote operating surgery system 850 and the personal computer 840 are located at a second healthcare provider, the remote operating surgery system 850 is similar to the remote operating surgery system 850, and the personal computer 840 is similar to the personal computer 820. In one aspect, a first patient undergoes a surgical procedure that requires surgical treatment of a medical condition and uses the remote operating surgery system 850 at a first healthcare provider. A video recording of the surgical procedure is archived in the database 830. Later, a second patient is planned to undergo a surgical treatment that requires surgical treatment of the same medical condition and uses the remote operating surgery system 850 at a second healthcare provider. Prior to performing the surgical procedure on the second patient, a healthcare provider accesses the database 830 through a secure Internet connection and searches the database 830 for video recordings of the surgical site of a similar procedure. In one aspect, the healthcare provider treating the second patient can search the database 830 for the video recording of the surgical procedure of the first patient without obtaining knowledge of the identity of the first patient. In this way, the privacy of the first patient is maintained. In one aspect, the video recording of the surgical procedure of the first patient includes highlights and / or annotations created by the healthcare provider who treated the first patient.

[0044] (Computer-based pattern matching and analysis) The surgical planning tool 800 can include pattern matching and analysis algorithms implemented in the form of computer-executable code. In one aspect, the pattern matching and analysis algorithms are stored in a non-volatile memory device of the surgical planning tool 800 and are configured to analyze video recordings archived in the database 830. As discussed above, each of the video recordings archived in the database 830 can be tagged or embedded with specific metadata information. This metadata information can include information such as the patient's age, gender, and other information describing the patient's health or medical history. Additionally, as discussed above, the metadata information can include highlights or annotations created by medical personnel. In one aspect, these highlights and annotations are embedded with the video recording and archived with the video in the database 830.

[0045] In one aspect, the pattern matching and analysis algorithms include an image analysis component that identifies patterns of shapes and colors shared among a plurality of video recordings stored in the database 830. Next, the pattern matching and analysis algorithms examine the tagged metadata associated with this subset of video recordings to determine whether any words or phrases are frequently associated with the videos within this subset. These analyses performed by the pattern matching and analysis algorithms can be used to assist medical personnel in making decisions about patient anatomical structures, suitable surgical approaches, pathologies, potential complications, and the like.

[0046] (Method of using the surgical planning tool) Figure 9 shows a method 900 of using a surgical planning tool. In one aspect, the surgical planning tool is similar to the surgical planning tool 800 in FIG. 8. At 910, the medical device receives facts or characteristics that describe a medical patient, e.g., the medical condition the patient suffers from. The medical device can receive this fact or situation via a user interface disposed in a teleoperated surgical system (e.g., the teleoperated surgical system 10 in FIG. 1 or the teleoperated surgical system 50 in FIG. 4) or alternatively through a personal computer similar to the personal computer 820 in FIG. 2. At 920, the medical device uses the facts or characteristics received at 910 to retrieve at least one associated video record of a surgical procedure from a medical device database. At 930, the medical device uses the video record to determine surgical planning information. In one aspect, the surgical planning information includes the types of instruments used in the procedure being recorded. At 940, the medical device displays the surgical planning information determined at 930 to the user.

[0047] (Method of Surgical Instrument Inventory Management Based on Observation of Surgical Instrument Use) Table 1 identifies some exemplary surgical instruments and corresponding operational life decreases due to everyday intraoperative use, irregular intraoperative use, and sterilization, according to some embodiments.

[0048] [Table 1]

[0049] In some embodiments, for the purpose of extending the lifespan, i.e., adding additional surgical use, the instrument can be refurbished. However, a refurbished instrument may start with a shorter lifespan than a new instrument. In some embodiments, the number of surgeries in which a surgical instrument can be used varies depending on the use of the instrument. Irregular use can result in more rapid degradation of the instrument. For example, irregular use of a pair of scissors instrument for suturing can result in accelerated degradation. Moreover, the sterilization of the instrument may involve high temperatures and chemical treatments, which can cause wear and tear of the instrument.

[0050] Figure 10 is an exemplary diagram representing a storage atlas within a computer-readable storage device 1004 according to some embodiments. The storage atlas 1002 includes a first information structure 1006 indicating instances of previously performed surgical procedures. A second information structure 1008 associates a surgical procedure with the surgical activities performed during the surgical procedure and the surgical instruments used during the surgical procedure. A third information structure 1010 associates the surgical activities with the surgical instrument operating states. A fourth information structure 1012 associates the daily surgical instrument use and lifespan degradation with the surgical instrument operating states. A fifth information structure 1014 associates the non-daily surgical instrument use and lifespan degradation with the surgical instrument operating states. A sixth information structure 1016 associates the surgical instrument sterilization and lifespan degradation with the sterilization events. A seventh information structure 1018 associates the video images of the surgical scenes recorded during the surgical procedure with the instrument operating states recorded during the surgical procedure. In some embodiments, the various information structures 1004-1018 are evaluated to generate an eighth information structure 1020 that associates surgical activities with the surgical instrument activity states and surgical instrument lifespan degradation.

[0051] FIG. 11 is an exemplary diagram showing an example of a seventh information structure 1018 included in an atlas 1002 within a storage device 1004, which associates recorded video images from individual surgeries with corresponding surgical instrument actuator state information. In one aspect, to generate a chronological record of surgical activities and corresponding surgical instrument actuator states during a surgical procedure, video images of patient anatomical structures and the instruments used to act on those anatomical structures during the surgery and the corresponding surgical instrument actuator states are recorded and timestamped (t1, t2... tn) during the surgery. The timestamps are used to temporarily align the video images with the surgical instrument actuator states.

[0052] During surgery, the user may annotate the video recording and the surgical instrument actuation state recording with metadata indicating the corresponding surgical activity. The annotation may include, for example, one or more or a combination of written notes tagged to the video information and / or the surgical instrument actuation state information, coloring or highlighting (e.g., telestretching) of images in the video recording. The annotation may be timestamped for use in temporarily aligning them with the corresponding video recording information and the corresponding recorded surgical instrument state information.

[0053] During a remotely operated surgical procedure, the surgical activity includes the use of at least one surgical instrument. During the surgical activity, the surgical instrument is actuated in one or more actuator states under the control of the surgeon. Actuation of the surgical instrument to support the surgical activity in one or more surgical states can result in degradation of the effectiveness of the instrument for its intended use. As described more fully below, a record indicating the remaining life of the instrument is modified to track the progression of this reduction in effectiveness. For example, a life count may be decremented in response to use of the instrument in the surgical activity.

[0054] Figures 12A - 12C are exemplary diagrams showing an exemplary surgical instrument 1202 and an actuator assembly 1203 in three different exemplary operating states according to several embodiments. The exemplary instrument 1202 includes a jaw end effector 1204 that can transition between an open state and a closed state and a continuum of partially open / partially closed states therebetween. The exemplary instrument 1202 also includes a two - degree - of - freedom (2dof) list 1206 that can move between different two - dimensional (x,y) position states. The exemplary actuator assembly 1203 includes a first actuator 1208 that, in some embodiments, includes a jaw motor (JM) used to operate the jaw end effector 1204. The exemplary actuator assembly 1203 includes a second actuator 1210 that, in some embodiments, includes a list motor (WM) used to operate the list 1206. During surgery, the surgical instrument 1202 may transition through a plurality of operating states corresponding to different activities during a surgical procedure. As represented in Figure 12A, for example, a surgical procedure may include a first surgical activity where the first actuator 1208 (JM) positions the jaw end effector 1204 in a fully open state and the second actuator 1210 (WM) positions the list 1206 in a first position state (x1,y1). As represented in Figure 12B, for example, a surgical procedure may include a second surgical activity where the first actuator 1208 transitions the jaw end effector 1204 to a fully closed state and the second actuator 1210 transitions the list 1206 to a second position state (x2,y2). As represented in Figure 12C, for example, a surgical procedure may include a third surgical activity where the first actuator 1208 positions the jaw end effector 1204 in a partially open / partially closed state and the second actuator 1210 transitions the list 1206 to a third position state (x3,y3).

[0055] FIG. 13 is an exemplary diagram depicting an exemplary instance 1020i of an eighth information structure 1020 of an atlas 1002 stored in a computer-readable storage device 1004 according to some embodiments. The exemplary eighth information structure instance 1020i associates surgical activities during a selected surgical procedure with possible surgical instrument actuator states during the activity. The exemplary eighth information structure instance 1020i associates possible surgical instrument actuator states with corresponding surgical instrument life decrements.

[0056] The first column of the exemplary eighth information structure instance 1020i shows a list A1, A2, A3... AN of surgical activities to be performed during an exemplary surgical procedure. The second column of the information structure instance 1020i shows instruments I1, I2, I3 to be used during the surgical activity. The third column shows a first possible surgical instrument actuation state that can occur during the corresponding surgical activity. The fourth column shows a second possible surgical instrument actuation state that can occur during the corresponding surgical activity. Referring to the third and fourth columns, the first instrument I1 can operate in either of two possible surgical instrument actuator (SIA) states, SIA 11 and SAI 12 The second instrument I2 can operate only in one surgical instrument actuator state, SIA 21 The third instrument I3 can operate in either of two possible surgical instrument actuator states, SIA 31 and SIA 32 The fifth column shows the life decrement corresponding to the corresponding first possible surgical instrument actuation state. The sixth column shows the life decrement corresponding to the corresponding second possible surgical instrument actuation state. Referring to the fifth and sixth columns, the first instrument I1 operating in the first actuator state SIA 11 of the first instrument is associated with the first decrement count I1 C1 of the first instrument, and the first instrument I1 operating in the second actuator state SIA 12 of the first instrument is associated with the second decrement count I1 C2is associated with. The first actuator state SIA of the second instrument 21 The second instrument I2 that operates in is the first decreasing count I2 of the second instrument C1 is associated with, and the second actuator state SIA of the second instrument 22 The second instrument I2 that operates in is the second decreasing count I2 of the second instrument C2 is associated with. The first actuator state SIA of the third instrument 31 The third instrument I3 that operates in is the first decreasing count I3 of the third instrument C1 is associated with, and the second actuator state SIA of the third instrument 32 The third instrument I3 that operates in is the second decreasing count I3 of the third instrument C2 is associated with.

[0057] Note that some surgical activities may involve the use of instruments such as endoscopes, which do not wear or deteriorate as a result of their use. Also, some instruments such as instrument I2 are operable only in a single daily actuator state and not in an alternative second actuator state, and thus are associated with only a single category of life decrement. Moreover, in some embodiments, the life decrement associated with an instrument actuator state is determined as a function of energy usage over time between instrument actuator states, and thus the life decrement of such an instrument can be variable depending on energy usage. According to some embodiments, the greater the energy used, the greater the life decrement. For example, faster operation of an instrument during an actuator state can result in more energy usage than the same instrument operating at a slower speed in the same actuator state.

[0058] Referring to the first row of the exemplary information structure instance 1020i of FIG. 13, for example, during activity A1, the first instrument I1 can be operated in the first operating state SIA of the first instrument 11 which decreases the remaining life 668 of the first instrument by the first decreasing count I1 C1Only gradually reduce it. Alternatively, during activity A1, the first instrument I1 can be operated in the second operating state SIA of the first instrument 12 which gradually reduces the remaining life 668 of the first instrument by only the second decrement count I1 C2 Only gradually reduce it.

[0059] Referring to the second row of the exemplary information structure instance 1020i of FIG. 13, for example, during a surgery, the first activity A2 of the surgery can include a second instrument I2 that is operated in the first operating state SIA of the second instrument 21 which reduces the remaining life 668 of the second instrument by only the first decrement count I2 C1 Note that in this example, there is no second operating state for the second instrument. Referring to the Nth row of the exemplary information structure instance 1020i of FIG. 13, for example, during the Nth activity AN, the third instrument I3 can be operated in the first operating state SIA of the third instrument 31 which reduces the remaining life 668 of the third instrument by only the first decrement count I3 C1 Alternatively, during activity AN, the third instrument I3 can be operated in the second operating state SIA of the third instrument 32 which reduces the remaining life 668 of the third instrument by only the second decrement count I3 C2 Only gradually reduce it.

[0060] FIG. 14 is an exemplary flow diagram representing process 1402 for configuring processor 58 to determine wear of a surgical instrument that occurs during a surgical procedure according to some embodiments. At block 1404, surgical instrument identification is received as an input to the computer processing system associated with electronic equipment cart 56. The surgical instrument identification includes information identifying each individual instrument scheduled for use in the surgical procedure and corresponding remaining lifetime information identified instruments. At block 1406, an identification of the surgical procedure is received as an input to the computer processing system associated with electronic equipment cart 56. At block 1408, an instance of an eighth information structure 1010 is generated that associates surgical activity with surgical instrument activity status and surgical instrument life degradation using information included in atlas 1002 within information structures 1008, 1010, 1012, 1014, 1016.

[0061] During the execution of the identified surgical procedure, block 1410 tracks the operating state of the surgical instrument actuator. In decision block 1412, a determination is made as to whether the current instrument actuator state matches the actuator state associated with the instrument's decreasing life. In response to no match, the control loop returns to block 1410 and tracking continues. In response to a match, block 1414 decrements the remaining life 668 of the identified instrument based on the decreasing life associated with the matching instrument actuator state. Decision block 1416 determines whether a surgical procedure is being performed. In response to a determination that a surgical procedure has not yet been performed, the control then returns to block 1410, and block 1410 continues to track the surgical instrument actuator state, for example, based on other identified actuator state transition information. In response to a determination that a surgical procedure is being performed, block 1418 uses the FRID reader 535 to read the instrument identifier information 666 and the corresponding remaining life information 668 from the storage device 660 within the RFID 662 associated with the instrument, and transmits the remaining life information for the identified instrument to the inventory management system 860.

[0062] For example, assuming that process 1402 of FIG. 14 is executed to use the exemplary information structure instance 1020i of FIG. 13 during the execution of a surgical procedure, block 1410 tracks the operating states of the surgical instrument actuators for each of instruments I1, I2, I3. In decision block 1412, a determination is made as to whether any one or more of the current instrument actuator states of the three surgical instrument actuators match the actuator states associated with the corresponding instruments' life decrements. In response to no match, the control loop returns to block 1410 and the tracking continues. In response to a match, block 1414 decrements the remaining life 668 of the identified instrument based on the life decrement associated with the matching actuator state. For example, during tracking during surgical activity A1, decision block 1412 determines whether the current actuator state of instrument I1 matches either actuator state SIA 11 or SIA 21 . In response to a match between SIA 11 and the actuator state of I1, the life 668 of I1 is decremented by I1 C1 . In response to a match between SIA 21 and the actuator state of I1, the life 668 for I1 is decremented by I1 C2 . Next, control flows to decision block 1416, which determines whether there are any additional surgical states to be executed. In this example, surgical states A2 through AN must be executed after surgical state A1. Accordingly, control returns to block 1410, which continues to track the surgical instrument actuator states during surgical states A2 through AN, for example, based on other identified actuator state transition information. After completion of all surgical activities A1 through AN, block 1418 configures the processor to send the updated life information for the identified instrument to the inventory management system 860.

[0063] FIG. 15 is an exemplary diagram representing a sterilization chamber 1502 according to some embodiments, where a surgical instrument 520 disposed inside the sterilization chamber 1502 is associated with an FRID device 662 and a storage device 660. Following sterilization of the instrument prior to use of the instrument in a subsequent surgery, an RFID reader 535 associated with the sterilization chamber 1502 reads instrument identifier information 666 and corresponding remaining life information 668 from the memory 660 for transmission to an inventory management system 860.

[0064] Exemplary embodiments have been shown and described, but extensive modifications, changes, and substitutions are envisioned in the foregoing disclosure, and in some cases, some configurations of the embodiments may be utilized without the corresponding use of other configurations. For example, in some embodiments, a processor 58 is coupled to a memory device, such as a storage device 1004, that includes a set of instructions executable on the processor 58 to cause the processor 58 to perform operations. In some embodiments, the operations include providing a first information structure in a memory device that associates a surgical instrument identifier with a remaining useful life of the identified surgical instrument. The operations further include tracking, during performance of a surgical procedure, the surgical instrument actuator state of the identified surgical instrument. The operations further include decreasing, by an amount of surgical instrument life reduction, the associated remaining useful life of the identified surgical instrument in response to the tracked surgical instrument actuator state matching a surgical instrument wear-down actuation state during performance of the surgical procedure.

[0065] Those skilled in the art will recognize many variations, alternatives, and modifications. Accordingly, the scope of the disclosure should be limited only by the following claims, and it is appropriate that the claims be construed broadly in a manner consistent with the scope of the embodiments disclosed herein.

Claims

1. A method for tracking the use of a surgical instrument in a surgical system by a computer processor of the surgical system, the method comprising: identifying at least a first surgical activity and a second surgical activity that contribute to instrument degradation of the surgical instrument during the performance of a surgical procedure; adjusting remaining useful life information of the surgical instrument stored in a computer-readable storage device to indicate a reduced useful life. Adjusting the remaining useful life information includes: tracking the occurrence of each of the first surgical activity and the second surgical activity during the performance of the surgical procedure; reducing the remaining useful life information stored in the computer-readable storage device by a first amount of life reduction, based in part on the use of the surgical instrument during the occurrence of the first surgical activity; reducing the remaining useful life information stored in the computer-readable storage device by a second amount of life reduction, based in part on the use of the surgical instrument during the occurrence of the second surgical activity. A method.

2. The method according to claim 1, wherein the first amount of life reduction is different from the second amount of life reduction.

3. The method according to claim 1, further comprising sending a display of the remaining useful life of the surgical instrument to an inventory management system through a network.

4. A surgical system including a surgical instrument, the surgical system comprising: a processor; a memory device holding a set of instructions executable by the processor to cause the processor to perform operations, the operations including: identifying at least a first surgical activity and a second surgical activity that contribute to instrument degradation of the surgical instrument during the performance of a surgical procedure; adjusting remaining useful life information of the surgical instrument stored in a computer-readable storage device to indicate a reduced useful life. Adjusting the remaining useful life information includes: tracking the occurrence of each of the first surgical activity and the second surgical activity during the performance of the surgical procedure; reducing the remaining useful life information stored in the computer-readable storage device by a first amount of life reduction, based in part on the use of the surgical instrument during the occurrence of the first surgical activity; ​ Based at least in part on the use of the surgical instrument during the occurrence of the second surgical activity, reducing the remaining useful life information stored in the computer-readable storage device by a second amount of life reduction; A surgical system. **Claim 5** The surgical system according to claim 4, wherein the first amount of life reduction is different from the second amount of life reduction. **Claim 6** A method of tracking the use of a surgical instrument in a surgical system by a computer processor of the surgical system, the method comprising: Identifying at least a first surgical activity and a second surgical activity that contribute to the instrument degradation of the surgical instrument during the performance of a surgical procedure; Adjusting remaining useful life information of one or more surgical instruments stored in a computer-readable storage device to indicate a reduced useful life, the adjusting comprising: Adjusting the remaining useful life information includes: Tracking the occurrence of each of the first surgical activity and the second surgical activity during the performance of the surgical procedure; Reducing the remaining useful life information stored in the computer-readable storage device by a first amount of life reduction that is a function of energy use during the tracked first surgical activity; Reducing the remaining useful life information stored in the computer-readable storage device by a second amount of life reduction that is a function of energy use during the tracked second surgical activity; A method. **Claim 7** The method according to claim 6, wherein the first amount of life reduction is different from the second amount of life reduction. **Claim 8** The method according to claim 6, further comprising sending a display of the remaining life of the surgical instrument through a network to an inventory management system. **Claim 9** A surgical system including a surgical instrument, the surgical system comprising: A processor; A memory device holding a set of instructions executable by the processor to cause the processor to perform operations, the operations comprising: The operations are: Identifying at least a first surgical activity and a second surgical activity that contribute to the instrument degradation of the surgical instrument during the performance of a surgical procedure; Adjusting remaining useful life information of one or more surgical instruments stored in a computer-readable storage device to indicate a reduced useful life, the adjusting comprising: Adjusting the remaining useful life information includes: Tracking the occurrence of each of the first surgical activity and the second surgical activity during the performance of the surgical procedure; Reducing the remaining useful life information stored in the computer-readable storage device by a first amount of life reduction that is a function of energy use during the first surgical activity being tracked; Reducing the remaining useful life information stored in the computer-readable storage device by a second amount of life reduction that is a function of energy use during the second surgical activity being tracked, including; A surgical system.

10. The surgical system according to claim 9, wherein the first amount of life reduction is different from the second amount of life reduction.

11. The first surgical activity includes the operation of a first surgical instrument, The second surgical activity includes the operation of a second surgical instrument, The method according to claim 1.

12. The first surgical activity includes the operation of a first surgical instrument, The second surgical activity includes the operation of a second surgical instrument, The surgical system according to claim 4.

13. The first surgical activity includes the operation of a first surgical instrument, The second surgical activity includes the operation of a second surgical instrument, The method according to claim 6.

14. The first surgical activity includes the operation of a first surgical instrument, The second surgical activity includes the operation of a second surgical instrument, The surgical system according to claim 9.

Images