Teleoperated surgical system with scan-based localization

Abstract

This invention relates to a remote surgical system with scan-based positioning. A method for use with the remote surgical system is provided, the method comprising: determining patient location information during setup for performing a surgical procedure within an instance of the surgical system; determining a match between the determined patient location and a patient location signature; and transmitting a support arm control signal within the surgical system corresponding to the matched support arm signature.

Description

[0001] This application is a divisional application of Chinese patent application 2017800733786 (PCT / US2017 / 061144), entitled “Remote Surgical System with Scan-Based Positioning”, filed on November 10, 2017, with an international filing date of November 10, 2017, and entering the national phase on May 28, 2019.

[0002] Priority requirements

[0003] This application claims the benefit of priority to U.S. Patent Application No. 62 / 421,089, filed November 11, 2016, which is incorporated herein by reference in its entirety. Technical Field

[0004] This invention relates to medical devices used during surgery. More specifically, it relates to the positioning of components of a surgical system. Background Technology

[0005] Surgeons typically conduct extensive research before performing surgical procedures. Traditionally, surgeons have limited themselves to studying general anatomical models, such as photographs or drawings. More recently, various preoperative diagnostic procedures (e.g., X-rays, CT scans, MRI, etc.) have made patient-specific anatomical information available.

[0006] In some cases, it is desirable to provide surgeons with additional information regarding relevant anatomy and surgical procedures. In one aspect, it is desirable to provide surgeons planning surgery for a specific patient with video recordings of the surgical site from earlier procedures performed on that specific patient. In another aspect, it is desirable to provide surgeons with one or more video recordings of surgical procedures performed on other patients similar to those planned for a specific patient. In one aspect, it is desirable to provide such information to the surgeon before performing a specific surgical procedure. Moreover, in another aspect, it may be desirable to provide this information to the surgeon during the operation.

[0007] In one aspect, it is desirable to configure a video database comprising intraoperative surgical site video recordings of various procedures performed by different types of patients. In another aspect, it is desirable to configure a medical device capable of video recording to also include an input that allows the surgeon using the device to highlight and annotate the video recordings in real time during recording. In yet another aspect, it is desirable to configure a computer-based pattern matching algorithm to search the individual records in the video database, identify related video records, and provide the surgeon with such relevant information for a specific surgical procedure. Summary of the Invention

[0008] The following summary introduces certain aspects of the subject matter of the invention to provide a basic understanding. This summary is not a broad overview of the subject matter and is not intended to identify key or essential elements or to define the scope of the subject matter. While this summary contains information relating to various aspects and embodiments of the subject matter, its sole purpose is to set forth some aspects and embodiments in general form as a prelude to the more detailed description that follows.

[0009] In one aspect, a method for use with a remotely operated surgical system is provided. Patient location information is determined during the setup of the surgical system for a surgical procedure. A match is determined between the patient location information determined during setup and a corresponding patient location signature. During setup, a support arm control signal is transmitted, corresponding to the matched patient location signature.

[0010] On the other hand, a method for use with a remotely operated surgical system is provided. During the setup of the surgical system for the surgical procedure, an operating room layout is determined. A match is determined between the operating room layout determined during setup and a corresponding operating room layout signature. An image representing the location of surgical system modules corresponding to the matched corresponding operating room layout signature is generated. Attached Figure Description

[0011] Figure 1 This is a floor plan of a minimally invasive remote surgical system.

[0012] Figure 2 This is a perspective view of the surgeon's console.

[0013] Figure 3 This is a perspective view of an electronic trolley.

[0014] Figure 4 This is a schematic diagram of a remote surgical system.

[0015] Figure 5A This is a perspective view of the patient-side trolley of the minimally invasive remote surgical system according to an embodiment.

[0016] Figure 5B This is an illustrative simplified block diagram showing an example positioning of a mechanical support arm of a surgical system that is remotely operated during a surgical procedure, according to some embodiments.

[0017] Figure 5C-5E The accompanying drawings are illustrative of a segmented mechanical support arm having its first, second, and third segments and being in three different horizontal positions, according to some embodiments.

[0018] Figure 5F-5G This indicates that, according to some embodiments Figure 5C-5E The third paragraph contains certain details and illustrative diagrams showing the third paragraph in two different vertical positions.

[0019] Figure 6 This is a front view of the surgical instruments.

[0020] Figure 7 It is a perspective view of the instrument's manipulator.

[0021] Figure 8 This is a diagram of a surgical planning tool.

[0022] Figure 9 This is a flowchart of the method for using surgical planning tools.

[0023] Figures 10A-10B These are illustrative top-front views of two example patients with different anatomical dimensions. The two example patients have similar incision patterns suitable for the same surgical procedure, but with different spacing between the incisions due to different anatomical spacing.

[0024] Figure 11A-11B These are illustrative top-front views of two different example patients who have different anatomical dimensions and similar incision patterns suitable for the same second surgical procedure.

[0025] Figure 12A-12B This is an illustrative side view of the incisions of two example patients, who are placed on an operating room table at different tilt angles, undergoing the same surgical procedure and having the same type of surgical instruments inserted into the surgical incisions.

[0026] Figures 13A-13C This is an illustrative diagram showing the arrangement of three different surgical system modules.

[0027] Figure 14 These are illustrative drawings showing a storage atlas in a computer-readable storage device according to some embodiments.

[0028] Figure 15 The accompanying drawings are illustrative examples illustrating a second information structure included within a map according to some embodiments.

[0029] Figure 16 The accompanying drawings are illustrative examples illustrating a third information structure included within a map according to some embodiments.

[0030] Figure 17 This is an illustrative drawing showing an example first control signal rule information structure that associates patient anatomical feature information and patient position information with mechanical support arm position control signals according to some embodiments.

[0031] Figure 18 This is an illustrative flowchart illustrating the process of generating control signals based at least in part on patient anatomical information, patient location information, and surgical procedure type, according to some embodiments.

[0032] Figure 19 This is an illustrative drawing showing an example second rule information structure that associates operating room layout information and operating room personnel information with system module information according to some embodiments.

[0033] Figure 20 This is an illustrative flowchart illustrating the process of generating computer-displayed image information representing the location of a recommendation system module, based at least in part on operating room layout information, surgical personnel activity information, and surgical procedure type information, according to some embodiments. Detailed Implementation

[0034] This specification and accompanying drawings, which illustrate aspects of the invention, embodiments, implementation methods, or applications, should not be considered limiting—the claims define the protected invention. Various mechanical, compositional, structural, electrical, and operational changes may be made without departing from the scope of this specification and claims. In some cases, well-known circuits, structures, or techniques have not been shown or described in detail so as not to obscure the invention. The same numerals in two or more drawings denote the same or similar elements.

[0035] Wherever possible, an element described in detail with reference to one embodiment, implementation, or application may be included in other embodiments, implementations, or applications that are not specifically shown or described therein. For example, if an element is described in detail with reference to one embodiment but not with reference to a second embodiment, it may still be claimed that the element is included in the second embodiment. Therefore, to avoid unnecessary repetition in the following description, one or more elements shown and described in connection with one embodiment, implementation, or application may be incorporated into other embodiments, implementations, or aspects unless one or more elements would render an embodiment or implementation ineffective, or unless two or more elements provide conflicting functions.

[0036] Various aspects of this invention are primarily based on the use of the da Vinci surgical instrument, commercially available from Intuitive Surgical, Inc. of Sunnyvale, California. Surgical system (specifically, as da) Xi TM HD TMThis description refers to an implementation of the IS4000 surgical system (a commercially available model). However, those skilled in the art will understand that the inventive aspects disclosed herein can be embodied and implemented in various ways, including robotic and (if applicable) non-robotic embodiments and implementations. Surgical systems (e.g., model IS4000da) Xi TM Surgical system, model IS3000 da Vinci The implementation of the surgical system is merely exemplary and should not be construed as limiting the scope of the inventive aspects disclosed herein.

[0037] According to various aspects, this disclosure describes a surgical planning tool that includes a medical device configured to video record the execution of a surgical procedure. The video recording can be embedded with various metadata, such as highlights created by medical personnel. Additionally, the video recording can be tagged with various metadata, such as text annotations describing certain topics of the video, the identity of the patient corresponding to the video recording, and information about the patient's history or medical history. In one aspect, the tagged metadata is embedded in the video recording.

[0038] According to another aspect, the robotic arm configured to mount surgical instruments is positioned at least partially based on the location of at least one patient anatomical feature. According to yet another aspect, the robotic surgical system module mounted on a trolley is capable of being positioned near the patient at least partially based on the configuration of the operating room in which the surgical procedure is performed. In remotely operated robot-assisted surgical procedures, surgical instruments extend into the patient's body cavity through surgical ports at incisions in the patient's anatomy. The incision location and the surgical port location are determined at least partially based on the type of surgical procedure, the patient's body dimensions, and the patient's position on the operating table. In some embodiments, the surgical instruments are positioned during surgery to have a central axis of rotation at the port in order to minimize strain on the anatomical tissue surrounding the port.

[0039] In some embodiments, multiple surgical system modules mounted on a trolley are arranged around the patient during surgery. One or more of the surgical system modules include robotic surgical arms configured to mount surgical instruments. In some embodiments, each of the multiple robotic surgical arms mounts a different surgical instrument. During the surgical procedure, each instrument extends to a different location within the patient's body cavity through a different surgical port. Typically, multiple medical personnel move around the operating room during surgery. Medical personnel must be able to access the patient at any time, both for routine surgical activities and to respond to emergencies. For example, a surgical procedure may involve a series of surgical activities such as cutting, cauterization, and suturing. Different surgical activities may require the use of different surgical instruments or different combinations of surgical instruments. During a surgical procedure, medical personnel in the operating room may need to move between the patient and the surgical system modules mounted on the trolley or between multiple surgical system modules mounted on the trolley to change surgical instruments extending through one or more surgical ports in order to reconfigure the remotely operated surgical system or perform other surgical activities. For example, when a surgical activity is completed, it may be possible to remove the surgical instruments used during a single surgical activity and to insert different surgical instruments in their positions to perform different surgical activities. The operating room in which the surgical procedure is performed can have limited space in which the surgical system modules mounted on the trolley are positioned to facilitate access to the surgical port, while still allowing sufficient spacing between the surgical system modules mounted on the trolley and the patient table, or between such modules themselves, so that medical personnel can move freely and efficiently around the operating room during the procedure.

[0040] Video recordings, information structures that associate the location of the robotic surgical arm with the patient's position and / or with the patient's physical characteristics (such as height and weight), and information structures that associate the location of the surgical system modules with the operating room layout can be archived in a locally implemented electronic medical record database or on a cloud data storage service. The video recordings can then be made available to interested healthcare institutions. The information structures are used in conjunction with surgical system setup guidelines to determine the location of the robotic surgical arm and surgical system modules.

[0041] Healthcare institutions can use the aforementioned metadata tags to search medical device databases based on one or more of the surgical procedure to be performed, patient physical characteristics, and operating room layouts for the structural relationships between videos and information of interest. Additionally, in one aspect, the surgical planning tool includes computer-based pattern matching and analysis algorithms. In one aspect, the pattern matching algorithm filters videos stored in electronic medical record databases to identify correlations between visual characteristics in the video recordings and relevant metadata tags created by medical personnel. The surgical planning tool can apply these correlations to newly encountered anatomical structures, thereby assisting medical personnel in performing procedures to determine patient anatomy, optimize surgical approaches, disease states, potential complications, etc.

[0042] On the other hand, pattern matching algorithms eliminate video and other records stored in the electronic medical record database to identify the most suitable robotic arm positioning for different patient positions and anatomy. Furthermore, pattern matching algorithms eliminate video and other records stored in the electronic medical record database to identify surgical system module positions that facilitate the most efficient use of operating room space for different operating room layouts. In some embodiments, scanning devices such as light-irradiation scanning devices (e.g., laser scanning devices, IR scanning devices, or RF scanning devices) can be used to record patient positioning and medical personnel movement. Alternatively or additionally, video can provide a record of medical personnel movement during the surgical procedure. Alternatively, for example, medical personnel can wear tracking devices such as RFID tags to track their movement. Recordings of patient positioning during previous surgeries can be used to determine the most efficient positioning of one or more robotic arms to achieve precise instrument positioning more quickly during surgical setup. Recordings of medical personnel movement during previous surgeries can be used to determine the most efficient positioning of surgical system modules to facilitate ease and speed of medical personnel movement during surgery.

[0043] Minimally invasive remote surgical system

[0044] Referring now to the accompanying drawings, in which the same reference numerals denote the same parts in several views. Figure 1 This is a floor plan of a minimally invasive remote surgical system 10, which is typically used to perform minimally invasive diagnostic or surgical procedures on a patient 12 lying on a movable operating table 14. The system includes a movable surgeon's console 16 for use by a surgeon 18 during the procedure. One or more assistants 20 may also participate in the procedure. The minimally invasive remote surgical system 10 also includes a movable patient-side trolley 22 and a movable electronic trolley 24. In some embodiments, the table 14, the surgeon's console 16, the patient-side trolley 22, and the electronic trolley 24 are equipped with wheels to provide mobility.

[0045] The patient-side trolley 22 includes multiple segmented mechanical support arms 72, each having one end rotatably mounted to a vertical support structure 74 and the other end on which a removably coupled surgical instrument 26 is mounted. In some embodiments, each mechanical support arm 72 includes a first segment 72-1, a second segment 72-2, and a third segment 72-3. During the setup of the surgical procedure, multiple segments of at least one support arm 72 are moved to position a surgical instrument for insertion into a minimally invasive incision in the body of the patient 12. During the surgical procedure, while the surgical instrument is inserted into the patient's body cavity, the surgeon 18 views the surgical site via a surgeon's console 16. Images of the surgical site can be obtained via an endoscope 28 (e.g., a stereoscopic endoscope), which can be manipulated by the patient-side trolley 22 to orient the endoscope 28. A computer processor located on an electronic trolley 24 is capable of processing the images of the surgical site for subsequent display to the surgeon 18 via the surgeon's console 16. The number of surgical instruments 26 used at one time will typically depend on the diagnostic or surgical procedure and space constraints within the operating room, as well as other factors. If one or more of the surgical instruments 26 must be changed during the procedure, the assistant 20 is able to remove the surgical instrument 26 from the patient-side trolley 22 and replace it with another surgical instrument 26 from the tray 30 in the operating room.

[0046] Figure 2 This is a perspective view of the surgeon's console 16. The surgeon's console 16 includes a left-eye display 32 and a right-eye display 34 for presenting the surgeon 18 with a coordinate-based, depth-sensing-enabled stereoscopic view of the surgical site. The console 16 also includes one or more control inputs 36. It is mounted on the patient-side trolley 22 ( Figure 1 One or more surgical instruments used on the surgical instrument 26 (shown in the diagram) move in response to the surgeon 18 manipulating the one or more control inputs 36. The control inputs 36 can provide guidance to the associated surgical instrument 26 (…). Figure 1 (shown in the figure) The same mechanical degrees of freedom are used to provide surgeon 18 with a sense that the remote presentation or control input 36 is integrated with the instrument 26, so that the surgeon has a strong feeling of direct control over the instrument 26. For this purpose, position, force and tactile feedback sensors (not shown) can be used to transmit the position, force and tactile sensation from the surgical instrument 26 back to the surgeon's hand via control input 36.

[0047] The surgeon's console 16 is typically located in the same room as the patient so that the surgeon can directly monitor the procedure, be physically present if necessary, and speak directly to the patient's assistant instead of via telephone or other communication mediums. However, the surgeon can also be located in a different room, a completely different building, or other location far from the patient, thus allowing for remote surgical procedures.

[0048] Figure 3 This is a perspective view of the electronic trolley 24. The electronic trolley 24 can be coupled to the endoscope 28 and includes a computer processor to process the captured images for subsequent display, such as on a surgeon's console or on another suitable monitor located locally and / or remotely. For example, if a stereoscopic endoscope is used, the computer processor on the electronic trolley 24 can process the captured images to present the surgeon with a coordinated stereoscopic image of the surgical site. This coordinate system can include alignment between relative images and can include adjusting the stereoscopic working distance of the stereoscopic endoscope. As another example, image processing can include compensating for imaging errors, such as optical aberrations, in the image capture device using previously determined camera calibration parameters. Optionally, the equipment in the electronic trolley can be integrated into the surgeon's console or the patient-side trolley, or it can be distributed in various other locations within the operating room.

[0049] Figure 4 The diagram schematically illustrates a remotely operated surgical system 50 (e.g., Figure 1 Minimally invasive remote surgical system 10). Surgeon's console 52 (e.g., Figure 1 The surgeon's console 16) can be used by the surgeon to control the patient-side trolley 54 (e.g., during minimally invasive procedures). Figure 1 The patient-side trolley 22). The patient-side trolley 54 is capable of using imaging devices such as stereoscopic endoscopes to capture images of the surgical site and output the captured images to the electronic trolley 56 (e.g., ...). Figure 1 The computer processor is located on the electronic trolley 24. The computer processor typically includes one or more data processing boards designed to execute computer-readable code stored in a non-volatile memory device within the computer processor. In one aspect, the computer processor is capable of processing the captured image in various ways prior to any subsequent display. For example, the computer processor can overlay the captured image with a virtual control interface before displaying the combined image to the surgeon via the surgeon's console 52.

[0050] Additionally or alternatively, the captured images can be image-processed by a computer processor located outside the electronic trolley 56. In one aspect, the remote surgical system 50 includes an optional computer processor 58 (shown as dashed lines), similar to the computer processor located on the electronic trolley 56, and the patient-side trolley 54 outputs the captured images to the computer processor 58 for image processing before display on the surgeon's console 52. In another aspect, the captured images are first image-processed by the computer processor on the electronic trolley 56, and then further image-processed by the computer processor 58 before being displayed on the surgeon's console 52. The remote surgical system 50 can include an optional display 60, as shown as dashed lines. The display 60 is coupled to and connected to the computer processor 58 located on the electronic trolley 56, and the captured images processed by these computer processors can be displayed on the display 60 in addition to the display on the surgeon's console 52.

[0051] Figure 5A This is a perspective view of a patient-side trolley 54 of a minimally invasive remote surgical system 10 according to an embodiment. The patient-side trolley 54 includes four mechanical support arms 72. Surgical instrument manipulators 73 (which include motors for controlling instrument movement) are mounted at the end of each support arm assembly 72. Additionally, each support arm 72 may optionally include one or more (e.g., unpowered and / or lockable) setting connectors for positioning the attached surgical instrument manipulator 73 relative to the patient receiving the surgery. As shown, the patient-side trolley 54 is placed on the floor. In other embodiments, the operating portion of the patient-side trolley can be mounted to a wall, a ceiling, an operating table 14 that also supports the patient's body 12, or other operating room equipment. Furthermore, although the patient-side trolley 54 is shown as including four surgical instrument manipulators 73, more or fewer surgical instrument manipulators 73 may be used.

[0052] Functional remote surgical systems typically include a vision system component that allows the user of the remote surgical system to view the surgical site from outside the patient's body 12. The vision system typically includes a camera instrument 26C for capturing video images and one or more video displays for displaying the captured video images. In some surgical system configurations, the camera instrument 26C includes optics for transmitting images from a distal end of the camera instrument 26C to the outside of the patient's body 12 using one or more imaging sensors (e.g., CCD or CMOS sensors). Alternatively, the imaging sensors(s)(s)(s)(s)(s) can be positioned distally to the camera instrument 26C(s), and signals generated by the sensors(s)(s)(s)) can be transmitted along leads or wirelessly for processing and display on the one or more video displays(s). An example of a video display is a stereoscopic display on the surgeon's console in a surgical system commercially available from Intuitive Surgical Operations, Sunnyvale, California.

[0053] refer to Figure 5A Attached to each surgical instrument manipulator 73 is a surgical instrument 26 that operates at the surgical site within the patient's body 12. Each surgical instrument manipulator 73 can be configured in various ways to allow the associated surgical instrument to move with one or more mechanical degrees of freedom (e.g., all six Cartesian degrees of freedom, five or fewer Cartesian degrees of freedom, etc.). Typically, mechanical or control constraints restrict each manipulator 73 to move its associated surgical instrument about a center of motion on the instrument that remains stationary relative to the patient, and this center of motion is typically located at the point where the instrument enters the body.

[0054] In one aspect, surgical instruments 26 are controlled via computer-aided remote operation. The functional minimally invasive remote operation surgical system includes control inputs that receive input from a user of the remote operation surgical system (e.g., a surgeon or other medical personnel). The control inputs communicate with one or more computer-controlled remote operation actuators (e.g., one or more motors coupled to surgical instruments 26). In this way, surgical instruments 26 move in response to movement of the medical personnel's control inputs. In another aspect, one or more control inputs are included in the surgeon's console (e.g., ...). Figure 2 The surgeon is shown in the surgeon's console 16. The surgeon can manipulate the control input 36 of the surgeon's console 16 to operate the remote actuators of the patient-side trolley 54. The force generated by the remote actuators is transmitted via a transmission system mechanism, which transmits the force from the remote actuators to the surgical instruments 26.

[0055] refer to Figure 5AIn one aspect, surgical instrument 26 and cannula 27 are removably coupled to manipulator 73, wherein surgical instrument 26 is inserted through cannula 27. One or more teleoperated actuators of manipulator 73 move the entire surgical instrument 26. Manipulator 73 also includes instrument carriage 75. Surgical instrument 26 is detachably connected to instrument carriage 75. In one aspect, instrument carriage 75 houses one or more internal teleoperated actuators that provide several controller movements, which surgical instrument 26 translates into various movements of an end effector on surgical instrument 26. Thus, the teleoperated actuators in instrument carriage 75 move only one or more components of surgical instrument 26, not the entire instrument. Inputs used to control the entire instrument or components of the instrument cause inputs (“master” commands) provided by a surgeon or other medical personnel to be translated by the surgical instrument into corresponding actions (“slave” responses).

[0056] In an alternative embodiment, the instrument carriage 75 does not house remotely operated actuators. Remotely operated actuators capable of various movements of the end effector of the surgical instrument 26 are housed in a location remote from the instrument carriage 75, such as elsewhere on the patient-side cart 54. Cable-based force transmission mechanisms or the like are used to transmit the motion of each remotely positioned remotely operated actuator to a corresponding actuator output located on the instrument carriage 75 and connected to the instrument. In some embodiments, the surgical instrument 26 is mechanically coupled to a first actuator that controls a first movement of the surgical instrument, such as longitudinal (z-axis) rotation. The surgical instrument 26 is mechanically coupled to a second actuator that controls a second movement of the surgical instrument, such as two-dimensional (x, y) motion. The surgical instrument 26 is mechanically coupled to a third actuator that controls a third movement of the surgical instrument, such as opening and closing of a forceps end effector.

[0057] Figure 5B This is an illustrative simplified block diagram showing an example positioning of the mechanical support arms 72A-72C of a surgical system 10 for remote operation during a surgical procedure, according to some embodiments. In some embodiments, the patient-side system 54 includes at least three mechanical support arms 72A-72C. In some embodiments, each of the mechanical support arms 72A-72C includes a first segment 72-1, a second segment 72-2, and a third segment 72-3 that are rotatably mounted. The centrally located mechanical support arm 72 can support an endoscope camera 26C adapted to capture images within the camera's field of view. The centrally located left and right mechanical support arms 72 can support instruments 26A and 26B, respectively, which are manipulating anatomical tissues. During the setup of the surgical procedure, the support arm segments are pre-positioned to support the endoscope and instruments in precise positions and orientations so that the surgeon can perform robot-assisted manipulations to execute medical procedures.

[0058] The user or operator O (typically a surgeon) performs surgical procedures on the patient P by manipulating control input devices 36 (such as handles and foot pedals at the main console 16). The operator can view video frames of images of the surgical site inside the patient's body via a stereoscopic monitor viewer 31. The computer processor 58 of the console 16 guides the movement of remotely controlled endoscopic surgical instruments 26A-26C via control lines 159, thereby enabling instrument movement using the patient-side system 24 (also known as a patient-side trolley).

[0059] Figure 5C-5E This is an illustrative drawing showing a segmented mechanical support arm 72 according to some embodiments, wherein the first segment 72-1, the second segment 72-2, and the third segment 72-3 of the mechanical support arm 72 are in three different horizontal positions. For simplicity, only one mechanical support arm is shown, although the surgical system includes multiple support arms, such as... Figure 1 As shown. The first segment 72-1 includes a first end 81-1 and a second end 81-2. The second segment 72-2 includes a first end 82-1 and a second end 82-2. The third segment 72-3 includes a first end 83-1 and a second end 83-2.

[0060] The first segment 72-1 is rotatably mounted at its first end 81-1 for horizontal rotation about a corresponding vertical axis 91 at the vertical support structure 74. The second segment 72-2 is rotatably mounted at its first end 82-1 for horizontal rotation about a corresponding vertical axis 92 at the second end 81-2 of the corresponding first segment 72-1. The third segment 72-3 is mounted at its first end 83-1 for horizontal rotation about a corresponding vertical axis 93 at the second end 82-2 of the corresponding second segment 72-2.

[0061] Figure 5C This is an illustrative drawing showing the three segments of the segmented mechanical support arm 72 extending fully to the right (as seen from the perspective of the drawing). Figure 5D This shows (relative to the first paragraph 72-1 and the second paragraph 72-2) in Figure 5C The illustration shows the first segment 72-1 rotating clockwise around the vertical axis 91 and the second segment 72-2 rotating counterclockwise around the vertical axis 92, with the rotational position of the third segment 72-3 remaining unchanged relative to the second segment 72-2. Figure 5E Yes, it is shown (relative to the first paragraph 72-1 and the second paragraph 72-2 in) Figure 5A The illustration shows the first segment 72-1 rotating counterclockwise around the vertical axis 91 and the second segment 72-2 rotating clockwise around the vertical axis 92, with the rotational position of the third segment 72-3 remaining unchanged relative to the second segment 72-2.

[0062] Figure 5F-5GThis indicates that, according to some embodiments Figure 5C-5E The accompanying drawings show certain details of the third segment 72-3 and illustrate the third segment 72-3 (also known as the machine manipulator) in two different vertical positions. The third mechanical support arm segment 72-3 comprises first, second, third, and fourth sub-segments 101-104. The first sub-segment 101 comprises a first end 101-1 and a second end 101-2. The second sub-segment 102 comprises a first end 102-1 and a second end 102-2. The third sub-segment 103 comprises a first end 103-1 and a second end 103-2. The fourth sub-segment 104 comprises a first end 104-1 and a second end 104-2.

[0063] A first sub-segment 101 is rotatably mounted at its first end 101-1 for vertical rotation about a corresponding horizontal axis 111 at a second end 82-2 of the second segment 72-2. A second sub-segment 102 is rotatably mounted at its first end 102-1 for vertical rotation about a corresponding horizontal axis 112 at a second end 101-2 of the first sub-segment 101. A third sub-segment 103 is rotatably mounted at its first end 103-1 for vertical rotation about a corresponding horizontal axis 113 at a second end 102-2 of the second sub-segment 102. A fourth sub-segment 104 is rotatably mounted at its first end 104 for vertical rotation about a corresponding horizontal axis 114 at a second end 103-2 of the third sub-segment 103. According to some embodiments, the fourth sub-segment 104 includes a mounting structure 120 for securely mounting surgical instruments during surgical procedures.

[0064] Figure 5F The accompanying drawing is an illustrative drawing showing four sub-segments of a third mechanical support arm segment 72-3 in a generally folded or retracted configuration position according to some embodiments. Figure 5G The accompanying drawing is an illustrative illustration showing four sub-segments of a third mechanical support arm segment 72-3 in a partially configured position according to some embodiments. More specifically, in Figure 5G In the middle, the rotational position of the first sub-segment 101 about the horizontal axis 111 is relative to its position in the middle. Figure 5F The rotational position of the second sub-segment 102 remains unchanged relative to its position within the [missing information]. Figure 5F The rotational position of the third sub-segment 103 rotates clockwise around the horizontal axis 112. Figure 5F The rotational position within rotates counterclockwise around the horizontal axis 113. The fourth sub-segment 104, relative to its position within... Figure 5F The rotational position in the middle rotates clockwise around the horizontal axis 114.

[0065] Figure 6This is a side view of a surgical instrument 26, which includes a distal portion 650 and a proximal control mechanism 640 connected by an elongated tube 610 having a central axis 611. The surgical instrument 26 is configured for insertion into a patient and for performing surgical or diagnostic procedures. The distal portion 650 of the surgical instrument 26 can provide any of a variety of end effectors 654, such as the forceps, needle actuators, cauterization devices, cutting tools, or imaging devices (e.g., endoscopes or ultrasound probes). The surgical end effector 654 can include functional mechanical degrees of freedom, such as opening or closing jaws or a blade that translates along a path. In the illustrated embodiment, the end effector 654 is coupled to the elongated tube 610 via a wrist 652, which allows the end effector to be oriented relative to the central axis 611 of the elongated tube. The surgical instrument 26 may also contain stored information (e.g., stored in a semiconductor memory associated with the instrument), which may be permanent or may be updated by a surgical system configured to operate the surgical instrument 26. Therefore, the surgical system can provide one-way or two-way communication between the surgical instrument 26 and one or more components of the surgical system.

[0066] Figure 7 This is a perspective view of a third arm segment 72-3 (also referred to as a surgical instrument manipulator) according to some embodiments. The third arm segment 72-3 is shown without surgical instruments mounted. The third arm segment 72-3 includes first, second, third, and fourth sub-segments 101-104 rotatably mounted about a horizontal axis 111-114, as referenced. Figure 5F-5G As shown and as described above, the third arm segment 72-3 also includes an instrument carriage 75 to which surgical instruments (e.g., surgical instrument 26) can be detachably attached. The instrument carriage 75 houses a plurality of remote-operated actuators. Each remote-operated actuator includes an actuator output 705. When the surgical instrument is mounted on the instrument manipulator 72-3, the proximal instrument control mechanism (e.g., Figure 6 One or more instrument inputs (not shown) of the proximal control mechanism 640 are mechanically coupled to corresponding actuator outputs 705. In one aspect, the mechanical coupling is direct, wherein the actuator output 705 directly contacts the corresponding instrument input. In another aspect, the mechanical coupling occurs through an intermediate interface (e.g., a drape configured to provide a sterile barrier between the third segment / instrument manipulator 72-3 and the associated surgical instrument).

[0067] In one aspect, movement of one or more instrument inputs caused by a corresponding remote-operated actuator results in movement of the mechanical degrees of freedom of the surgical instrument. For example, in one aspect, the surgical instrument mounted on the instrument manipulator 72-3 is surgical instrument 26, such as... Figure 6 As shown. Reference Figure 6In one aspect, movement of one or more instrument inputs caused by the corresponding teleoperated actuators of the proximal control mechanism 640 causes the elongated tube 610 (and the attached wrist 652 and end effector 654) to rotate about the elongated tube's central axis 611 relative to the proximal control mechanism 640. In another aspect, movement of one or more instrument inputs caused by the corresponding teleoperated actuators results in movement of the wrist 652, thereby orienting the end effector 654 relative to the elongated tube's central axis 611. In yet another aspect, movement of one or more instrument inputs caused by the corresponding teleoperated actuators results in movement of one or more movable elements of the end effector 654 (e.g., jaw members, blade members, etc.). Thus, various mechanical degrees of freedom of the surgical instruments mounted on the instrument manipulator 26 can be moved by operation of the teleoperated actuators of the instrument carriage 75.

[0068] Annotate the recorded video

[0069] Figure 8 An illustrative drawing shows an example surgical planning tool 800. In one aspect, the surgical planning tool 800 includes a remotely operated surgical system 850 that communicates with an electronic medical device record database 830. The remotely operated surgical system 850 shown here is similar to... Figure 4 The diagram shows a remote surgical system 850. In one aspect, an electronic medical record database 830 includes the medical records of patients treated at a specific hospital. Database 830 can be implemented on a server located at the hospital site. Medical record entries contained in database 830 can be accessed from a hospital computer via an intranet network. Alternatively, database 830 can be implemented on a remote server located off-site, for example, using one of several cloud data storage servers. In this case, the medical record entries of database 830 are stored on a cloud server and can be accessed by a computer using the Internet.

[0070] In one aspect, a surgical procedure is performed on a first patient using a remote surgical system 850. An imaging device associated with the remote surgical system 850 captures images of the surgical site and displays the captured images as video frames on the monitor of the surgeon's console 52. In another aspect, a medical personnel at the surgeon's console 52 use an input device on the surgeon's console 52 to highlight or annotate a particular patient's anatomical structure shown in the displayed video. An example of such an input device is... Figure 2The control input 36 shown is coupled to a cursor, which operates in conjunction with a graphical user interface overlaid on the displayed video. The graphical user interface can include a QWERTY keyboard, pointing devices such as a mouse and interactive screen display, a touchscreen display, or other devices for data or text input. Therefore, medical personnel can highlight a specific tissue of interest or enter text annotations in the displayed image.

[0071] In one aspect, the surgical site video is additionally displayed on a monitor located on an electronic trolley 56. In another aspect, the monitor on the electronic trolley is a touchscreen user interface, which can be used by medical personnel to highlight and annotate certain portions of the patient's anatomy shown in the image displayed on the monitor on the electronic trolley. By touching a portion of the patient's anatomy displayed on the touchscreen user interface, the user can highlight that portion of the displayed image. Additionally, a graphical interface including a QWERTY keyboard can be overlaid on the displayed image. The user can use the QWERTY keyboard to input text annotations.

[0072] In one aspect, surgical site videos captured by imaging devices associated with the remote surgical system 850 are recorded by the remote surgical system 850 and stored in a database 830, in addition to being displayed to the user in real-time or near real-time. Highlights and / or annotations associated with the recorded video made by the user can also be stored in the database 830. In one aspect, user-made highlights are embedded in the recorded video before being stored in the database 830. The recorded video can later be retrieved for viewing. In one aspect, a viewer of the recorded video can choose whether to display or suppress highlights from the view. Similarly, annotations associated with the recorded video can also be stored in the database 830. In one aspect, user-made annotations are used to label the recorded video and can be used as a means of identifying subjects contained in the recorded video. For example, an annotation may describe the condition of a certain disease state. This annotation is used to label the recorded video. Later, a person wishing to view the recorded procedure regarding that disease state can use keyword search to locate the video.

[0073] Retrieve stored videos

[0074] In some cases, it is desirable for medical personnel to be able to view video recordings of past surgical procedures performed on a given patient. In one aspect, a patient who previously underwent a first surgical procedure to treat a medical condition subsequently requires a second surgical procedure to treat a recurrence of the same medical condition or to treat an anatomical structure located near the surgical site of the first procedure. In another aspect, the surgical site event of the first surgical procedure is captured in the surgical site video recording, and the video recording is archived in database 830 as part of the patient's electronic medical record. Before performing a second surgical procedure on the patient, medical personnel can perform a search of database 830 to locate video recordings of the patient's earlier surgical procedures.

[0075] In some cases, it is desirable for medical personnel planning to perform surgical procedures on a patient to be able to view video recordings of similar surgical procedures performed on individuals with certain characteristics similar to that patient. In one aspect, before archiving each video recording in database 830, the surgical site video recording of the surgical procedure can be tagged with metadata information such as the patient's age, sex, body mass index, genetic information, and the type of procedure the patient underwent. In another aspect, the metadata information used to tag the video recording is automatically retrieved from the patient's existing medical records and then used to tag the video recording before archiving it in database 830. Therefore, before performing a medical procedure on a patient, medical personnel can search in database 830 for video recordings of similar procedures performed on patients who share certain common characteristics with that patient. For example, if medical personnel plan to perform a prostatectomy on a 65-year-old male patient with an elevated body mass index using a remote surgical system 850, they can search in database 830 for video recordings of surgical sites of prostatectomies performed on other men of similar age with similarly elevated body mass indices using the remote surgical system 850.

[0076] In one aspect, video recordings of the surgical procedure are transmitted from database 830 to an optional personal computer 820 (shown as dashed lines) and are available for viewing by medical personnel planning to perform the procedure. Additionally or alternatively, video recordings of earlier surgical procedures can be transmitted from database 830 to a remote surgical system 850 and are available for viewing preoperatively or intraoperatively. In one aspect, the video recordings are displayed on a monitor located on the surgeon's console 52 via the remote surgical system 850. In another aspect, video recordings of the first surgical procedure are displayed on a monitor located on an electronic trolley 56.

[0077] Cloud-based video database

[0078] In one respect, Database 830 is implemented on a remote server using cloud data storage services and is accessible to multiple healthcare organizations. (Reference) Figure 8As shown by the dashed lines, the surgical planning tool 800 optionally includes a remote surgical system 850 (as shown by the dashed lines) and a personal computer 840 (as shown by the dashed lines). In one aspect, the remote surgical system 850 is similar to the remote surgical system 820, and the personal computer 840 is similar to the personal computer 820, except that the remote surgical system 850 and personal computer 820 are located at a first healthcare facility, while the remote surgical system 850 and personal computer 840 are located at a second healthcare facility. In one aspect, a first patient requires surgical treatment for a medical condition, and the surgical procedure is performed at the first healthcare facility using the remote surgical system 850. Video recordings of the surgical procedure are archived in a database 830. Later, a second patient requires surgical treatment for the same medical condition and plans to receive surgical treatment at the second healthcare facility using the remote surgical system 850. Before performing the surgical procedure on the second patient, medical personnel access the database 830 via a secure internet connection and search the database 830 for video recordings of surgical sites for similar procedures. In one respect, medical personnel treating the second patient can retrieve video recordings of the first patient's surgical procedure from database 830 without knowing the first patient's identity. In this way, the first patient's privacy is maintained. In another respect, the video recordings of the first patient's surgical procedure include highlights and / or annotations made by the medical personnel treating the first patient.

[0079] Computer-based pattern matching and analysis

[0080] The surgical planning tool 800 can include pattern matching and analysis algorithms implemented in the form of computer-executable code. In one aspect, the pattern matching and analysis algorithms are stored in the non-volatile memory device of the surgical planning tool 800 and configured to analyze video recordings archived in a database 830. As previously described, each video recording archived in the database 830 can be tagged with and / or embedded with certain metadata information. This metadata information can include patient information, such as patient age, gender, and other information describing the patient's health history or medical history. Additionally, as previously discussed, the metadata information can include highlights or annotations made by medical personnel. In one aspect, these highlights and annotations are embedded in the video recordings and archived together with the videos in the database 830.

[0081] In one aspect, the pattern matching and analysis algorithm includes an image analysis component that identifies patterns of shape and color shared among multiple video recordings stored in database 830. The pattern matching and analysis algorithm then examines the metadata of tags associated with a subset of video recordings to determine whether any words or phrases are frequently associated with videos within that subset. These analyses performed by the pattern matching and analysis algorithm can be used to help medical personnel determine patient anatomy, preferred surgical methods, disease states, potential complications, etc.

[0082] Methods using surgical planning tools

[0083] Figure 9 A method 900 using a surgical planning tool is illustrated. In one aspect, the surgical planning tool is similar to... Figure 8 The surgical planning tool 800. At 910, the medical device receives facts or characteristics describing the medical patient, such as the patient's medical condition. The medical device can be accessed via a remotely operated surgical system (e.g., Figure 1 The remote surgical system 10 or Figure 4 The user interface on the remote surgical system 50, or alternatively, through a similar... Figure 2 The personal computer at 820 receives this fact or situation. At 920, the medical device uses the fact or characteristic received at 910 to retrieve at least one relevant video recording of the surgical procedure from the medical device database. At 930, the medical device uses the video recording to determine surgical planning information. In one aspect, the surgical planning information includes the type of instrument used in the recorded procedure. At 940, the medical device displays the surgical planning information determined at 930 to the user.

[0084] Mechanical support arm positioning and surgical system module positioning

[0085] It should be understood that the positioning of the robotic arm 72 during a surgical procedure, including the positioning of its multiple segments 72-1, 72-2, and 72-3, can depend on a variety of factors, such as the type of surgical procedure, patient anatomy, patient position, and operating room layout. Different surgeries may involve different surgical incision patterns. Different patient anatomy may result in different spacing between incisions used in the surgical procedure. During surgery, different patients may require different positions or orientations relative to the operating room table. Different operating room layouts may require repositioning of the robotic arm to accommodate the physical constraints introduced by the OR (operating room) layout.

[0086] Figures 10A-10BThese are illustrative top-front views of two example patients 1202 and 1206 with different anatomical dimensions. The two example patients have similar incision patterns suitable for the same first surgical procedure, but with different spacing between the incisions due to different anatomical spacing. Figure 10A This is an illustrative drawing showing a first patient 1202 with smaller anatomical dimensions, the first patient 1202 having a first surgical incision 1204 having a first pattern with a first spacing. Figure 10B This is an illustrative drawing showing a second patient 1206 with larger anatomical dimensions, having a second incision 1208 with a second pattern of a second spacing. It should be understood that the patterns of the first incision 1204 in the first patient 1202 and the second incision 1208 in the second patient 1206 are similar because each patient undergoes the same surgery. However, due to the different anatomical dimensions of the two patients, the spacing between the first incision 1204 in the first patient 1202 and the second incision 1208 in the second patient 1206 is different. It should be understood that... Figure 5A-5G Each of the plurality of mechanical support arms 72, including each of its multiple sub-segments 72-1, 72-2, 72-3, is positioned at different locations to arrange an instrument 26 for insertion into a first incision 1204 in a first patient 1202 and for insertion of the instrument 26 into a second incision in a second patient. For example, in a computer-readable storage device (described more fully below), a processor 58 is used to record the mechanical support arm setup information.

[0087] Figure 11A-11B These are illustrative top-front views of two different example patients 1212 and 1216, who have different anatomical dimensions and similar incision patterns 1214 and 1218 suitable for the same second surgical procedure. Due to their different body sizes, the third patient 1212 and the fourth patient 1216 have different spacing between their incisions, resulting from different anatomical spacing. It can be seen that... Figure 11A The incision pattern of the second surgical procedure shown in the image is similar to... Figures 10A-10B The incision patterns for the first surgical procedure shown in the images differ. It should be understood that... Figure 5A-5G The positions of the multiple mechanical support arms 72 for the second surgical procedure will differ from their positions for the first surgical procedure. For example, in a computer-readable storage device (described more fully below), a processor 58 records the mechanical support arm setup information.

[0088] Figure 12A-12BThis is an illustrative side view of two example patients 1302 and 1306, who are placed on operating table 14 at different tilt angles, and who undergo the same surgical procedure and have the same type of surgical instruments inserted into the surgical incision. Figure 12A This is an illustrative drawing showing an example first patient 1302 lying on an operating table 14, which is aligned parallel to the ground such that the head, abdomen, and legs are horizontal. Four surgical instruments are inserted into the surgical incision in the first patient's body for the surgical procedure. Figure 12B This is an illustrative drawing showing an example second patient 1306 lying on an operating table 14, which is tilted at an angle so that the head is raised higher and the legs are lowered. Four surgical instruments are inserted into the surgical incision in the second patient for the surgical procedure. It should be understood that... Figure 5A-5G Each of the plurality of mechanical support arms 72, including each of its multiple sub-segments 72-1, 72-2, 72-3, is positioned at different locations to arrange an instrument 26 for insertion into an incision in a first patient 1202 and for insertion of the instrument 26 into an incision in a second patient 1204. It should be understood that ranges with different angles can be used, for example, a straight 0-degree range for the first patient 1302 and a top-down 30-degree range for the second patient 1306; in this case, the kinematics of the arms 72 can be used to readjust the arm configuration of the patient 1306 corresponding to the endoscope configuration. For example, in a computer-readable storage device (described more fully below), a processor 58 is used to record the mechanical support arm configuration information.

[0089] It should be understood that, for example, the positioning of surgical system modules (e.g., the positioning of the main control console 16, surgical instrument tray 30, electronic trolley 24, and patient-side trolley 54) can depend on a variety of factors, such as the type of surgical procedure, the operating room layout during surgery, and the movement patterns of operating room personnel. Different operating room layouts may require different system module positioning due to the different fixtures within the operating room. For example, different operating room personnel movement patterns may require different system module positioning to facilitate efficient and safe movement of personnel during surgical procedures.

[0090] Figures 13A-13C This is an illustrative diagram showing the arrangement of three different surgical system modules. Figure 13A The accompanying illustration shows the operating room containing fixtures F1-F3 arranged as shown, wherein surgical system modules 16, 30, 24, and 54 are positioned in the first arrangement shown. For example, the fixtures may include structures such as poles, cabinets, or walls, or obstacles such as power outlets. Figure 13B The operating room includes Figure 13AThe illustrative drawing also includes fixation devices F1-F3 arranged in the middle and fixation device F4 arranged as shown in the figure, wherein surgical system modules 16, 30, 24 and 54 are positioned in a second arrangement as shown in the figure. Figure 13C The illustrative drawing shows the operating room containing the fixtures F5-F7 arranged as shown, wherein surgical system modules 16, 30, 24 and 54 are positioned in the third arrangement shown.

[0091] Refer again Figures 5A-5B This paper illustrates a position sensor 1000 for sensing the position of objects within an operating room, according to some embodiments. During surgery, the position sensor 1000 senses the position of the patient, the position of operating room fixtures, the position of system modules, and the position of operating room personnel. Figures 5A-5B Position sensors 1000 are shown arranged at the four corners, at a certain height so as to be within the patient's line of sight. Figure 5B Only two of the four sensors 1000 are visible in the system. The position sensors generate information indicating the location of patient anatomical features, such as body surfaces, the patient's skeletal posture on the operating table, and the relative posture of the surgical system to the patient's anatomy. In some embodiments, the position sensors include laser scanning sensors that scan the operating room and its contents to determine the location of items within the room. In some embodiments, the position sensor 1000 includes an infrared (IR) scanning sensor. For example, in a computer-readable storage device (described more fully below), a processor 58 records information indicating the sensed patient location, operating room layout, and system module locations. In some embodiments, operators are equipped with RFID tags that can be tracked to identify personnel within the operating room.

[0092] Figure 14 This is an illustrative drawing illustrating a storage map 1002 in a computer-readable storage device 1004 according to some embodiments. The storage map 1002 includes a first information structure 1006 indicating instances of previously performed surgical procedures. A second information structure 1008 associates the surgical procedure with patient anatomical features, patient placement on the operating table, and the position of mechanical support arms during the surgical procedure. A third information structure 1010 associates the surgical procedure with operating room layout features, movement patterns of operating room personnel during the surgical procedure, and the position of surgical system modules during the surgical procedure. A first rule information structure 1012 associates patient anatomical feature information and patient position information with mechanical setup control signals. A second rule information structure 1014 associates operating room layout information and operating room personnel information with system module position information. Figure 15 The accompanying drawings are illustrative examples illustrating a second information structure 1008 included within a map 1002 according to some embodiments. Figure 16The accompanying drawings are illustrative examples illustrating a third information structure 1010 included within a map 1002 according to some embodiments.

[0093] Figure 17 This is an illustrative drawing showing an example first rule information structure 1012 that associates patient anatomical features and patient position information with mechanical support arm position control signals according to some embodiments. The first rule is developed based on data from previous surgeries represented in a first data information structure 1006 and a second data information structure 1008. The first rule correlates patterns of patient anatomy and patient position with control signals used to control the position setting of the mechanical support arm 76 for use during a first type of surgical procedure.

[0094] More specifically, for the first type of surgical procedure in the example, the first rule information structure 1012 will include the surgical procedure signature (SigST1...SigST). l ), patient anatomical signature (SigAn1...SigAn) n ) and patient location (SigPos1...SigPos m Signature and robotic arm position control signal (CNTL) ARM1 ...CNTL ARMP The patient anatomical signature is associated with various medical records. In some embodiments, the patient anatomical signature includes a multidimensional vector. In some embodiments, the patient anatomical signature indicates patient health record information, such as the patient's weight, height, age, demographic information, and preoperative three-dimensional images, such as computed tomography (CT) or magnetic resonance imaging (MRI). Additionally, the patient anatomical signature can include surgical planning information (e.g., surface scans of the patient's body using a structured light imaging system) or other depth sensing techniques (e.g., time-of-flight and radiofrequency imaging) and a geometric or physical-based model generated based on the aforementioned information (which can be used to assess deformation of the patient's body in a given configuration). In some embodiments, the patient position signature includes a multidimensional vector. In some embodiments, scanned patient position information is used to determine patient position information, such as pneumoperitoneum pressure, measurements from pressure pads on the operating table, and a three-dimensional model of the patient's body on the operating table obtained through intraoperative imaging techniques (including cone-beam CT, intraoperative CT, or MRI, surface scanning techniques, or multi-view surface reconstruction from multiple cameras). According to some embodiments, different robotic arm position control signals are associated with different combinations of the patient anatomical signature and the patient position signature. The patient position information serves as coarse information indicating the position of the patient's body within the operating room. Patient anatomical information serves as detailed information indicating the location of specific anatomical features of the patient's body.

[0095] According to some embodiments, machine learning techniques can be used to generate first rules corresponding to the patient anatomical signature and the patient position signature. More specifically, for example, a classifier can be used in conjunction with expertise to associate the patient anatomical signature and the patient position signature with robotic arm position control signals. Based on, for example, input from an expert surgeon, surgical data within a first data information structure 1006 and a second data information structure 1008 is evaluated to determine the appropriate position of the robotic arm 72, including the positions of its segments 72-1, 72-2, and 72-3 and the positions of its sub-segments 101-104. Control signals for implementing the determined robotic arm setup position are associated with the patient anatomical signature and the patient position signature in a first rule information structure 1012. According to some embodiments, the patient anatomical signature and the patient position signature can be combined to produce a combined signature corresponding to the control signals.

[0096] Figure 18 This is an illustrative flowchart illustrating process 1802, which, according to some embodiments, generates control signals based at least in part on patient anatomy information, patient location information, and surgical procedure type. According to some embodiments, a computer processor 58 is configured to execute process 1802. During the setup of the surgical procedure, rule block 1804 and matching block 1814 determine whether to trigger the emission of a control signal based on patient anatomy information, patient location information, surgical procedure type, and rules from a first rule information structure 1012.

[0097] More specifically, during the setup of the mechanical support arm 72 for the surgical procedure using system 10, rule block 1804 receives patient anatomy information 1806 that can be input to processor 58. Rule block 1804 receives patient position information obtained using position sensor 1000 and input to processor 58. Rule block 1804 receives surgery type information, which may include, for example, incision pattern information input to processor 58.

[0098] Furthermore, during the surgical procedure, the control signal rule storage block 1812 provides patient anatomical signature information, patient location signature information, and surgical procedure type information from the first rule block portion 1012A of the first control signal rule information structure 1012 to rule block 1804. Rule block 1804 compares the patient anatomical features, patient location, and surgical procedure type information provided by blocks 1806, 1808, and 1810, respectively, with the associated patient anatomical signature, patient location signature, and surgical system type information from the first rule block portion 1012A. It should be understood that in some embodiments, the computer processor 58 is configured to transform the patient anatomical feature information, patient location information, and surgical procedure type information obtained through system 10 into a format suitable for comparison with the signature and status information from the first portion 1012A of the control signal rule information structure 1012. In particular, in some embodiments, the raw patient anatomical features / patient location / surgical procedure type are processed to derive a classification (signal / probability), and then the classification is looked up in a table to determine the position of the robotic arm based on the determined classification.

[0099] Decision module 1806 determines whether a match exists between the patient anatomical features, patient location, and surgical procedure type information provided on one side and the rule information from the first rule block portion 1012A on the other side. It should be understood that, in machine learning embodiments, the match is determined based on the range of similarity between the patient anatomical features, patient location, and surgical procedure type and the rule information. Therefore, for example, combinations of patient anatomical features, patient location, and surgical procedure type within a certain threshold limit of a rule are determined to match that rule.

[0100] In response to determining a match between the combination of patient anatomical features, patient location, and surgical procedure type information and a rule, block 1814 transmits a control signal from the second rule portion 1012B of the control signal rule information structure 1012, corresponding to a matching rule. For example, in response to determining that the patient anatomical features, patient location, and surgical procedure type information received during the surgical procedure matches SigAn2, SigPos1, and SigSur1 of the first portion 1012A, block 1816 transmits a signal CNTL from the second portion 1012B of the control signal information structure 1012. ARMS1 The transmitted control signal has a value corresponding to a combination of anatomical location, patient location, and surgical procedure signature associated with it in the signature of information structure 1012. The control signal causes the mechanical support arm 72 to move to a position corresponding to these associated signatures. More specifically, according to some embodiments, the transmitted control signal has a value that causes the rotational positions of the first to third segments 72-1 to 27-3 to be controlled to move about corresponding axes 91-93 and causes the first to fourth sub-segments 101-104 to be controlled to rotate about corresponding axes 111-114.

[0101] Figure 19 This is an illustrative drawing showing an example second rule information structure 1014 that associates operating room layout information and operating room personnel information with system module information according to some embodiments. The second rule is developed based on data from previous surgeries represented in a first data information structure 1006 and a third data information structure 1010. The second rule correlates the patterns of operating room layout and surgical personnel activity with system module location information, which can be presented on a display 60 to provide guidance on the placement of surgical system modules during the setup of surgical procedures.

[0102] For example, the first type of surgical procedure, the second rule information structure 1014 will include the surgical type signature (SigST1...SigST). l Operating room layout signature (SigLT1...SigLT) n ) and personnel activity signatures (SigPA1...SigPA) m ) and the system module layout information displayed on the display screen 60, displaying image identifiers (ImageID1...ImageID) P The operating room layout signature is associated with various elements. In some embodiments, the operating room layout signature includes a multi-dimensional vector. In some embodiments, the operating room layout signature indicates the location of fixtures, room dimensions and mounting points, and models of hinged devices such as lights mounted on joists. In some embodiments, the personnel activity signature includes a multi-dimensional vector. In some embodiments, the recorded personnel activity information includes personnel roles, spatiotemporal movement trajectories, and action history. According to some embodiments, different system module locations are associated with different combinations of the operating room layout signature and the operating room personnel activity signature. According to some embodiments, machine learning techniques can be used to generate second rules. More specifically, for example, a classifier can be used in conjunction with expertise to correlate the operating room layout signature and the operating room personnel activity signature with the system module locations.

[0103] Figure 20 This is an illustrative flowchart illustrating a process 2002 for generating computer-displayed image information indicating the location of a recommended system module, based at least in part on operating room layout information, surgical personnel activity information, and surgical procedure type information, according to some embodiments. A computer processor 58 is configured to execute process 2002 according to some embodiments. During the setup of the surgical procedure, rule block 2004 and matching block 2014 determine whether to display an image indicating the recommended location of the surgical system module based on patient anatomy information, patient location information, surgical procedure type, and rules from a second rule information structure 1014.

[0104] More specifically, during the setup of surgical system modules 16, 24, 30, and 54 for the surgical procedure, rule block 2004 receives operating room layout information 2006 that can be input to processor 58. Rule block 2004 receives operating room personnel activity information input to processor 58. Rule block 2004 receives surgery type information, which may include, for example, incision pattern information input to processor 58.

[0105] Furthermore, during the surgical procedure, image rule storage block 2012 provides patient operating room layout signature information, personnel activity signature information, and surgical procedure type information from the second rule block portion 1014A of the second rule information structure 1014 to rule block 2004. Rule block 2004 compares the operating room layout, surgical personnel activity information, and surgical procedure type information provided by blocks 2006, 2008, and 2010, respectively, with the associated operating room layout signature, personnel activity signature, and surgical system type information from the second rule block portion 1014A. It should be understood that in some embodiments, computer processor 58 is configured to transform the operating room layout information, surgical personnel activity information, and surgical procedure type information obtained through system 10 into a format suitable for comparison with signature and status information from the first portion 1014A of the control signal rule information structure 1014.

[0106] Decision module 2006 determines whether a match exists between the operating room layout, personnel activities, and surgical procedure type information provided on one side and the rule information from the second rule block portion 1014A on the other side. It should be understood that, in machine learning embodiments, the match is determined based on the range of similarity between the operating room layout, personnel activities, and surgical procedure type and the rule information. Therefore, for example, combinations of operating room layout, personnel activities, and surgical procedure types within a certain threshold limit of a rule are determined to match that rule.

[0107] In response to determining a match between a combination of operating room layout, personnel activity, and surgical procedure type information and a rule, block 2014 transmits an image signal from the second rule portion 1014B of the corresponding matching rule in the second rule information structure 1014. For example, in response to determining that the operating room layout, personnel activity, and surgical procedure type information received during a surgical procedure matches SigOR2, SigAct1, and SigSur1 of the first portion 1014A, block 2016 transmits a signal IMGA1 from the second portion 1014B of the information structure 1014. According to some embodiments, an image, such as a representation, is generated on the display screen 60. Figures 13A-13C One of the images shows the layout of the operating room and the location of the system modules.

[0108] Although illustrative embodiments have been shown and described, a wide range of modifications, alterations, and substitutions are contemplated in the foregoing disclosure, and in some cases, some features of the embodiments may be employed without the need for corresponding use of other features. For example, in some embodiments, processor 58 is coupled to a memory device such as storage device 1004, which includes a set of instructions executable on processor 58 to cause processor 58 to perform operations. In some embodiments, operations include determining patient location information during setup of a surgical system for a surgical procedure. Operations further include determining a match between the patient location information determined during setup and a corresponding patient location signature. Operations also include transmitting a support arm control signal within the surgical system during setup, the signal corresponding to the matched corresponding patient location signature.

[0109] Furthermore, in some embodiments, processor 58 is coupled to a memory device such as storage device 1004, which includes a set of instructions executable on processor 58 to cause processor 58 to perform operations including determining operating room layout information during setup of a surgical system for a surgical procedure. The operations also include determining a match between the operating room layout information determined during setup and a corresponding operating room layout signature. The operations further include generating an image representing the location of a surgical system module that corresponds to a matching corresponding operating room layout signature.

[0110] Those skilled in the art will recognize many variations, alternatives, and modifications. Therefore, the scope of this disclosure should be limited only by the appended claims, and the claims should be interpreted broadly in accordance with the scope of the embodiments disclosed herein.

Claims

1. A method for use with a remote surgical system, the method comprising: Operating room layout information is determined during the setup of the surgical system used in the surgical procedure; Determine the match between the operating room layout information determined during the setup and the corresponding operating room layout signature; Personnel activity information is determined during the setup of the surgical system used in the surgical procedure; Determine the match between the identified personnel activity information and the corresponding personnel activity signatures; During the setup of the surgical system used for the surgical procedure, surgical procedure type information is determined; Determine the match between the identified surgical procedure type information and the corresponding surgical procedure signature; as well as An image is generated representing the location of the surgical system module corresponding to the combination of the matching operating room layout signature, the matching personnel activity signature, and the matching surgical procedure signature.

2. The method according to claim 1, Determining the operating room layout information includes scanning with a laser position sensor.

3. The method according to claim 1, Determining the operating room layout information includes scanning with an infrared position sensor.

4. The method according to claim 1, further comprising: For each occurrence of the setup of one or more instances of the surgical system used to perform surgical procedures, operating room layout information and system module locations are recorded during the setup. as well as Based at least in part on the recorded operating room layout information and the recorded system module location information, the corresponding operating room layout signature is determined to be associated with the corresponding operating room layout.

5. The method according to claim 1, further comprising: For each occurrence of a setup of one or more instances of the surgical system used to perform a surgical procedure, operating room layout information is recorded during the setup, personnel activity information is recorded during the setup, and system module location information is recorded during the setup. as well as Based at least in part on the recorded operating room layout information, the recorded personnel activity information, and the recorded system module location information, the corresponding combination of the corresponding operating room layout signature and the corresponding personnel activity signature is determined to be associated with the corresponding system module.

6. A surgical system comprising surgical instruments and surgical instrument actuators, comprising: processor; A memory device that holds a set of instructions executable on the processor to enable the surgical system to perform operations including: Operating room layout information is determined during the setup of the surgical system used in the surgical procedure; Determine the match between the operating room layout information determined during the setup and the corresponding operating room layout signature; Personnel activity information is determined during the setup of the surgical system used in the surgical procedure; Determine the match between the identified personnel activity information and the corresponding personnel activity signatures; During the setup of the surgical system used for the surgical procedure, surgical procedure type information is determined; Determine the match between the identified surgical procedure type information and the corresponding surgical procedure signature; as well as An image is generated representing the location of the surgical system module corresponding to the combination of the matching operating room layout signature, the matching personnel activity signature, and the matching surgical procedure signature.

7. The system according to claim 6, Determining the operating room layout information includes scanning with a laser position sensor.

8. The system according to claim 6, Determining the operating room layout information includes scanning with an infrared position sensor.

9. The system according to claim 6, further comprising: For each occurrence of the setup of one or more instances of the surgical system used to perform surgical procedures, operating room layout information and system module locations are recorded during the setup. as well as Based at least in part on the recorded operating room layout information and the recorded system module location information, the corresponding operating room layout signature is determined to be associated with the corresponding operating room layout.

10. The system of claim 6, further comprising: For each occurrence of a setup of one or more instances of the surgical system used to perform a surgical procedure, operating room layout information is recorded during the setup, personnel activity information is recorded during the setup, and system module location information is recorded during the setup. as well as Based at least in part on the recorded operating room layout information, the recorded personnel activity information, and the recorded system module location information, the corresponding combination of the corresponding operating room layout signature and the corresponding personnel activity signature is determined to be associated with the corresponding system module.

Images