System and method for detecting physical contact of a surgical instrument with patient tissue

Abstract

A tissue contact detection system tracks a temperature of a surgical instrument associated with a surgical system used in a surgical procedure over time during the surgical procedure. The system determines, based on the tracked temperature of the surgical instrument, that the temperature of the surgical instrument changed from a first temperature to a second temperature that is at least a predetermined amount from the first temperature, and determines, based on the determination that the temperature of the surgical instrument changed from the first temperature to the second temperature, that the surgical instrument is in physical contact with patient tissue. In response to determining that the surgical instrument is in physical contact with the patient tissue, the system performs a mitigation operation configured to mitigate the physical contact of the surgical instrument with the patient tissue.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 890,536, filed August 22, 2019, entitled “Systems and methods for detecting physical contact of surgical instruments with patient tissue,” the contents of which are incorporated herein by reference in their entirety. Background Technology

[0003] Surgical instruments that come into prolonged physical contact with patient tissue can damage it. For example, surgical instruments that operate at temperatures higher than the patient's tissue (e.g., endoscopes) may burn the tissue after prolonged contact. As another example, surgical instruments may damage delicate tissue after prolonged contact.

[0004] However, in some cases, surgeons may find it difficult to determine when surgical instruments make physical contact with patient tissue. For example, during minimally invasive surgical procedures, surgeons may not be able to determine from endoscopic views of the surgical site when surgical instruments make physical contact with patient tissue at or near the surgical site. Summary of the Invention

[0005] The following description presents a simplified overview of one or more aspects of the methods and systems described herein to provide a basic understanding of these aspects. This overview is not a comprehensive summary of all anticipated aspects, and is neither intended to identify key or decisive elements of all aspects, nor to depict the scope of any or all aspects. Its sole purpose is to present, in a simplified form, some concepts of one or more aspects of the methods and systems described herein as a prelude to the more detailed descriptions presented below.

[0006] An exemplary system may include a memory storing instructions and a processor communicatively coupled to the memory and configured to execute instructions to track over time the temperature of a surgical instrument associated with a surgical system used in the surgical procedure during a surgical procedure; determine, based on the tracked temperature of the surgical instrument, a change in the temperature of the surgical instrument from a first temperature to a second temperature that changes by at least a predetermined amount from the first temperature; determine, based on the determination of the change in the temperature of the surgical instrument from the first temperature to the second temperature, that the surgical instrument is in physical contact with patient tissue; and, in response to determining that the surgical instrument is in physical contact with patient tissue, perform a mitigation operation configured to relieve the physical contact between the surgical instrument and patient tissue.

[0007] Another exemplary system may include: a temperature sensor configured to detect the temperature of a surgical instrument included in a surgical system for surgical procedures during surgery; and a processor communicatively coupled to the temperature sensor and configured to execute instructions to track the temperature of the surgical instrument detected by the temperature sensor over time during surgery; determine, based on the tracked temperature of the surgical instrument, a change in the temperature of the surgical instrument from a first temperature to a second temperature that changes by at least a predetermined amount from the first temperature; determine, based on the determination of the change in the temperature of the surgical instrument from the first temperature to the second temperature, that the surgical instrument is in physical contact with patient tissue; and, in response to determining that the surgical instrument is in physical contact with patient tissue, perform a mitigation operation configured to relieve the physical contact between the surgical instrument and the patient tissue.

[0008] An exemplary method may include, during surgical procedures, tracking over time the temperature of a surgical instrument included in a surgical system for the surgical procedure by a tissue contact detection system; determining, based on the tracked temperature of the surgical instrument, a change in the temperature of the surgical instrument from a first temperature to a second temperature that changes by at least a predetermined amount from the first temperature; determining, based on the determination that the temperature of the surgical instrument has changed from the first temperature to the second temperature, that the surgical instrument is in physical contact with patient tissue; and, in response to determining that the surgical instrument is in physical contact with patient tissue, performing a mitigation operation by the tissue contact detection system, the mitigation operation being configured to relieve the physical contact between the surgical instrument and the patient tissue. Attached Figure Description

[0009] The accompanying drawings illustrate various embodiments and are part of the specification. The illustrated embodiments are merely examples and do not limit the scope of this disclosure. Throughout the drawings, the same or similar reference numerals denote the same or similar elements.

[0010] Figure 1 An exemplary tissue contact detection system based on the principles described herein is illustrated.

[0011] Figure 2 The illustration shows a functional diagram of an exemplary environment in which surgical procedures are performed according to the principles described herein.

[0012] Figure 3 An exemplary cannula is illustrated for use with surgical instruments in accordance with the principles described herein.

[0013] Figure 4 An exemplary surgical instrument with multiple temperature sensors is illustrated, based on the principles described herein.

[0014] Figure 5 An exemplary temperature profile of a surgical instrument based on tracked temperature data, according to the principles described herein, is illustrated.

[0015] Figure 6-8 An exemplary notification of tissue contact is illustrated, which can be presented by means of a display device based on the principles described herein.

[0016] Figure 9 An exemplary method for performing mitigation operations based on the principles described herein is illustrated.

[0017] Figure 10 An exemplary computer-assisted surgical system based on the principles described herein is illustrated.

[0018] Figure 11 An exemplary method for detecting physical contact between surgical instruments and patient tissues based on the principles described herein is shown.

[0019] Figure 12 An exemplary computing device based on the principles described herein is illustrated. Detailed Implementation

[0020] This document describes a tissue contact detection system and method. As will be described in more detail below, the tissue contact detection system can track the temperature of surgical instruments included in a surgical system used for surgical procedures over time during surgical procedures. Based on the tracked temperature of the surgical instruments, the tissue contact detection system can determine a change in the temperature of the surgical instruments from a first temperature to a second temperature that changes by at least a predetermined amount from the first temperature. Based on the determination of the temperature change of the surgical instruments from the first temperature to the second temperature, the tissue contact detection system can determine that the surgical instruments are in physical contact with patient tissue. In response to determining that the surgical instruments are in physical contact with patient tissue, the tissue contact detection system can perform a mitigation operation configured to relieve the physical contact between the surgical instruments and the patient tissue.

[0021] To illustrate, during minimally invasive surgical procedures, the endoscope included in the computer-assisted surgical system can be used to illuminate the surgical area within the patient's body and provide an image of the surgical area for the surgeon to view and use while performing the surgical procedure. During the surgical procedure, the endoscope can operate at a temperature higher than the nominal temperature of the patient's tissue (e.g., 37°C), for example, 50°C. When the axis of the endoscope is in physical contact with the patient's tissue, the patient's tissue acts as a conductive heat sink, and the temperature of the endoscope decreases (e.g., from 50°C to 47°C).

[0022] During surgical procedures, a tissue contact detection system can track the temperature of the endoscope over time and determine temperature changes (e.g., from 50°C to 47°C). The system can determine that this temperature change exceeds a predetermined amount (e.g., 2°C) and thus determine that the endoscope is in physical contact with the patient's tissue. To mitigate physical contact between the surgical instrument and the patient's tissue, the tissue contact detection system can present or guide the surgical system to present notifications of this physical contact (e.g., visual notifications, warning messages, auditory notifications, etc.). The surgeon can respond to the notification by removing the endoscope from physical contact with the patient's tissue. Additionally or alternatively, the tissue contact detection system can reduce or guide the surgical system to reduce the intensity of illumination emitted from the endoscope and / or turn off auxiliary illumination (e.g., fluorescent excitation illumination).

[0023] The systems and methods described herein offer a variety of benefits. For example, they can reduce the risk of damage to patient tissues due to prolonged physical contact with surgical instruments. Furthermore, the determination of physical contact between surgical instruments and patient tissues is highly reliable because it is based on detected temperature changes in the surgical instruments, which are detectable, have low noise, and are unaffected by electrical noise in the operating environment. In addition, the systems and methods described herein are capable of detecting physical contact between surgical instruments and patient tissues located outside the endoscopic view. These and other benefits of the systems and methods described herein will become apparent in the following description.

[0024] Figure 1 An exemplary tissue contact detection system 100 (“System 100”) is illustrated, which can be configured to determine physical contact between surgical instruments and patient tissue. System 100 can be included in, implemented by, or connected to any surgical system or other computing system described herein. For example, System 100 can be implemented by a computer-assisted surgical system. As another example, System 100 can be implemented by a separate computing system communicatively coupled to a computer-assisted surgical system.

[0025] As shown in the figure, system 100 includes, but is not limited to, storage facility 102 and processing facility 104, which are selectively and communicatively coupled to each other. Facilities 102 and 104 may each include or be implemented by hardware and / or software components (e.g., processor, memory, communication interface, instructions stored in memory for execution by the processor, etc.). For example, facilities 102 and 104 may be implemented by any component of a computer-assisted surgical system. In some examples, facilities 102 and 104 may be distributed among multiple devices and / or multiple locations that may serve a particular implementation.

[0026] Storage facility 102 may maintain (e.g., store) executable data used by processing facility 104 to perform any of the operations described herein. For example, storage facility 102 may store instructions 106 that can be executed by processing facility 104 to perform any of the operations described herein. Instructions 106 may be implemented by any suitable application, software, code, and / or other instance of executable data. Storage facility 102 may also maintain any data received, generated, managed, used, and / or transmitted by processing facility 104.

[0027] Processing facility 104 may be configured to perform (e.g., execute instructions 106 stored in storage facility 102 to perform) various operations associated with detecting physical contact between surgical instruments and patient tissue. For example, processing facility 104 may be configured to track the temperature of surgical instruments included in a surgical system used for surgical procedures over time during surgery. Based on the tracked temperature of the surgical instruments, processing facility 104 may determine that the temperature of the surgical instruments changes from a first temperature to a second temperature that changes by at least a predetermined amount from the first temperature. Based on the determination that the temperature of the surgical instruments changes from the first temperature to the second temperature, processing facility 104 may determine that the surgical instruments are in physical contact with patient tissue. In response to determining that the surgical instruments are in physical contact with patient tissue, processing facility 104 may perform or direct the surgical system to perform mitigation operations configured to alleviate the physical contact between the surgical instruments and patient tissue. These and other operations that may be performed by processing facility 104 are described herein. In the following description, any reference to operations performed by system 100 may be understood as being performed by processing facility 104 of system 100.

[0028] As mentioned, system 100 can be configured to track the temperature of surgical instruments used in a surgical procedure over time during the procedure. Figure 2 The illustration depicts a functional diagram of an exemplary environment in which a surgical procedure is performed. As shown, surgical instruments 202 are used to perform one or more procedures on a patient during a surgical procedure. As a result, surgical instruments 202 may be positioned at different times near patient tissue 204 located in the surgical area associated with the patient. Surgical procedures may include any procedure in which manual and / or instrumental techniques are used to manipulate the patient to investigate, diagnose, and / or treat a patient's physical condition. For example, surgical instruments 202 may be used to perform minimally invasive surgical procedures on tissue within a patient's body. In other examples, surgical instruments 202 may be used to perform open surgical procedures, such as when a portion of the surgical area (e.g., the tissue being manipulated) is inside the patient and another portion of the surgical area (e.g., the space around the tissue where one or more surgical instruments may be placed) is outside the patient's body.

[0029] Surgical instrument 202 can be implemented using any suitable surgical tool (e.g., a tool with tissue interaction capabilities), medical instrument, monitoring device (e.g., an endoscope), sensing device (e.g., a force-sensing surgical instrument), diagnostic device, or similar tool that can be used in surgical procedures (e.g., a computer-assisted surgical procedure in which the surgical instrument is at least partially inserted into the patient and manipulated to perform a minimally invasive surgical procedure on the patient). Surgical instrument 202 is associated with (e.g., connected to, integrated into, or implemented by) surgical system 206. In some examples, surgical system 206 is connected to, integrated into, or implemented by a computer-assisted surgical system that utilizes robotics and / or teleoperation technology to control surgical instrument 202 to perform surgical procedures (e.g., minimally invasive surgical procedures). Exemplary computer-assisted surgical systems are described in more detail below. In some examples, system 100 is implemented entirely by surgical instrument 202 and / or surgical system 206. As another example, system 100 may be implemented wholly or partially by a separate computing system communicatively coupled to surgical instrument 202 and / or surgical system 206 (e.g., by means of a network).

[0030] Surgical instrument 202 includes a temperature sensor 208 configured to measure the temperature of surgical instrument 202. Temperature sensor 208 can be implemented by any suitable temperature sensing device, such as, but not limited to, thermocouples, resistance thermometers, thermistors, semiconductor-based temperature sensors, fiber optic temperature probes, etc. In some examples, temperature sensor 208 may be connected to system 100, integrated with system 100, or included in system 100.

[0031] The temperature sensor 208 can be positioned anywhere on the surgical instrument 202, which can be adapted to a particular implementation. In some examples, such as Figure 2 As shown, temperature sensor 208 is positioned on the axial wall 210 of surgical instrument 202. Alternatively, temperature sensor 208 may be positioned on the distal end 212 of surgical instrument 202. In some examples, temperature sensor 208 is configured to measure the temperature of the outer surface of surgical instrument 202. Therefore, temperature sensor 208 may be positioned in contact with the outer surface of surgical instrument 202. For example, temperature sensor 208 may be positioned on the exterior of axial wall 210. In alternative examples, temperature sensor 208 may be positioned inside surgical instrument 202. In some examples where temperature sensor 208 is not in contact with the outer surface of surgical instrument 202, the temperature value measured by temperature sensor 208 may be correlated with the temperature of the outer surface of surgical instrument 202.

[0032] like Figure 2As shown, temperature sensor 208 is integrated with surgical instrument 202. For example, temperature sensor 208 is built into surgical instrument 202, and temperature data generated by temperature sensor 208 is transmitted to surgical system 206 via surgical instrument 202 (e.g., via wiring located within surgical instrument 202). Alternatively, temperature sensor 208 may be independent of surgical instrument 202. Figure 3 An exemplary embodiment with the temperature sensor separated from the surgical instrument is illustrated. As shown, the temperature sensor 302 and wiring 304 are integrated into a sleeve 306. The sleeve 306 is formed of a material with high thermal conductivity, such as a metal, a polymer with thermally conductive additives, etc. The sleeve 306 is configured to be positioned on the axis of the surgical instrument (e.g., around the axial wall 210 of the surgical instrument 202). In some examples, the sleeve 306 can be selectively removed from the surgical instrument 202. In this way, the temperature sensor 302 can be used with a variety of different surgical instruments as needed. In other examples, the sleeve 306 can be permanently fixed to the surgical instrument 202. Wiring 304 is configured to transmit temperature data to a computing system (e.g., to the surgical system 206). The sleeve 306 is capable of tracking the temperature of conventional surgical instruments that do not include an integrated temperature sensor. Furthermore, the sleeve 306 facilitates and improves temperature tracking for surgical instruments with low thermal conductivity (e.g., surgical instruments with non-metallic axes).

[0033] In some examples, the temperature sensor may be integrated with the surgical instrument 202 (e.g., on the outer surface of the shaft wall 210), and a thermally conductive sleeve (similar to sleeve 306) may be provided on the surgical instrument 202 to improve the detection of tissue contact.

[0034] Refer again Figure 2 The surgical instrument 202 may include one or more other sensors (not shown), such as displacement transducers, orientation sensors, position sensors, etc., for generating kinematic information (hereinafter referred to as "surgical instrument sensors"). Kinematic information may include posture (e.g., position and / or orientation), motion (e.g., velocity, direction, acceleration, etc.), state (e.g., open, closed, retracted, etc.), and / or other attributes of the surgical instrument 202. System 100 and / or surgical system 206 may be configured to use kinematic information to track (e.g., determine posture, motion, and / or state) and / or control the surgical instrument 202. The surgical instrument 202 may also include other sensors configured to generate additional information suitable for a particular implementation. As will be explained in more detail below, system 100 may also be configured to track the operation of the surgical instrument 202 based on parameters sensed by the surgical instrument sensors and determine whether the surgical instrument 202 is in physical contact with patient tissue 204 based on the tracked operation.

[0035] The foregoing embodiments have described the use of a single temperature sensor. In alternative embodiments, the surgical instrument 202 and / or cannula 306 may include multiple temperature sensors configured to measure the temperature of the surgical instrument 202 at different locations on the surgical instrument 202. For example, as Figure 4 As shown, the surgical instrument 402 includes a plurality of temperature sensors 404 (e.g., temperature sensors 404-1, 404-2, and 404-3), which are positioned at different locations along the axis 406 of the surgical instrument 402. For example, temperature sensor 404-1 is located approximately 5 mm from the distal end of the surgical instrument 402, temperature sensor 404-2 is located approximately 25 mm from the distal end of the surgical instrument 402, and temperature sensor 404-3 is located approximately 40 mm from the distal end of the surgical instrument 402. Although in Figure 4 Three temperature sensors 404 are shown, but the surgical instrument 402 may include fewer or more temperature sensors to suit a particular implementation. Furthermore, the temperature sensors 404 may be positioned anywhere on the surgical instrument 402 to suit a particular implementation.

[0036] As described above, system 100 can be configured to track the temperature of surgical instruments (e.g., surgical instruments 202 and / or 402). Tracking can be performed in any suitable manner. In some examples, tracking includes collecting and / or storing temperature data representing the measured temperature of the surgical instruments. For example, system 100 may periodically (e.g., every second, every 5 seconds, etc.) collect temperature data from temperature sensors (e.g., temperature sensor 208, temperature sensor 302, and / or temperature sensor 404) and store the collected temperature data (e.g., in storage facility 102). In examples where the surgical instruments include multiple temperature sensors, tracking includes collecting and / or storing temperature data from each temperature sensor. In some examples, tracking may also include processing the collected temperature data, such as noise removal, signal amplification, temperature data conversion (e.g., converting a measured temperature value of the internal temperature of surgical instrument 202 to a temperature value of the external surface of surgical instrument 202), generating temperature profiles, etc.

[0037] Based on the temperature of the tracked surgical instruments, system 100 can detect the physical contact between the surgical instruments and the patient's tissue. Reference will now be made to... Figure 5 Describe various examples of the physical contact between surgical instruments and patient tissues based on temperature detection of tracked surgical instruments. Figure 5 An exemplary temperature profile 500 of a surgical instrument based on tracked temperature data is illustrated. Temperature profile 500 depicts the temperature of the tracked surgical instrument over time.

[0038] In some examples, system 100 is configured to determine physical contact between the surgical instrument and patient tissue when system 100 determines that the temperature of the surgical instrument has changed from a first temperature to a second temperature that has changed by at least a predetermined amount from the first temperature. In some examples, the first temperature and / or the second temperature are each instantaneous temperature values. For example, the first temperature could be the most recent highest temperature, such as the highest temperature that occurred during a predetermined time interval (e.g., 5 seconds). Similarly, the second temperature could be the most recent lowest temperature, such as the lowest temperature that occurred during a predetermined time interval. Alternatively, the first temperature and / or the second temperature could each be the average of multiple temperature values ​​sampled during a predetermined time interval (e.g., 5 seconds). In yet another example, which will be explained below, the first temperature and / or the second temperature are each steady-state temperatures (e.g., the temperature of the surgical instrument when its temperature is at a steady state).

[0039] The predetermined amount can be set in any suitable manner and is typically set as a value indicating tissue contact. For example, the predetermined amount can be a predetermined temperature difference (e.g., 5°C) or a percentage of a first temperature (e.g., 10%). If the temperature change is at least equal to the predetermined amount, system 100 determines that the surgical instrument is in physical contact with the patient's tissue. However, if the temperature change is less than the predetermined amount, system 100 does not determine that the surgical instrument is in physical contact with the patient's tissue.

[0040] To illustrate, such as Figure 5 As shown, the temperature of the surgical instrument is approximately 55.7°C at time t1 and approximately 44.5°C at time t2. If the predetermined temperature difference is 7°C, system 100 infers that the temperature drop is due to physical contact between the surgical instrument and the patient's tissue, since the temperature change is 11.2°C, which is greater than the predetermined temperature difference of 7°C. Therefore, system 100 determines that the surgical instrument is in physical contact with the patient's tissue at time t2. Similarly, the temperature of the surgical instrument is approximately 51.4°C at time t4 and approximately 42.0°C at time t5. Therefore, system 100 determines that the surgical instrument is in physical contact with the patient's tissue at time t5, since the temperature change is 9.4°C, which is greater than the predetermined temperature difference of 7°C. On the other hand, the temperature of the surgical instrument is approximately 50.0°C at time t7 and approximately 44.3°C at time t8. Therefore, system 100 does not determine that the surgical instrument is in physical contact with the patient's tissue at time t8, since the temperature change is 5.7°C, which is less than the predetermined temperature difference of 7°C.

[0041] In some examples, system 100 determines that the surgical instrument is in physical contact with the patient tissue only when the temperature of the surgical instrument changes in the direction of the patient tissue temperature (e.g., nominal body temperature of 37°C). In other words, if the first temperature of the surgical instrument is higher than the temperature of the patient tissue, then only a decrease in the temperature of the surgical instrument (i.e., a second temperature of the surgical instrument lower than the first temperature) indicates physical contact with the patient tissue. On the other hand, if the first temperature of the surgical instrument is lower than the temperature of the patient tissue, then only an increase in the temperature of the surgical instrument (i.e., a second temperature of the surgical instrument higher than the first temperature) indicates physical contact with the patient tissue.

[0042] In additional or alternative examples, system 100 may determine the physical contact between the surgical instrument and patient tissue based on the determination that a first temperature and / or a second temperature of the surgical instrument is in a steady state. System 100 may determine that the temperature of the surgical instrument is in a steady state in any suitable manner. In some examples, the temperature of the surgical instrument is considered to be in a steady state when the temperature of the surgical instrument varies by no more than a predetermined steady-state amount within a predetermined steady-state time period. The predetermined steady-state amount may be any suitable quantity and may be specified as a temperature difference (e.g., ±2°C) or a percentage (e.g., ±5%). The predetermined steady-state time period may be any suitable time period (e.g., 15 seconds).

[0043] To illustrate, such as Figure 5 As shown, the surgical instrument is initialized at time t0 (e.g., connected to the surgical system, docked, opened, operated initially, etc.) and operates during the initialization phase from time t0 to time t3. During the initialization phase, the temperature of the surgical instrument rises and fluctuates (e.g., due to manual operation of the surgical instrument, navigation of the surgical instrument to the surgical area within the patient's body, activation of optical illumination, movement of the surgical instrument, etc.). Therefore, the temperature of the surgical instrument does not reach a steady state between time t0 and time t3. Therefore, although the temperature change from time t1 to time t2 may be greater than a predetermined amount, system 100 does not determine that the surgical instrument is in physical contact with the patient's tissue because the temperature of the surgical instrument has not yet reached a steady state. However, the temperature of the surgical instrument reaches a steady state between time t3 and time t4 and between time t5 and time t6. Therefore, if the second temperature changes from the first temperature by at least a predetermined amount, system 100 can determine that the surgical instrument is in physical contact with the patient's tissue between time t5 and time t6.

[0044] In some examples, the determination of physical contact between the surgical instrument and the patient's tissue is conditional at least on the second temperature of the surgical instrument being in a steady state. For example, system 100 does not determine physical contact between the surgical instrument and the patient's tissue at any time between time t1 and time t3 because the temperature of the surgical instrument is not in a steady state. On the other hand, assuming the second temperature changes from the first temperature by at least a predetermined amount, system 100 determines that the surgical instrument is in physical contact with the patient's tissue between time t5 and time t6 because the temperature of the surgical instrument is in a steady state from time t5 to time t6.

[0045] In additional or alternative examples, the determination of physical contact between the surgical instrument and patient tissue is conditional at least on the surgical instrument being at a first steady state. For example, system 100 does not determine physical contact between the surgical instrument and patient tissue at any time between time t0 and time t3 because the temperature of the surgical instrument is not at a steady state. On the other hand, assuming that the second temperature changes from the first temperature by at least a predetermined amount, system 100 determines that the surgical instrument is in physical contact with patient tissue between time t5 and t6 because the temperature of the surgical instrument is at a steady state from time t3 to time t4.

[0046] In some examples, the determination of physical contact between the surgical instrument and patient tissue is based on the determination of the non-contact state temperature of the surgical instrument. The non-contact state temperature is the normal operating temperature of the surgical instrument when it is not in contact with patient tissue. The non-contact state temperature of the surgical instrument can be determined in any suitable manner. In some examples, the non-contact state temperature may be a first instance of the steady-state temperature during the use of the surgical instrument (e.g., Figure 5 (The temperature from time t3 to time t4). Alternatively, the non-contact state temperature can be predetermined (e.g., based on empirical analysis).

[0047] As mentioned, system 100 is configured to determine physical contact between the surgical instrument and patient tissue when system 100 determines that the temperature of the surgical instrument has changed from a first temperature to a second temperature that has changed by at least a predetermined amount from the first temperature. In some examples, the non-contact state temperature is set as the first temperature used for all real-time determinations of tissue contact during the surgical procedure. For illustration, system 100 may determine that a first instance of steady-state temperature occurs from time t3 to time t4, and thus determine that the non-contact state temperature of the surgical instrument is the temperature at time t3 to t4, i.e., approximately 51.4°C. System 100 may set the first temperature as the non-contact state temperature (e.g., 51.4°C). The temperature of the surgical instrument changes from approximately 50.0°C at time t7 to approximately 44.3°C at time t8. If the predetermined amount is set to 7°C, the temperature change from time t7 to time t8 is 5.7°C, which is less than the predetermined amount. However, system 100 determines that the surgical instrument is in physical contact with the patient's tissue at time t8 because the temperature of the surgical instrument changes from a first temperature (non-contact state temperature, i.e., 51.4°C) to a second temperature (approximately 44.3°C at time t8) that changes by 7.1°C from the first temperature, which is greater than a predetermined amount. In this way, physical contact with the tissue can be detected even if the surgical instrument may not have fully returned to its non-contact state temperature after a previous temperature decrease.

[0048] In additional or alternative embodiments, system 100 determines whether a surgical instrument is in physical contact with patient tissue based on the detected rate of temperature change. The rate of temperature change can indicate whether a surgical instrument is in physical contact with patient tissue or whether the temperature change is due to some other reason. For example, a surgical instrument may change temperature faster when in physical contact with patient tissue than when in contact with another surgical instrument operating at a temperature higher than that of the patient tissue.

[0049] System 100 can determine the rate of temperature change in any suitable manner. In some examples, system 100 determines the average rate of temperature change over a period of time, such as a period between the most recent highest temperature and the most recent lowest temperature (e.g., between time t7 and time t8) within a regular time interval (e.g., every 5 seconds), a period between consecutive steady-state temperatures (e.g., between time t4 and time t5), and so on. If the measured rate of temperature change is greater than or equal to a minimum rate of change (e.g., 1 °C / min), less than or equal to a maximum rate of change (e.g., 3 °C / min), or within a specific rate of change range (e.g., between 1 °C / min and 3 °C / min), system 100 can determine that the surgical instrument is in physical contact with the patient's tissue. The minimum, maximum, and / or range of the rate of change can be predetermined (e.g., based on empirical analysis) or can be set based on the detected non-contact temperature of the surgical instrument.

[0050] To illustrate, such as Figure 5 As shown, the temperature change from time t1 to time t2 may be due to the initialization of the surgical instruments and / or the navigation of the surgical instruments to the surgical area within the patient's body, while the temperature change from time t4 to time t5 may be due to contact with the patient's tissue. If the minimum rate of change is 1.0 °C / min and the maximum rate of change is 3.0 °C / min, the temperature change rate from time t1 to time t2 is 0.9 °C / min, and the temperature change rate from time t4 to time t5 is 2.0 °C / min, then system 100 determines that the surgical instruments are in physical contact with the patient's tissue at time t5 rather than at time t2, because only the temperature change rate from time t4 to time t5 falls between the minimum and maximum values ​​of the rate of change.

[0051] As an alternative to tracking the rate of temperature change, when system 100 determines that the temperature of the surgical instrument changes from a first temperature to a second temperature that changes by at least a predetermined amount from the first temperature within a predetermined time interval, system 100 may determine that the surgical instrument is in physical contact with the patient's tissue. For example, system 100 may determine that the surgical instrument is in physical contact with the patient's tissue at time t5 because the temperature change of the surgical instrument from time t4 to time t5 exceeds a predetermined amount of temperature change and the amount of time elapsed from time t4 to time t5 is within a predetermined time interval (e.g., within 30 seconds).

[0052] In additional or alternative embodiments, if the second temperature falls within the contact state temperature range, system 100 determines that the surgical instrument is in physical contact with the patient tissue. The contact state temperature of the surgical instrument refers to the steady-state temperature of the surgical instrument when it is in contact with the patient tissue. Since the temperature of the patient tissue is typically constant (approximately 37°C), the contact state temperature of the surgical instrument can be measured or calculated empirically or based on the non-contact state temperature of the surgical instrument.

[0053] The contact state range refers to the temperature range within which the temperature of the surgical instrument may change while it remains in contact with the patient's tissue, and can be expressed as a temperature value (e.g., ±2°C) or a percentage (e.g., ±5%). For example, if the non-contact state temperature of the surgical instrument is 51.4°C (see... Figure 5 If the measured temperature of the surgical instrument is within the contact state range, such as 42°C ± 2°C or 42°C ± 5%, then the system 100 can determine (e.g., by calculating or determining based on pre-stored data) that the contact state temperature of the surgical instrument is 42°C. Therefore, if the measured temperature of the surgical instrument is within the contact state range, such as 42°C ± 2°C or 42°C ± 5%, then the system 100 can determine that the surgical instrument is in physical contact with the patient's tissue from time t5 to t6, rather than from time t8 to t9.

[0054] In some cases, when surgical instruments physically contact patient tissue, the non-contact temperature of the instruments may not be high (or low) enough to produce an easily detectable temperature change. Therefore, system 100 can be configured to maintain the non-contact temperature of the surgical instruments at a predetermined level (e.g., 47°C), or at a level where the non-contact temperature changes from the tissue temperature by at least a predetermined non-contact temperature difference (e.g., 10°C), such as by passively applying heat to or removing heat from the surgical instruments. The non-contact temperature of the surgical instruments can be increased or decreased such that the non-contact temperature of the surgical instruments changes from the temperature of the patient tissue by at least the predetermined non-contact temperature difference. In some examples, system 100 is configured to adjust the non-contact temperature of the surgical instruments in response to determining that the change in the non-contact temperature of the surgical instruments from the temperature of the patient tissue is less than a predetermined non-contact temperature difference. By adjusting the non-contact temperature of the surgical system as just described, system 100 can ensure that physical contact with patient tissue can be accurately detected.

[0055] In the foregoing embodiments, system 100 determines the physical contact of the surgical instrument with the patient tissue based on the tracked temperature of the surgical instrument. In the above embodiments, the temperature of the surgical instrument is tracked based on temperature values ​​measured by temperature sensors on the surgical instrument (e.g., temperature sensor 208 or temperature sensor 302). In embodiments where the surgical instrument includes multiple temperature sensors (see, for example...), the temperature of the surgical instrument is tracked based on the temperature values ​​measured by temperature sensors on the surgical instrument. Figure 4 The system tracks the temperature of the surgical instrument at each location where a temperature sensor is located. Therefore, in some examples, if the system 100 determines that at any one or more locations on the surgical instrument, the temperature of the surgical instrument changes from a first temperature to a second temperature that changes by at least a predetermined amount from the first temperature, the system 100 can determine that the surgical instrument is in physical contact with patient tissue. Furthermore, the system 100 can determine specific locations on the surgical instrument that are in physical contact with patient tissue based on the temperature tracked at each location. In alternative examples, if the system 100 determines that at only two or more locations on the surgical instrument, the temperature of the surgical instrument changes from a first temperature to a second temperature that changes by at least a predetermined amount from the first temperature, the system 100 can determine that the surgical instrument is in physical contact with patient tissue.

[0056] In embodiments where the surgical instrument includes multiple temperature sensors, the sensitivity of tissue contact detection can also depend on the location of the temperature sensors on the surgical instrument. For example, for temperature data obtained from a temperature sensor located near the distal end of the endoscope (e.g., 5 mm from the distal end), a predetermined amount can be set to be greater than the temperature data obtained from a temperature sensor located further distally (e.g., 25 mm from the distal end).

[0057] In the foregoing embodiments, system 100 determines when a surgical instrument physically contacts patient tissue based on the temperature of the tracked surgical instrument. In other embodiments, system 100 may also determine when a surgical instrument physically contacts patient tissue based on the operation of the tracked surgical instrument. System 100 can use the tracking operation of the surgical instrument in conjunction with the tracking temperature of the surgical instrument to better determine when the surgical instrument physically contacts patient tissue.

[0058] As used herein, the operation of surgical instruments can include any mechanical, electrical, optical, hardware, and / or software-based operation that may serve a particular implementation. For example, the operation of surgical instruments can include movement of the surgical instruments, operation of functional features of the surgical instruments (e.g., energizing cauterization instruments, opening and closing forceps or scissors, triggering anastomosis instruments, activating fluorescence excitation illumination, etc.), adjustment of surgical instrument settings (e.g., adjusting the exposure level or zoom level of an endoscope, etc.), detection of reflected illumination via an image sensor, detection of system malfunctions or errors (e.g., detecting collisions between surgical instruments), generation of fault codes, etc.

[0059] System 100 can track the operation of surgical instruments in any suitable manner. In some examples, tracking includes collecting surgical data representing the operation of surgical instruments during surgery and / or processing surgical data (e.g., to reduce noise, classify and categorize events, apply timestamps, etc.). Surgical data can be generated by system 100, surgical instruments, surgical systems associated with surgical instruments, and / or by any other device associated with surgical instruments that may serve a particular implementation. Surgical data generated during surgery can include various types of data. For example, surgical data generated during surgery can include kinematic data, image data, sensor data, surgical instrument data, and / or any other type of data that may serve a particular implementation.

[0060] Kinematic data can represent the posture of the surgical instrument, the movement of the surgical instrument, and any other position- and / or motion-based information that may be suitable for a particular implementation. Image data can represent one or more images captured by the surgical instrument. For example, image data can represent one or more still images and / or videos captured by an imaging device (e.g., a stereoscopic endoscope). Sensor data can include any data generated by surgical instrument sensors included in or associated with the surgical instrument. Sensor data can represent any sensing parameters that may serve a particular implementation. In some examples, certain kinematic and image data can be generated by and / or based on parameters sensed by sensors of the surgical system. Therefore, sensor data can include such kinematic and image data. Surgical instrument data can include any other data generated or maintained by the surgical instrument, such as the identification (“ID”) of the surgical instrument, the operating status of the surgical instrument (e.g., on, off, charging, idle, etc.), fault codes of the surgical instrument, etc.

[0061] As mentioned, system 100 can be configured to further determine the physical contact between the surgical instruments and patient tissue based on the tracked operation of the surgical instruments. For example, as Figure 5 As shown, system 100 can determine that the temperature of the surgical instrument changes from a first temperature at time t7 to a second temperature at time t8 where the first temperature changes by a predetermined amount. However, system 100 can further determine, based on the tracked operation of the surgical instrument, that the surgical instrument is an endoscope and that the illumination source providing illumination via the surgical instrument is turned off at time t7. Therefore, system 100 can infer that the temperature decrease of the surgical instrument is due to a decrease in the illumination output of the surgical instrument. Therefore, system 100 does not determine that the surgical instrument is in physical contact with the patient's tissue at time t8.

[0062] As another example, system 100 can determine that the temperature of the surgical instrument changes from a first temperature at time t4 to a second temperature at time t5 where the first temperature changes by a predetermined amount. Furthermore, system 100 can further determine that the surgical instrument had moved immediately before the temperature change (e.g., at or before time t4). Therefore, system 100 can determine that the surgical instrument was in physical contact with the patient's tissue at time t5 because the temperature change occurred immediately after the movement of the surgical instrument.

[0063] As mentioned, system 100 can determine, based on the temperature of the tracked surgical instrument, when the temperature of the surgical instrument changes from a first temperature to a second temperature that changes by at least a predetermined amount from the first temperature, that the surgical instrument will make physical contact with the patient tissue. In some examples, the predetermined amount can vary based on the tracking operation of the surgical instrument. For example, the predetermined amount can be decreased when the tracking operation of the surgical instrument indicates that physical contact with the patient tissue is likely (e.g., movement prior to a temperature change, increased reflected illumination intensity detected by the imaging device, increased saturation in the imaging device, surgical instrument sensor data indicating contact with another object, etc.). Similarly, the predetermined amount can be increased when the tracking operation of the surgical instrument indicates that physical contact with the patient tissue is unlikely (e.g., no movement of the surgical instrument, decreased reflected illumination intensity detected by the imaging device, decreased brightness of the image captured by the imaging device, detection of a collision between the surgical instrument and another surgical instrument, etc.).

[0064] In the foregoing embodiments, the surgical instrument has been described as having a non-contact state temperature higher than the patient tissue temperature, so the system 100 detects physical contact with the patient tissue when the temperature of the surgical instrument decreases. Similarly, the systems and methods described herein are also applicable to surgical instruments having a non-contact state temperature lower than the patient tissue temperature. In this case, the system 100 detects physical contact with the patient tissue when the temperature of the surgical instrument increases.

[0065] The foregoing examples provide various exemplary methods and criteria for determining physical contact between a surgical instrument and patient tissue based on the temperature and / or operation of the tracked surgical instrument. However, system 100 is not limited to the specific methods and criteria described in the examples above, but may include variations and modifications of methods and criteria suitable for a particular implementation. Furthermore, system 100 may utilize any combination or sub-combination of methods and criteria to determine physical contact between a surgical instrument and patient tissue. System 100 may combine various methods and criteria in any suitable manner to help system 100 determine when a surgical instrument is in physical contact with patient tissue.

[0066] As described above, system 100 can be configured to perform a mitigation operation in response to determining that a surgical instrument is in physical contact with patient tissue. This mitigation operation is configured to reduce the likelihood of damage to the patient tissue caused by the physical contact between the surgical instrument and the patient tissue. In some examples, system 100 performs a mitigation operation by guiding the surgical instrument and / or a surgical system associated with the surgical instrument to perform another mitigation operation.

[0067] In some examples, mitigation actions include any suitable action configured to notify the operator of surgical instruments of physical contact between the instruments and patient tissue. For example, system 100 may direct surgical system 206 to present a notification of physical contact between surgical instruments and patient tissue. The notification can be in any format, such as, but not limited to, visual (e.g., warning lights, warning icons displayed on a graphical user interface viewed by the user, messages displayed on a graphical user interface, etc.), audio (e.g., warning tones, audible warning messages, etc.), and tactile (e.g., vibration of a manual controller of a surgical instrument, etc.). Furthermore, the notification can be provided by any suitable device, including but not limited to display devices included in surgical system 206 (e.g., stereoscopic viewers in a surgeon's console, auxiliary display devices, mobile devices associated with the user, etc.), speakers (e.g., speakers included in a surgeon's console, speakers on mobile devices, etc.), manual control devices (e.g., controllers for manually controlling the operation of surgical instruments, etc.), and so on.

[0068] Figure 6 The illustration depicts an exemplary notification that can be presented to a user (e.g., a surgeon, technician, etc.). As shown, image 600, presented by an image display system included in a surgical system (e.g., surgical system 206), shows an endoscopic view of a surgical area captured by an endoscope. Image 600 includes a message bubble 602 overlaid on the endoscopic view of the surgical area. As shown, message bubble 602 includes a message stating, “Endoscope in contact with tissue.” It should be understood that message bubble 602 can be located anywhere and can provide any information suitable for a particular implementation. In some examples, message bubble 602 may include an optional option (not shown) configured to eliminate or minimize message bubble 602 upon user selection. Figure 6 As can be clearly seen, system 100 is configured to detect when surgical instruments (e.g., endoscopes) included in the surgical system come into physical contact with patient tissue located outside the endoscopic field of view during minimally invasive surgical procedures.

[0069] In cases where multiple surgical instruments are located in the surgical area, the notification can identify the specific surgical instrument that is in physical contact with the patient's tissue. In some examples, the warning message can name the specific surgical instrument that is in physical contact with the patient's tissue (see, for example...). Figure 6Additionally or alternatively, endoscopic images can graphically identify (e.g., highlight, point to, label with icons, etc.) specific surgical instruments. Specific surgical instruments in physical contact with patient tissue can be identified in any suitable manner. In some examples, system 100 can obtain (e.g., from surgical instruments in physical contact with patient tissue) the ID of the surgical instrument and provide the ID of the surgical instrument to the surgical system. The surgical system can track the position of surgical instruments located in the surgical area (e.g., using kinematic data, label-based computer vision tracking, image object recognition, etc.) and identify specific surgical instruments in physical contact with patient tissue based on the ID of the surgical instrument.

[0070] Figure 7 The illustration depicts an exemplary notification that can be presented to a user to identify a specific surgical instrument in physical contact with patient tissue. As shown, image 700, presented by an image display system included in the surgical system (e.g., surgical system 206), shows an endoscopic view of the surgical area captured by an endoscope, including views of an ultrasound instrument 702 and a scissor-type instrument 704. When system 100 determines that the ultrasound instrument 702 is in physical contact with patient tissue, a message bubble 706 is displayed on the endoscopic view of the surgical area and points to the ultrasound instrument 702. Figure 7 As shown, message bubble 706 includes a message stating, "Instrument in contact with tissue." It should be understood that message bubble 706 can be located anywhere and can provide any information suitable for a particular implementation. Using this configuration, the user can quickly identify which surgical instrument is in physical contact with the patient's tissue and take corrective action. Furthermore, since ultrasound instrument 702 must be in physical contact with the patient's tissue to capture and generate ultrasound images, the features described herein can help the user know when ultrasound instrument 702 is in physical contact with the patient's tissue.

[0071] In the example where system 100 identifies a specific location on a surgical instrument that is in physical contact with patient tissue, a notification may display or identify that location. The specific location on the surgical instrument in physical contact with patient tissue can be identified in any suitable manner. As described above, system 100 can use the tracking temperature of each location on the surgical instrument to determine the specific location on the surgical instrument that is in physical contact with patient tissue. System 100 may provide information identifying the specific location to a surgical system (e.g., surgical system 206). Also as described above, the surgical system may track the position of the surgical instrument within the surgical area. Based on the tracked position of the surgical instrument and the specific location on the surgical instrument, the surgical system may be configured to provide a notification to identify the specific location on the surgical instrument that is in physical contact with patient tissue.

[0072] Figure 8An exemplary notification that can be presented to a user to identify a specific location on a surgical instrument that is in physical contact with patient tissue is illustrated. As shown, image 800 presented by an image display system included in a surgical system (e.g., surgical system 206) shows an endoscopic view of a surgical area captured by an endoscope, including a view of an ultrasound instrument 802 located in the surgical area. When system 100 determines that a distal region of the ultrasound instrument 802 is in physical contact with patient tissue, a message bubble 804 is displayed on the endoscopic view of the surgical area and points to the distal region of the ultrasound instrument 802. As shown, the message in message bubble 804 states: “Tissue contact detected here.” It should be understood that message bubble 804 can be located anywhere and can provide any information suitable for a particular implementation.

[0073] As a supplement to or alternative to notification, mitigation measures may include any action configured to reduce the temperature of surgical instruments and thereby reduce the risk of damage to patient tissues. To this end, system 100 may adjust or direct the operation of the surgical instruments and / or the surgical system associated with them. For example, when surgical instruments 202 are operated by an imaging device (e.g., an endoscope), system 100 may direct surgical system 206 to adjust the output of illumination provided to the surgical area by means of surgical instruments 202. This adjustment may be made, for example, by reducing the intensity of light provided to the surgical instruments, intermittently interrupting the light output provided to the surgical instruments, or turning off the light source (e.g., a fluorescent excitation illumination source, a blue light source, etc.). As another example, system 100 may direct surgical system 206 to adjust the operation of the imaging device (e.g., reducing resolution, reducing frame rate, etc.) to reduce the heat generated by the imaging device. In other examples, system 100 may direct surgical system 206 to reduce passive heat applied to the surgical instruments, increase cooling of the surgical instruments, reduce the amount of energy provided by means of the surgical instruments (e.g., cauterization energy), and so on.

[0074] Figure 9 An exemplary method 900 for performing mitigation operations is illustrated. Although Figure 9 The figure illustrates an exemplary operation according to one embodiment, but other embodiments may omit, add, reorder, and / or modify it. Figure 9 Any of the operations shown.

[0075] In operation 902, system 100 determines whether the surgical instrument is in physical contact with the patient's tissue. Operation 902 can be performed in any of the manner described herein. If system 100 does not determine that the surgical instrument is in physical contact with the patient's tissue, system 100 returns to the beginning and continues to monitor for tissue contact. However, if system 100 determines that the surgical instrument is in physical contact with the patient's tissue, system 100 proceeds to operation 904.

[0076] In operation 904, system 100 checks whether a predetermined amount of time has elapsed since the first detection of tissue contact. The predetermined amount of time can be any period of time suitable for a particular implementation (e.g., 2 minutes). In some examples, the predetermined amount of time is set by system 100. Additionally or alternatively, the predetermined amount of time can be configured by the user. If system 100 determines that no predetermined amount of time has elapsed since the first detection of tissue contact, system 100 proceeds to operation 906. In operation 906, system 100 presents or directs the surgical system to present notification of physical contact between the surgical instrument and the patient's tissue. Operation 906 can be performed in any of the manner described herein.

[0077] However, if system 100 determines that a predetermined amount of time has elapsed since the first detection of tissue contact, system 100 proceeds to operation 908. In operation 908, system 100 adjusts or guides the surgical system to adjust the operation of the surgical instruments to alleviate physical contact between the surgical instruments and the patient's tissue. Operation 908 can be performed in any of the manner described herein.

[0078] After operations 906 and 908, system 100 returns to operation 902 to determine whether the surgical instruments are still in physical contact with the patient's tissue. If system 100 determines that the surgical instruments are no longer in physical contact with the patient's tissue, system 100 returns to the start to monitor physical contact with the patient's tissue. System 100 can also deactivate or guide the surgical system to deactivate notifications and / or restore or guide the surgical system to resume normal operation of the surgical instruments.

[0079] In method 900, a notification of physical contact between the surgical instrument and patient tissue is presented, but the operation of the surgical instrument is not adjusted unless a predetermined amount of time has elapsed. In this way, system 100 allows the user to perform an action to alleviate tissue contact before the surgical system automatically performs an action to alleviate tissue contact. However, the processing for performing the mitigation action is not limited to the aforementioned sequence. In an alternative embodiment, system 100 may be configured to present or guide the surgical system to present a notification only after the surgical instrument has maintained physical contact with patient tissue for another predetermined amount of time (e.g., three minutes). In such an embodiment, it can be assumed that the physical contact with patient tissue is only temporary and that the surgical instrument may move before injury to the patient tissue. In other alternative embodiments, system 100 may be configured to provide or guide the surgical system to provide a notification and simultaneously adjust or guide the surgical system to adjust the operation of the surgical instrument. In yet another embodiment, any predetermined amount of time and / or the sequence of performing mitigation actions may be set based on the type of surgical instrument. For example, for an endoscope operating at a higher non-contact state temperature than a scissor-type instrument, the predetermined amount of time considered in operation 904 may be set shorter than that for a scissor-type instrument.

[0080] Figure 10An exemplary computer-assisted surgical system 1000 (“surgical system 1000”) is illustrated. As described herein, system 100 may be implemented by surgical system 1000, implemented by components included in surgical system 1000, connected to surgical system 1000, and / or otherwise used in conjunction with surgical system 1000.

[0081] As shown in the figure, the surgical system 1000 may include a control system 1002, a user control system 1004, and an auxiliary system 1006 that are communicatively coupled to each other. The surgical team can use the surgical system 1000 to perform computer-aided surgical procedures on the patient 1008. As shown, the surgical team may include a surgeon 1010-1, an assistant 1010-2, a nurse 1010-3, and an anesthesiologist 1010-4, all of whom can be collectively referred to as "surgical team members 1010". Additional or alternative surgical team members may appear during the surgical procedure as they may serve a specific implementation.

[0082] although Figure 10 The illustration depicts a minimally invasive surgical procedure in progress; however, it should be understood that the surgical system 1000 can be similarly used to perform open surgical procedures or other types of surgical procedures, which can similarly benefit from the accuracy and convenience of the surgical system 1000. Furthermore, it should be understood that surgical procedures using the surgical system 1000 throughout can include not only the operational phases of surgical procedures, such as... Figure 10 As shown, it can also include other appropriate stages of the preoperative, postoperative and / or surgical procedures.

[0083] like Figure 10 As shown, the manipulation system 1002 may include a plurality of manipulator arms 1012 (e.g., manipulator arms 1012-1 to 1012-4), to which a plurality of surgical instruments may be coupled. Although the manipulation system 1002 is depicted and described herein as including four manipulator arms 1012, it should be understood that the manipulation system 1002 may include only a single manipulator arm 1012 or any other number of manipulator arms that may be used for a particular implementation.

[0084] The manipulator arm 1012 and / or surgical instruments attached to it may include one or more displacement transducers, orientation sensors, and / or position sensors for generating raw (i.e., uncorrected) kinematic information. One or more components of the surgical system 1000 may be configured to use kinematic information to track (e.g., determine their position and orientation) and / or control the surgical instruments. Furthermore, the surgical instruments attached to the manipulator arm 1012 may include one or more temperature sensors for generating temperature data. The system 100 may be configured to use the temperature data to track the temperature of the surgical instruments during surgical procedures.

[0085] User control system 1004 can be configured to facilitate surgeon 1010-1's control of manipulator arm 1012 and surgical instruments attached to manipulator arm 1012. For example, surgeon 1010-1 can interact with user control system 1004 to remotely move or manipulate manipulator arm 1012 and surgical instruments. To this end, user control system 1004 can provide surgeon 1010-1 with images (e.g., high-definition 3D images, synthetic medical images, etc.), such as images 600, 700, and / or 800 of the surgical area associated with patient 1008 captured by an imaging system (e.g., an endoscope performed by surgical instrument 202). In some examples, user control system 1004 may include a stereoscopic viewer with two displays, where surgeon 1010-1 can view stereoscopic images of the surgical area associated with patient 1008 and generated by a stereoscopic imaging system. Surgeon 1010-1 can utilize the images to perform one or more procedures, wherein one or more surgical instruments are attached to manipulator arm 1012.

[0086] To facilitate control of surgical instruments, the user control system 1004 may include a set of master controls. These master controls can be manipulated by the surgeon 1010-1 to control the movement of surgical instruments (e.g., by utilizing robotic and / or teleoperation technologies). The master controls can be configured to detect various hand, wrist, and finger movements of the surgeon 1010-1. In this way, the surgeon 1010-1 can intuitively perform procedures using one or more surgical instruments.

[0087] The auxiliary system 1006 may include one or more computing devices configured to perform the main processing operations of the surgical system 1000. In such a configuration, the one or more computing devices included in the auxiliary system 1006 may control and / or coordinate operations performed by various other components of the surgical system 1000 (e.g., the manipulation system 1002 and the user control system 1004). For example, the computing devices included in the user control system 1004 may send instructions to the manipulation system 1002 by means of the one or more computing devices included in the auxiliary system 1006. As another example, the auxiliary system 1006 may receive and process image data representing images (e.g., images 600, 700, and / or 800) captured by an imaging device attached to one of the manipulator arms 1012 from the manipulation system 1002.

[0088] In some examples, the assistive system 1006 may be configured to present visual content to a surgical team member 1010 who may not have access to images provided to the surgeon 110-1 at the user control system 1004. To this end, the assistive system 1006 may include a display monitor 1014 configured to display one or more user interfaces, such as images of the surgical area (e.g., 2D images, composite images, etc.), information associated with the patient 1008 and / or surgical procedures, and / or any other visual content that may serve a particular implementation. For example, the display monitor 1014 may display an image of the surgical area along with additional content displayed concurrently with the image (e.g., graphical content, contextual information, etc.). In some embodiments, the display monitor 1014 is implemented as a touchscreen display, with which the surgical team member 1010 may interact (e.g., via touch gestures) to provide user input to the surgical system 1000.

[0089] The operating system 1002, the user control system 1004, and the auxiliary system 1006 can be communicatively coupled to each other in any suitable manner. For example, such as Figure 10 As shown, the operating system 1002, user control system 1004, and auxiliary system 1006 can be communicatively coupled via control line 1016, which can represent any wired or wireless communication link suitable for a particular implementation. Therefore, the operating system 1002, user control system 1004, and auxiliary system 1006 can each include one or more wired or wireless communication interfaces, such as one or more local area network interfaces, Wi-Fi network interfaces, cellular interfaces, etc.

[0090] Figure 11 An exemplary method 1100 for detecting physical contact between surgical instruments and patient tissue is shown. Although Figure 11 The figure illustrates an exemplary operation according to one embodiment, but other embodiments may omit, add, reorder, combine, and / or modify it. Figure 11 Any of the steps shown. Figure 11 One or more operations shown can be performed by system 100, any of its components and / or any implementation thereof.

[0091] In operation 1102, a tissue contact detection system tracks the temperature of surgical instruments associated with the surgical system used in the surgical procedure over time during the procedure. Operation 1102 can be performed in any of the manner described herein.

[0092] In operation 1104, the tissue contact detection system determines, based on the temperature of the tracked surgical instrument, that the temperature of the surgical instrument changes from a first temperature to a second temperature that changes by at least a predetermined amount from the first temperature. Operation 1104 can be performed in any of the manner described herein.

[0093] In operation 1106, the tissue contact detection system determines the physical contact between the surgical instrument and the patient's tissue based on the determination of the temperature change of the surgical instrument from a first temperature to a second temperature. Operation 1106 can be performed in any of the manner described herein.

[0094] In operation 1108, in response to determining that the surgical instrument is in physical contact with the patient's tissue, the tissue contact detection system performs a mitigation operation configured to relieve the physical contact between the surgical instrument and the patient's tissue. Operation 1108 can be performed in any of the manner described herein.

[0095] In some examples, a non-transitory computer-readable medium may be provided for storing computer-readable instructions, based on the principles described herein. When executed by a processor of a computing device, the instructions may direct the processor and / or the computing device to perform one or more operations, including one or more operations described herein. Such instructions may be stored and / or transmitted using any of a variety of known computer-readable media.

[0096] As used herein, a non-transitory computer-readable medium may include any non-transitory storage medium that contributes to providing data (e.g., instructions) that can be read and / or executed by a computing device (e.g., by a processor of the computing device). For example, a non-transitory computer-readable medium may include, but is not limited to, any combination of non-volatile storage media and / or volatile storage media. Exemplary non-volatile storage media include, but are not limited to, read-only memory, flash memory, solid-state drives, magnetic storage devices (e.g., hard disks, floppy disks, magnetic tapes, etc.), ferroelectric random access memory (“RAM”), and optical discs (e.g., optical discs, digital video discs, Blu-ray discs, etc.). Exemplary volatile storage media include, but are not limited to, RAM (e.g., dynamic RAM).

[0097] Figure 12 The figure illustrates an exemplary computing device 1200, which may be specifically configured to perform one or more processes described herein. Any system, unit, computing device, and / or other component described herein may be implemented by computing device 1200.

[0098] like Figure 12 As shown, computing device 1200 may include a communication interface 1202, a processor 1204, a storage device 1206, and an input / output (“I / O”) module 1208 that are communicatively connected to each other via communication infrastructure 1210. Although Figure 12An exemplary computing device 1200 is shown, but Figure 12 The components shown are not intended to be limiting. Additional or alternative components may be used in other embodiments. A more detailed description will now follow. Figure 12 Components of the computing device 1200 shown.

[0099] Communication interface 1202 can be configured to communicate with one or more computing devices. Examples of communication interface 1202 include, but are not limited to, wired network interfaces (such as network interface cards), wireless network interfaces (such as wireless network interface cards), modems, audio / video connections, and any other suitable interfaces.

[0100] Processor 1204 generally refers to any type or form of processing unit capable of processing data and / or interpreting, executing, and / or directing the execution of one or more of the instructions, procedures, and / or operations described herein. Processor 1204 may perform operations by executing computer-executable instructions 1212 (e.g., applications, software, code, and / or other executable data instances) stored in storage device 1206.

[0101] Storage device 1206 may include one or more data storage media, devices, or configurations and may take any type, form, and combination of data storage media and / or devices. For example, storage device 1206 may include, but is not limited to, any combination of non-volatile media and / or volatile media described herein. Electronic data (including the data described herein) may be stored temporarily and / or permanently in storage device 1206. For example, data representing computer-executable instructions 1212 configured to boot processor 1204 to perform any of the operations described herein may be stored in storage device 1206. In some examples, data may be arranged in one or more databases residing within storage device 1206.

[0102] I / O module 1208 may include one or more I / O modules configured to receive user input and provide user output. I / O module 1208 may include any hardware, firmware, software, or a combination thereof that supports input and output capabilities. For example, I / O module 1208 may include hardware and / or software for capturing user input, including but not limited to a keyboard or keypad, a touchscreen component (e.g., a touchscreen display), a receiver (e.g., an RF or infrared receiver), a motion sensor, and / or one or more input buttons.

[0103] I / O module 1208 may include one or more means for presenting output to a user, including but not limited to a graphics engine, a display (e.g., a screen), one or more output drivers (e.g., display drivers), one or more audio speakers, and one or more audio drivers. In some embodiments, I / O module 1208 is configured to provide graphical data to the display for presentation to the user. The graphical data may represent one or more graphical user interfaces and / or any other graphical content that may be used in a particular implementation.

[0104] Various exemplary embodiments have been described in the foregoing description with reference to the accompanying drawings. However, it will be apparent that various modifications and changes can be made thereto, and additional embodiments can be implemented without departing from the scope of the invention as set forth in the appended claims. For example, certain features of one embodiment described herein may be combined with or substitute for features of another embodiment described herein. Therefore, the specification and drawings are to be considered illustrative rather than restrictive.

Claims

1. A system comprising: Memory that stores instructions; as well as A processor, which is communicatively coupled to the memory and configured to execute the instructions to: The temperature of surgical instruments associated with the surgical system used in the surgical procedure is tracked over time during the surgical procedure. Based on the tracked temperature of the surgical instrument, a second temperature is determined where the temperature of the surgical instrument changes from a first temperature to a change of at least a predetermined amount from the first temperature. Based on the determination of the temperature change of the surgical instrument from the first temperature to the second temperature, it is determined that the surgical instrument is in physical contact with the patient's tissue. In response to the determination that the surgical instrument is in physical contact with the patient's tissue, a relief procedure is performed.

2. The system according to claim 1, wherein: The processor is further configured to execute the instructions to determine that the temperature of the surgical instrument changes from the first temperature to the second temperature at at least a predetermined rate of change or within a predetermined time interval, and The determination of physical contact between the surgical instrument and the patient's tissue is further based on determining that the temperature of the surgical instrument changes from the first temperature to the second temperature at at least the predetermined rate of change or within the predetermined time interval.

3. The system according to claim 1, wherein: The processor is further configured to execute the instructions to determine that the second temperature is within a predetermined contact state temperature range, and The determination of physical contact between the surgical instrument and the patient tissue is further based on determining that the second temperature is within the predetermined contact state temperature range.

4. The system according to claim 1, wherein: The processor is further configured to execute the instructions to track the operation of the surgical instruments over time during the surgical procedure, and The determination of physical contact between the surgical instrument and the patient's tissue is further based on the tracking operation of the surgical instrument.

5. The system according to claim 1, wherein: The processor is further configured to execute the instructions to: The movement of the surgical instruments is tracked over time during the surgical procedure, and Based on the tracked movement of the surgical instrument, it is determined that the surgical instrument moved immediately before its temperature changed from the first temperature to the second temperature, and The determination that the surgical instrument is in physical contact with the patient's tissue is further based on the determination that the surgical instrument has moved immediately before the temperature of the surgical instrument changes from the first temperature to the second temperature.

6. The system according to any one of claims 1-5, wherein, The processor is further configured to execute the instructions to maintain the non-contact temperature of the surgical instrument at a predetermined level.

7. The system according to any one of claims 1-5, wherein, The processor is further configured to execute the instructions to: Determine that the change in temperature of the surgical instrument in its non-contact state from the temperature of the patient tissue is less than another predetermined amount, and In response to determining that the change in the non-contact state temperature from the temperature of the patient tissue is less than the other predetermined amount, the surgical system is directed to increase or decrease the non-contact state temperature of the surgical instrument such that the change in the non-contact state temperature of the surgical instrument from the temperature of the patient tissue is at least the other predetermined amount.

8. The system according to any one of claims 1-5, wherein, At least one of the first temperature and the second temperature is in a steady state.

9. The system according to any one of claims 1-5, wherein, The relief procedure includes providing notification of physical contact between the surgical instruments and the patient's tissues.

10. The system according to any one of claims 1-5, wherein: The surgical instruments include an endoscope, and The mitigation procedure includes adjusting the illumination output from the distal end of the endoscope.

11. The system according to any one of claims 1-5, wherein, The mitigation measures include reducing the passive heat added to the surgical instruments.

12. The system according to any one of claims 1-5, wherein, The tracking of the temperature of the surgical instrument includes receiving temperature data representing the temperature of the surgical instrument from a temperature sensor on the surgical instrument over time.

13. The system according to any one of claims 1-5, wherein: The tracking of the temperature of the surgical instrument includes receiving temperature data representing the temperature of the surgical instrument at each of the plurality of locations from multiple temperature sensors located at multiple locations on the surgical instrument over time. The processor is further configured to execute the instructions to identify, based on the temperature data, the location of physical contact between the surgical instrument and the patient's tissue, and The relief procedure includes providing notification of the location on the surgical instrument that is in physical contact with the patient's tissue.

14. A system comprising: A temperature sensor configured to detect the temperature of surgical instruments included in a surgical system for the surgical procedure during surgery; as well as A processor, communicatively coupled to the temperature sensor and configured to execute the following instructions: The temperature of the surgical instruments, detected by the temperature sensor, is tracked over time during the surgical procedure. Based on the tracked temperature of the surgical instrument, a second temperature is determined where the temperature of the surgical instrument changes from a first temperature to a change from the first temperature by at least a predetermined amount. Based on the determination of the temperature change of the surgical instrument from the first temperature to the second temperature, it is determined that the surgical instrument is in physical contact with the patient's tissue. In response to the determination that the surgical instrument is in physical contact with the patient's tissue, a relief procedure is performed.

15. The system according to claim 14, wherein, The temperature sensor is located on a sleeve covering the shaft of the surgical instrument.

16. A non-transitory computer-readable storage medium storing instructions, which, when executed by one or more processors, cause the one or more processors to perform a method for operating a medical device, the method comprising: During surgical procedures, the temperature of surgical instruments included in the surgical system used for the surgical procedure is tracked over time by a tissue contact detection system. The tissue contact detection system determines, based on the temperature of the tracked surgical instrument, the temperature of the surgical instrument as it changes from a first temperature to a second temperature that changes by at least a predetermined amount from the first temperature; The tissue contact detection system determines the physical contact between the surgical instrument and the patient's tissue based on the temperature change of the surgical instrument from a first temperature to a second temperature. as well as In response to the determination that the surgical instrument is in physical contact with the patient's tissue, the tissue contact detection system performs a mitigation operation.

17. The non-transitory computer-readable storage medium of claim 16, wherein the method further comprises: The tissue contact detection system determines that the temperature of the surgical instrument changes from the first temperature to the second temperature at at least a predetermined rate of change or within a predetermined time interval. The determination of physical contact between the surgical instrument and the patient's tissue is further based on determining that the temperature of the surgical instrument changes from the first temperature to the second temperature at at least the predetermined rate of change or within the predetermined time interval.

18. The non-transitory computer-readable storage medium of claim 16, wherein the method further comprises: The tissue contact detection system determines that the second temperature is within a predetermined contact state temperature range. The determination of physical contact between the surgical instrument and the patient's tissue is further based on determining that the second temperature is within the predetermined contact state temperature range.

19. The non-transitory computer-readable storage medium of claim 16, wherein the method further comprises: During the surgical procedure, the tissue contact detection system tracks the movement of the surgical instruments over time. The determination of physical contact between the surgical instrument and the patient's tissue is further based on the tracking operation of the surgical instrument.

20. The non-transitory computer-readable storage medium of claim 16, wherein the method further comprises: During the surgical procedure, the tissue contact detection system tracks the movement of the surgical instruments over time. as well as The tissue contact detection system determines, based on the tracked movement of the surgical instrument, that the surgical instrument moved immediately before its temperature changed from the first temperature to the second temperature. The determination that the surgical instrument is in physical contact with the patient's tissue is further based on the determination that the surgical instrument has moved immediately before the temperature of the surgical instrument changes from the first temperature to the second temperature.

Images