Display systems for surgical devices, and associated devices and methods

Abstract

Display systems for surgical devices (and associated systems, devices, and methods) are disclosed herein. In some embodiments, the display system includes a control unit having a first input, a second input, and a processor. The first input can be operably coupled to an endoscope and configured to receive a real-time video feed of a surgical site from the endoscope. The second input can be operably coupled to a non-robotic, handheld surgical device and configured to collect real-time data associated with the surgical device. The surgical device can be usable to perform a medical procedure at the surgical site. The processor can be operably coupled to the first and second inputs and configured to generate a graphical user interface (GUI). The GUI can include the real-time video feed of the surgical site and one or more overlays configured to display the real-time data associated with the surgical device.

Description

Atorney Docket No.: P330621.W0.01 (LM8018)DISPLAY SYSTEMS FOR SURGICAL DEVICES, AND ASSOCIATED DEVICES AND METHODSCROSS-REFERENCE TO RELATED APPLICATION(S)

[0001] This application claims priority to U.S. Provisional Application No. 63 / 703,084, filed October 3, 2024, which is incorporated herein by reference in its entirety.TECHNICAL FIELD

[0002] The present technology relates generally to display systems, such as for surgical devices. For example, several embodiments of the present technology relate to display systems for motor-driven surgical stapler devices and associated devices and methods.BACKGROUND

[0003] A surgical handle assembly and / or a surgical reloadable cartridge assembly can be used in a number of surgical devices. One example includes use in — or as part of — a surgical stapler. A surgical stapler is a fastening device used to clamp tissue between opposing jaw structures to join tissue using surgical fasteners. Surgical staplers can include two elongated members used to clamp the tissue. One of the elongated members can include one or more staple cartridges, and the other elongated member can include an anvil that can be used to form a staple when driven from the staple cartridge. Some surgical staplers are equipped with an electric motor that can provide the power to clamp tissue, deliver staples, and provide power for other aspects of a surgical stapler.BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Many aspects of the present disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale. Instead, emphasis is placed on illustrating clearly the principles of the present disclosure. The drawings should not be taken to limit the disclosure to the specific embodiments shown, but are provided for explanation and understanding.

[0005] FIG. 1 is a block diagram of a surgical display system configured in accordance with various embodiments of the present technology.Attorney Docket No.: P330621.W0.01 (LM8018)

[0006] FIG. 2 is a partially schematic diagram of a surgical stapling apparatus configured in accordance with various embodiments of the present technology.

[0007] FIG. 3 is a partially schematic, partial exploded view of a surgical handle assembly configured in accordance with various embodiments of the present technology.

[0008] FIGS. 4A and 4B are partially schematic diagrams of a graphical user interface configured in accordance with various embodiments of the present technology.

[0009] FIG. 5 is a partially schematic diagram of another graphical user interface configured in accordance with various embodiments of the present technology.

[0010] FIG. 6 is a partially schematic diagram of yet another graphical user interface configured in accordance with various embodiments of the present technology.

[0011] FIG. 7 is a partially schematic diagram of an overlay displayable on a graphical user interface, each configured in accordance with various embodiments of the present technology.

[0012] FIG. 8 is a partially schematic diagram of another overlay displayable on a graphical user interface, each configured in accordance with various embodiments of the present technology.

[0013] FIG. 9 is a partially schematic diagram of another overlay displayable on a graphical user interface, each configured in accordance with various embodiments of the present technology.DETAILED DESCRIPTION

[0014] The present disclosure is generally directed to display systems for surgical devices, and to associated devices and methods. Specific details of several embodiments of the present technology are described herein with reference to FIGS. 1-9. Although many of the embodiments are described below with reference to display systems for use with motor-driven surgical stapler devices, other applications and other embodiments in addition to those described herein are within the scope of the present technology. For example, unless otherwise specified or made clear from context, the display systems of the present technology can be used with other devices, such as surgical handle assemblies for other non-stapler medical devices.

[0015] It should be noted that other embodiments in addition to those disclosed herein are within the scope of the present technology. Further, embodiments of the present technologyAttorney Docket No.: P330621.W0.01 (LM8018) can have different configurations, components, and / or procedures than those shown or described herein. Moreover, a person of ordinary skill in the art will understand that embodiments of the present technology can have configurations, components, and / or procedures in addition to those shown or described herein and that these and other embodiments can be without several of the configurations, components, and / or procedures shown or described herein without deviating from the present technology.

[0016] As used herein, the term “physician” shall be understood to include any type of medical personnel who may be performing or assisting a medical procedure and, thus, is inclusive of a doctor, a nurse, a clinician, a medical technician, other similar personnel, and any combination thereof. As used herein, the term “patient” and “subject” should be considered to include human and / or non-human (e.g., animal) patients upon which a medical procedure is being performed.A. Overview

[0017] Physicians often use surgical devices to perform various medical procedures on patients. For example, physicians can use surgical staplers to clamp tissue between opposing jaw structures and join tissue using surgical fasteners (e.g., staples, sutures). Many non-robotic, handheld surgical devices, however, are unable to provide various forms of information that may be helpful to physicians due to, for example, the size of the surgical device, cost, and / or other design constraints. For example, although some surgical devices may include lights that can blink or change colors to indicate a status of the surgical device, such lights are limited in the type and amount of information they can convey to the physician. In another example, although endoscopes can provide video feeds of the surgical site within a patient in real-time, the viewing angle may fail to show all relevant aspects of the medical procedure and it may be difficult for the physician to interpret the video feed in real-time while performing the medical procedure. Moreover, endoscopes cannot provide many other forms of information, such as whether the surgical device is operating properly.

[0018] To address the above concerns, several embodiments of the present technology are directed to a display system that can provide a physician operating a surgical device (e.g., a non-robotic, handheld surgical device) with various information relating to the surgical device and the medical procedure. For example, the display system can display, on a graphical user interface (GUI) and in real-time, a video feed of the surgical procedure (e.g., as captured by an endoscope) and one or more icons or panels around and / or overlaid on the video feed.Atorney Docket No.: P330621.W0.01 (LM8018)The icons or panels can indicate whether the surgical device and / or other devices (e.g., the endoscope) are properly connected to a control unit, an operating mode of the surgical device, specifications of the surgical device, workflow instructions, warning messages, various operating parameters associated with the surgical device, etc.B. Selected Embodiments of Display Systems for Surgical Devices, and Associated Devices and Methods1. Display Systems

[0019] FIG. 1 is a block diagram of a surgical display system 100 (“system 100”) configured in accordance with various embodiments of the present technology. The system 100 can include an endoscope 110, an endoscope control box 120, a surgical device 130, a control unit 160, and a display 170. In the illustrated embodiment, the surgical device 130 includes a reload assembly 140 and a handle apparatus 150 operably coupled to the reload assembly 140. The various components of the system 100 can be operably coupled to one another via wired and / or wireless connections. As shown, during operation, the endoscope 110 and the reload assembly 140 can be positioned at least partially inside a patient’s body while the endoscope control box 120, the handle apparatus 150, the control unit 160, and the display 170 can be positioned outside the patient’s body.

[0020] The endoscope 110 (e.g., a laparoscope) can include a high-resolution camera and a light source to provide visualization of a surgical site inside the patient’s body in realtime. The endoscope control box 120 can manage various operating parameters of the endoscope, such as the illumination intensity and the zoom level of the camera, and process video signals received from the endoscope. In embodiments in which the system 100 includes the endoscope control box 120, the endoscope can be operably coupled to the control unit 160 directly and / or via the endoscope control box 120. In some embodiments, the system 100 can omit or lack the endoscope control box 120 and the endoscope 110 can be directly coupled to the control unit 160.

[0021] The surgical device 130 can include a non-robotic, handheld surgical device. For example, the surgical device 130 can comprise a surgical stapling apparatus, the reload assembly 140 can include jaws for clamping tissue and staples, and the handle apparatus 150 can be used to control the reload assembly 140. An example surgical stapling apparatus is described in detail below with reference to FIGS. 2 and 3. In other examples, the surgical device 130 can comprise non-stapler medical devices.Atorney Docket No.: P330621.W0.01 (LM8018)

[0022] The control unit 160 can include one or more inputs, one or more outputs, and one or more processors. The control unit 160 can receive various signals from the endoscope 110, the endoscope control box 120, and the surgical device 130 via the inputs (e.g., cable ports, communication module units). The processors (e.g., CPU(s), GPU(s), HPU(s), etc.) can be a single processing unit or multiple processing units in a device or distributed across multiple devices. The processors can process the signals received from the endoscope 110, the endoscope control box 120, and the surgical device 130, and send a display signal to the display 170 via the outputs (e.g., cable ports, communication module units). The display 170 can comprise an LCD display screen, an LED display screen, a projected, holographic, or augmented reality display (such as on a heads-up display device or a head-mounted device), and / or the like.

[0023] In operation, as discussed in further detail herein, the control unit 160 can generate one or more overlays (e.g., icons, panels) that can be combined with the video feed from the endoscope 110 and / or the endoscope control box 120. The overlays can include text and / or graphics to provide various forms of information based on the signals received from the endoscope 110 and / or the surgical device 130. The combined video feed and overlays can be displayed on the display 170 for the physician to view during a medical procedure. By generating and displaying the overlays, the system 100 can effectively and efficiently provide relevant and helpful information pertaining to the surgical site, the surgical device, and the medical procedure as a whole.2. Surgical Devices

[0024] FIG. 2 is a partially schematic diagram of a surgical stapling apparatus 230 configured in accordance with various embodiments of the present technology. The surgical stapling apparatus 230 can include a surgical reloadable cartridge assembly 240 (e.g., a disposable loading unit) and a surgical handle assembly 250. It is appreciated that the surgical stapling apparatus 230 can be an example of the surgical device 130 of FIG. 1, the surgical reloadable cartridge assembly 240 can be an example of the reload assembly 140 of FIG. 1, and the surgical handle assembly 250 can be an example of the handle apparatus 150 of FIG. 1. The surgical reloadable cartridge assembly 240 can be releasably secured to a distal end of an elongated body 261 of the surgical handle assembly 250, such as to a drive shaft 204 of the surgical handle assembly 250.Atorney Docket No.: P330621.W0.01 (LM8018)

[0025] In the illustrated embodiment, the reloadable cartridge assembly 240 includes a shaft 211, a first elongated member 207, and a second elongated member 209. The first elongated member 207 and the second elongated member 209 can be used to clamp tissue, and are also referred to herein as “jaws.” One of the elongated members (e.g., the first elongated member 207) can house one or more staple cartridges. The other elongated member (e.g., the second elongated member 209) can include an anvil that can be used to form a staple when driven from the staple cartridge (e.g., by a blade assembly (not shown in FIG. 2), as discussed in greater detail below). In some embodiments, the reloadable cartridge assembly 240 can further include a stabilizing bracket 208 that can help to hold the first elongated member 207 and the second elongated member 209 in alignment with one another and / or that can function as a tissue shield to prevent or hinder tissue from being clamped between the first elongated member 207 and the second elongated member 209 at a location more proximal than a distal end of the stabilizing bracket 208. In these and other embodiments, the reloadable cartridge assembly 240 can include one or more rows of staples having a linear length. For example, a row of staples can have a linear length between approximately 30mm and approximately 60mm. In a number of embodiments, third party reloadable cartridges and / or reloadable cartridge assemblies may be used with the surgical handle assembly 250 and embodiments of the surgical handle assembly 250 may be configured to receive the same.

[0026] The surgical handle assembly 250 can include a radial positioner 224, an articulation assembly activated by an articulation knob 222, a non-movable handle 216 (“the stationary handle 216”), and a movable handle 212. When the reloadable cartridge assembly 240 is releasably secured to the distal end of the elongated body 261 of the surgical handle assembly 250, the reloadable cartridge assembly 240 can be actuated using the articulation knob 222 and / or the radial positioner 224 to reach a stapling site. For example, the radial positioner 224 can be used to rotate the reloadable cartridge assembly 240. Additionally, or alternatively, the articulation knob 222 can be used to position the first elongated member 207 and / or the second elongated member 209 at a particular angle for stapling. The articulation knob 222 can be rotationally actuatable, and the reloadable cartridge assembly 240 can rotate about an axis of a particular plane in response to the articulation knob 222 being rotationally actuated by a physician. The movable handle 212 can be used to clamp and unclamp the first elongated member 207 and the second elongated member 209 together (e.g., to clamp or grip tissue).Attorney Docket No.: P330621.W0.01 (LM8018)

[0027] As shown, the surgical handle assembly 250 further includes a power trigger 210, a selector lever 217, a manual retraction or bailout handle 251, and a user feedback mechanism 214. Although not shown in FIG. 2, the surgical handle assembly 250 can include a power source, such as a battery. The power source (e.g., battery) can be rechargeable (e.g., via an AC power supply) or disposable. If a rechargeable battery is used, the battery can be positioned so that it can be either removed or recharged. If a disposable battery is used, the stationary handle 216 can include a drain so that the battery can be drained prior to disposal. In these and other embodiments, the surgical handle assembly 250 can include or be (e.g., releasably) coupled to a power cord 227. For example, the power cord 227 can be used to electrically couple the surgical handle assembly 250 (e.g., a battery and / or an electric motor of the surgical handle assembly 250) to a power source 223 positioned outside of the surgical handle assembly 250. In turn, the power source 223 can supply AC or DC current to the surgical handle assembly 250 to power various electronics of the surgical handle assembly 250.

[0028] FIG. 3 is a partially schematic, partial exploded view of the surgical handle assembly 250. As discussed above, a distal portion of the surgical handle assembly 250 can include the articulation knob 222 and the radial positioner 224. The drive shaft 204 (e.g., a drive rod) can cooperate with a gear rack (not shown in FIG. 3) to effect movement for a reloadable cartridge assembly (e.g., the reloadable cartridge assembly 240 of FIG. 2).

[0029] FIG. 3 illustrates a first handle half 216-1 and a second handle half 216-2 that together provide the stationary handle 216 for a user of the surgical handle assembly 250 and that together house a drivetrain 318 and / or other electrical and mechanical mechanisms of the surgical handle assembly 250. The first handle half 216-1 and / or the second handle half 216-2 may also include components that are used in the clamping and / or unclamping of the jaws.

[0030] In some embodiments, the drivetrain 318 can include a number of gears (not shown in FIG. 3), a power source (not shown), such as a battery, and / or an electric motor 319. The electric motor 319 can be battery-powered and / or can be connected to an external power source (e.g., the power source 223 of FIG. 2). In these and other embodiments, the drivetrain 318 can include the electric motor 319 and a number of gears, and a battery can be located elsewhere (e.g., another position between the first handle half 216-1 and the second handle half 216-2). As discussed in greater detail below, the gear rack can interact with the drivetrain 318, and the drive shaft 204 can be coupled to a distal end of the gear rack.Atorney Docket No.: P330621.W0.01 (LM8018)

[0031] The power trigger 210 and the selector lever 217 can be used to activate the drivetrain 318. The movable handle 212 can be used to clamp and unclamp the first elongated member 207 (FIG. 2) and the second elongated member 209 (FIG. 2) together, and the selector lever 217 can allow power to flow to the electric motor and / or block the power trigger 210 from being activated. Also shown in FIG. 3 is an outer shaft 361 of the elongated body 261, a nose cone 377, the bailout handle 251, a circuit board 355, an articulation lock 359, toggle plates 352, and a bailout bar 351. The circuit board 355 can include a microcontroller or microprocessor. Additional details and embodiments of surgical stapling apparatuses are provided in U.S. Patent Application No. 18 / 391,251, titled “SAFETY CONTROL ALGORITHMS FOR SURGICAL DEVICES, INCLUDING SAFETY CONTROL ALGORITHMS FOR SURGICAL STAPLER DEVICES, AND ASSOCIATED SYSTEMS, DEVICES, AND METHODS,” and filed on December 20, 2023, the disclosure of which is incorporated herein by reference in its entirety.

[0032] It is appreciated that while select features of the present technology may be described herein with reference to the surgical stapling apparatus 230 illustrated in FIGS. 2 and 3, surgical display systems configured in accordance with various embodiments of the present technology can be used with and for other surgical devices.3. Graphical User Interfaces

[0033] FIGS. 4A and 4B are partially schematic diagrams of a graphical user interface (GUI) 400 configured in accordance with various embodiments of the present technology. The GUI 400 can be displayed on a display (e.g., the display 170 of FIG. 1), and can include a video feed 402 and one or more overlays on and / or around the video feed 402. The video feed 402 can be a real-time video feed from an endoscope (e.g., the endoscope 110 of FIG. 1). For example, if the medical procedure being performed is a laparoscopic cholecystectomy, the video feed 402 can be a real-time laparoscopic video feed. In the illustrated embodiment, the video feed 402 shows a portion of a surgical device 404 (e.g., the surgical stapling apparatus 230 of FIG. 2), tissue 406, and another surgical tool 408 (e.g., forceps). As previously mentioned, a control unit (e.g., the control unit 160 of FIG. 1) operably coupled to the endoscope, the surgical device, and / or the display can generate the one or more overlays based on various data collected and / or input to the control unit.

[0034] Referring first to FIG. 4A, the GUI 400 displays a first overlay 410. In the illustrated embodiment, the video feed 402 does not fill the entire GUI 400. Thus, the GUI 400Atorney Docket No.: P330621.W0.01 (LM8018) can include a peripheral boundary about the video feed 402, and the first overlay 410 can be positioned within the peripheral boundary, such as at a bottom-right comer of the peripheral boundary, without overlapping with the video feed 402. In other embodiments, the first overlay 410 can be positioned elsewhere, can at least partially overlap with the video feed 402, and / or can have various shapes, sizes, colors, and / or opacities. In operation, the first overlay 410 can convey information related to the surgical device 404, the tissue 406, and / or the overall medical procedure. For example, the first overlay 410 can indicate whether the surgical device 404, the endoscope, and / or an endoscope control box (e.g., the endoscope control box 120 of FIG. 1) are properly connected to a control unit (e.g., the control unit 160) and / or to one another. In some embodiments, the first overlay 410 includes an indicator 412 that can display different colors to indicate different connection statuses (e.g., green to indicate that all components are properly connected, red to indicate that at least one component is not properly connected).

[0035] Referring next to FIG. 4B, the GUI 400 displays a second overlay 420, a third overlay 430, and a fourth overlay 440, such as in addition to the first overlay 410. In the illustrated embodiment, the second, third, and fourth overlays 420, 430, 440 are positioned along the bottom of the GUI 400. More specifically, the second, third, and fourth overlays 420, 430, 440 are positioned to at least partially overlap with a bottom portion of the video feed 402. In other embodiments, the second, third, and fourth overlays 420, 430, 440 can be positioned elsewhere; can be positioned to not overlap with the video feed 402; can have a different arrangement or order with respect to one another and / or the first overlay 410; and / or can have various shapes, sizes, colors, and / or opacities. In operation, the second, third, and fourth overlays 420, 430, 440 can convey information related to the surgical device, the tissue, and / or the overall medical procedure. The information conveyed by the second, third, and fourth overlays 420, 430, 440 can be different from the information conveyed by the first overlay 410.

[0036] In some embodiments, the GUI 400 can include a flyout menu. For example, the GUI 400 can selectively display, hide, resize, reposition, and / or change the opacity of the second, third, and / or fourth overlays 420, 430, and / or 440 in response to a user input (e.g., switch back and forth between FIGS. 4A and 4B). The user input can include toggling or clicking a portion of the GUI 400 (e.g., the first overlay 410) using a mouse, hovering a cursor over and / or in the vicinity of any one of the overlays 410, 420, 430, 440, tapping using a touchscreen, pressing a keyboard button, inputting a voice command, and / or a combination thereof (e.g., clicking the first overlay 410 to display the second, third, and / or fourth overlays 420, 430, and / or 440, then hovering the cursor over and / or in the vicinity of any one of theAtorney Docket No.: P330621.W0.01 (LM8018) overlays 410, 420, 430, 440 to continue displaying the second, third, and / or fourth overlays 420, 430, and / or 440). For example, the physician may want to hide the second, third, and / or fourth overlays 420, 430, and / or 440 to have an unobstructed view of the video feed 402.

[0037] In some embodiments, the second overlay 420 can comprise a panel indicating, in text and / or graphical form, a reload and handle state of the surgical device. For example, the second overlay 420 can display whether a reload assembly (e.g., the surgical reloadable cartridge assembly 240 of FIG. 2) is ready to fire (e.g.., ready to cut tissue and / or deploy a staple), loaded with at least one surgical fastener (e.g., a staple), running low on surgical fasteners, in the process of firing (e.g., in the process of cutting tissue and / or deploying a staple), successful in firing (e.g., successful in cutting tissue and / or deploying a staple), etc. Additionally, or alternatively, the second overlay 420 can display information about the reloadable assembly currently attached to the surgical handle apparatus, such as the stroke length of the reload assembly, the tip type (e.g., regular, curved) of the reload assembly, and / or the size (e.g., height, length) of staples loaded into the reload assembly.

[0038] In some embodiments, the third overlay 430 can comprise a panel showing, in text and / or graphical form, one or more messages to the physician. For example, the third overlay 430 can display workflow instructions, such as what step the physician should take next during the medical procedure. In another example, the third overlay 430 can display reload recommendations, such as a message that the reload assembly currently attached to the handle apparatus is not suitable for the current objective (e.g., based on various data collected, such as data indicating the thickness of tissue currently gripped by the jaws) and / or a message that the staples need to be sized up, sized down, or replaced with a different type. In yet another example, the third overlay 430 can display warnings, such as a message that the tissue is too thick for the current surgical device.

[0039] In some embodiments, the fourth overlay 440 can comprise a panel indicating, in text and / or graphical form, operating parameters of the surgical device. In the illustrated embodiment, the fourth overlay 440 includes a firing progress bar 442 and a force capacity bar 444. The firing progress bar 442 can indicate how far or close the staple is from or to being fiilly deployed. For example, the firing progress bar 442 can include a bar (e.g., a rectangular or elongate bar) and a marker 443 (e.g., a vertical line) in the bar to indicate the progress using the position of the marker 443. In some embodiments, two different colors (or other visual features) fill up the two portions of the bar on opposite sides of the marker 443 to better visualize the progress. In some embodiments, the firing progress bar 442 can be supplementedAtorney Docket No.: P330621.W0.01 (LM8018) by text indicating the firing progress as a percentage, as the distance fired (e.g., in mm or another unit), etc.

[0040] The force capacity bar 444 can present information indicating how much force a motor (e.g., the electric motor 319) is applying to clamp tissue and / or deploy a staple. For example, the force capacity bar 444 can include a color gradient bar and a marker 445 (e.g., a vertical line in the color gradient bar) that tracks resistive feedback from tissue clamped between jaws (e.g., the first elongated member 207 and the second elongated member 209) of the reload assembly. In some embodiments, the force capacity bar 444 can include a green region at the center of the color gradient bar and two red regions at the ends of the color gradient bar. The marker 445 can move toward the red region on the right to indicate that the motor is overstressing the reload assembly. For example, the tissue may be too thick or the surgical device may be at an improper angle. The marker 445 can move toward the red region on the left to indicate that the motor is proceeding too fast or too easily. For example, the force may be less than the expected range for a proper reload. In some embodiments, the third overlay 430 can display a warning message about what the control unit anticipates is the cause of the marker moving toward either red region of the force capacity bar 444. The marker 445 may move toward the green region at the center of the color gradient bar to indicate that the surgical device is firing as expected and / or that the motor is exerting nominal forces.

[0041] FIG. 5 is a partially schematic diagram of another GUI 500 configured in accordance with various embodiments of the present technology. The GUI 500 can be generally similar to the GUI 400 of FIG. 4. For example, the GUI 500 can display the video feed 402 and a plurality of overlays including a first overlay 510 having an indicator 512, a second overlay 520, a third overlay 530, and a fourth overlay 540. The first through fourth overlays 510, 520, 530, 540 can display the same or generally similar information as the first through fourth overlays 410, 420, 430, 440 of FIG. 4, respectively. However, as shown, the GUI 500 displays the overlays along the left side of the GUI 500. In particular, the first through third overlays 510, 520, 530 are positioned wholly within a peripheral boundary of the GUI 500 such that they do not obstruct viewing of the video feed 402. The fourth overlay 540 is positioned at the top-left comer of the GUI 500, partially obstructing viewing of the video feed 402. It is appreciated that the illustrated embodiment is merely an example layout of the overlays, and that the video feed 402 and the overlays 510, 520, 530, 540 can be arranged differently and / or have different shapes, sizes, colors, and / or opacities.Atorney Docket No.: P330621.W0.01 (LM8018)

[0042] FIG. 6 is a partially schematic diagram of yet another GUI 600 configured in accordance with various embodiments ofthe present technology. The GUI 600 can be generally similar to the GUI 400 of FIG. 4 or the GUI 500 of FIG. 5. For example, the GUI 600 can display the video feed 402 and a plurality of overlays including a first overlay 610 having an indicator 612, a second overlay 620, a third overlay 630, and a fourth overlay 640. The first through fourth overlays 610, 620, 630, 640 can display the same or generally similar information as the first through fourth overlays 410, 420, 430, 440 of FIG. 4, respectively. However, as shown, the first overlay 610 is positioned at the bottom-left comer of the GUI 600, the third overlay 630 is positioned along the left side of the GUI 600 above the first overlay 610, the fourth overlay 640 is positioned along the left side of the GUI 600 above the third overlay 630, and the second overlay 620 is positioned above the fourth overlay 640.

[0043] Furthermore, unlike in FIGS. 4 and 5, the third overlay 630 includes a firing progress bar 632 (e.g., the firing progress bar 442), and the fourth overlay 640 includes a force capacity bar 644 (e.g., the force capacity bar 444) and omits a firing progress bar. Also, the force capacity bar 644 includes discrete colored zones (e.g., red, yellow, green) as opposed to a spectrum of colors, as included in the force capacity bar 444 of FIG. 4. It is appreciated that the illustrated embodiment is merely an example layout of the overlays, and that the video feed 402 and the overlays can be arranged differently and / or have different shapes, sizes, colors, and / or opacities.

[0044] FIG. 7 is a partially schematic diagram of an overlay 740 displayable on a GUI, each configured in accordance with various embodiments of the present technology. In some embodiments, the overlay 740 can serve as a replacement for the fourth overlays 440, 540, and / or 640 in FIGS. 4—6. The overlay 740 can include a firing progress bar 742 and a force capacity dial 744. The firing progress bar 742 can be identical (or at least generally similar) to the firing progress bar 442 of FIG. 4. In some embodiments, the firing progress bar 742 can include an indicator 746 (e.g., a yellow triangle, or other suitable visual marker) to indicate that an anomalous event has occurred, and can omit the indicator 746 to indicate that no anomalous event has occurred. For example, an anomalous event can include a rapid gear change in the surgical device. If an anomalous event has occurred, the position of the indicator 746 relative to the firing progress bar 742 and / or associated text (e.g., text overlaid on or near the indicator 746) can indicate approximately where (e.g., along the firing stroke) the anomalous event occurred. Continuing with this example and as shown in the illustrated embodiment, the indicator 746 can be used to indicate that an anomalous event occurred at a firing distance ofAtorney Docket No.: P330621.W0.01 (LM8018) about 17 mm into the firing stroke. The presence of the indicator 746 and / or a message on another overlay (e.g., the third overlay 430 of FIG. 4B) can prompt the physician to inspect that particular area of the firing. The force capacity dial 744 can function similarly as the force capacity bar 444 to provide information about the motor (e.g., the electric motor 319 of FIG. 3), but includes a dial- or gauge-like graphic having an arrow that pivots to point to various points along a spectrum that includes multiple colored regions. It is appreciated that the overlay 740 is merely an example overlay. Additional or alternative overlays can, for example, display a color gradient bar or other indicator that indicates tissue thickness, angle or position of the surgical device, etc.

[0045] FIG. 8 is a partially schematic diagram of an overlay 840 displayable on a GUI and configured in accordance with various embodiments of the present technology. The overlay 840 can include a firing progress bar 842 and a force capacity bar 844. The firing progress bar 842 can be identical or substantially similar to the firing progress bar 442 of FIG. 4B. For example, the force capacity bar 844 can present information indicating how much force a motor (e.g., the electric motor 319) is applying to clamp tissue and / or deploy a staple. However, unlike the force capacity bar 444, the force capacity bar 844 of FIG. 8 can have different colors at the left and right ends. For example, the force capacity bar 844 can have a rainbow or inverse rainbow spectrum of colors that includes, from left to right or right to left, violet, indigo, blue, green, yellow, orange, and red. Having unique colors at different points along the length of the force capacity bar 844 can help the physician more readily distinguish between different situations corresponding to the left and right ends, such as when the tissue is deemed too thin versus when the tissue is deemed too thick.

[0046] FIG. 9 is a partially schematic diagram of an overlay 940 displayable on a GUI and configured in accordance with various embodiments of the present technology. The overlay 940 can include a combined firing progress and force capacity bar 942 (“the bar 942”), a marker 945, and an indicator 946. The indicator 946 can be identical (or at least substantially similar) to the indicator 746 of FIG. 7. The marker 945 can move relative to the bar 942 to represent the real-time firing progress of the surgical device. Moreover, as the marker 945 moves (e.g., from left to right), the marker 945 can leave a trail of colors indicating the amount of force applied by a motor at the various firing progress distances. For example, a violet region at one spot in the bar 942 can indicate that the motor applied less than nominal forces at the corresponding firing distance, a red region at another spot in the bar 942 can indicate that the motor applied more than nominal forces at the corresponding firing distance, and a green regionAtorney Docket No.: P330621.W0.01 (LM8018) at yet another spot in the bar 942 can indicate that the motor applied nominal forces at the corresponding firing distance. Therefore, the bar 942 can serve as both a firing progress bar and a historical plot of the force capacity applied throughout the firing progress. A physician can thus use the overlay 940 to, e.g., better assess staple line quality and be alerted to any anomalies.

[0047] Referring to FIGS. 4-9 together, the first through fourth overlays can, alone or in tandem, help the physician perform medical procedures by providing them with information to more accurately and efficiently assess various operating parameters of the procedure. In some cases, the overlays can provide relevant information at multiple, distinct stages of the procedure. For example, in embodiments involving the use of a surgical stapling apparatus (e.g., the surgical stapling apparatus 230), the overlays can, at a first operation stage in which the jaws are clamping the tissue but prior to firing, provide information about tissue thickness and / or the position of the surgical stapling apparatus. The physician may, before firing, adjust the surgical stapling apparatus, swap the reload assembly for another, etc. The overlays can then, at a second operation stage in which the reload assembly is firing, additionally or alternatively provide information about the force capacity of the motor. Therefore, the GUI can guide the physician in performing the medical procedure in real-time.C. Examples

[0048] Several aspects of the present technology are set forth in the following examples. Although several aspects of the present technology are set forth in examples directed to devices and methods, these aspects of the present technology can similarly be set forth in examples directed to methods and devices, respectively, in other embodiments. Additionally, these aspects of the present technology may be set forth in examples directed to systems and / or (e.g., non-transitory) computer-readable media in other embodiments.1. A control unit, comprising: a first input operably coupled to an endoscope and configured to receive a real-time video feed of a surgical site captured using the endoscope; a second input operably coupled to a non-robotic, handheld surgical device (surgical device) and configured to collect real-time data associated with the surgical device, wherein the surgical device is usable to perform a medical procedure at the surgical site; andAtorney Docket No.: P330621.W0.01 (LM8018) a processor operably coupled to the first and second inputs and configured to generate a graphical user interface (GUI) displayable on a screen, wherein the GUI includes: the real-time video feed of the surgical site, and one or more overlays, wherein the overlays are configured to display, in text or graphic form, the real-time data associated with the surgical device.2. The control unit of example 1 wherein the one or more overlays include an overlay configured to indicate whether the first input is properly coupled to the endoscope, whether the second input is properly coupled to the surgical device, or a combination thereof.3. The control unit of example 2 wherein the overlay includes an indicator configured to — display a first color to indicate that the first input is properly coupled to the endoscope and / or the second input is properly coupled to the surgical device, and display a second color different from the first color to indicate that the first input and / or the second input is not properly coupled to the endoscope or the surgical device, respectively.4. The control unit of any of examples 1-3 wherein the one or more overlays include an overlay configured to indicate a reload and handle state of the surgical device.5. The control unit of example 4 wherein the reload and handle state of the non- robotic, handheld surgical device is selected from the following: a reload assembly of the surgical device is ready to fire; the reload assembly is loaded with at least one surgical fastener; the reload assembly is running low on surgical fasteners; the reload assembly has previously been fired; the reload assembly is currently firing a surgical fastener; and the reload assembly was successful in firing the surgical fastener.6. The control unit of any of examples 1-5 wherein the one or more overlays include an overlay configured to display information about (a) surgical fasteners currentlyAtorney Docket No.: P330621.W0.01 (LM8018) loaded in a reload assembly of the surgical device and / or (b) one or more characteristics of the reload assembly.7. The control unit of example 6 wherein the one or more characteristics of the reload assembly include a stroke length and / or a tip type of the reload assembly.8. The control unit of any of examples 1-7 wherein the one or more overlays include an overlay configured to display a message, wherein the message includes at least one of workflow instructions, reload recommendations, or warnings.9. The control unit of any of examples 1-8 wherein the one or more overlays include an overlay configured to indicate a firing progress of a reload assembly of the surgical device.10. The control unit of example 9 wherein the overlay includes a firing bar and a marker, wherein a relative position of the marker within the firing bar indicates the firing progress of the reload assembly.11. The control unit of example 10 wherein the overlay is further configured to display an indicator to indicate that an anomalous event occurred, wherein a position of the indicator relative to the firing bar indicates where the anomalous event occurred along the firing progress of the reload assembly.12. The control unit of any of examples 9-11 wherein the overlay is configured to display at least one of a percentage or a firing distance indicative of the firing progress of the reload assembly.13. The control unit of any of examples 1-12 wherein the one or more overlays include an overlay configured to indicate a force capacity of a motor of the surgical device.14. The control unit of example 13 wherein the overlay includes (i) a color gradient bar or a dial and (ii) a marker, and wherein a position of the marker relative to the color gradient bar or the dial indicates the force capacity of the motor.Atorney Docket No.: P330621.W0.01 (LM8018)15. The control unit of example 13 or example 14 wherein the force capacity of the motor corresponds to resistive feedback from tissue clamped by jaws of the surgical device.16. The control unit of any of examples 1-15 wherein the one or more overlays include a combined firing progress and force capacity bar (bar) and a marker, wherein a position of the marker relative to the bar indicates a firing progress of a reload assembly of the surgical device, wherein the bar is configured to, as the marker moves relative to the bar, display one or more colors at one or more points along a length of the bar, and wherein each color indicates a force capacity of a motor of the surgical device when the marker was at the corresponding point along the length of the bar.17. A method for providing real-time information associated with a non-robotic, handheld surgical device (surgical device), the method comprising: receiving, from an endoscope, a real-time video feed of a surgical site; collecting, from the surgical device, real-time data associated with the surgical device, wherein the surgical device is usable to perform a medical procedure at the surgical site; and generating a graphical user interface (GUI), wherein generating the GUI comprises generating one or more overlays around and / or over the real-time video feed of the surgical site, wherein the one or more overlays are configured to display, in text or graphic form, the real-time data associated with the surgical device.18. The method of example 17, further comprising transmitting instructions for causing the GUI to be presented on a display.19. The method of example 17 or example 18, further comprising displaying the GUI on a display.20. The method of any of examples 17-19 wherein generating the one or more overlays includes: during a first operation stage of the medical procedure in which jaws of the surgical device are clamping tissue, displaying information about a thickness of the tissue; andAtorney Docket No.: P330621.W0.01 (LM8018) during a second operation stage of the medical procedure in which the surgical device is firing a surgical fastener into the tissue, displaying information associated with a force capacity of a motor of the surgical device.21. The method of any of examples 17-20, further comprising: receiving a toggling input; and showing or hiding at least a portion of the one or more overlays in response to the received toggling input.D. Conclusion

[0049] It will be apparent to those having skill in the art that changes may be made to the details of the above-described embodiments without departing from the underlying principles of the present disclosure. In some cases, well known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments of the present technology. Although steps of methods may be presented herein in a particular order, alternative embodiments may perform the steps in a different order. Similarly, certain aspects of the present technology disclosed in the context of particular embodiments can be combined or eliminated in other embodiments. Furthermore, while advantages associated with certain embodiments of the present technology may have been disclosed in the context of those embodiments, other embodiments can also exhibit such advantages, and not all embodiments need necessarily exhibit such advantages or other advantages disclosed herein to fall within the scope of the technology. Accordingly, the disclosure and associated technology can encompass other embodiments not expressly shown or described herein, and the invention is not limited except as by the appended claims.

[0050] To the extent any material incorporated herein by reference conflicts with the present disclosure, the present disclosure controls. Where the context permits, singular or plural terms may also include the plural or singular term, respectively. For example, throughout this disclosure, the singular terms “a,” “an,” and “the” include plural referents unless the context clearly indicates otherwise. Moreover, unless the word “or” is expressly limited to mean only a single item exclusive from the other items in reference to a list of two or more items, then the use of “or” in such a list is to be interpreted as including (a) any single item in the list, (b) all of the items in the list, or (c) any combination of the items in the list. Furthermore, as used herein, the phrase “and / or” as in “A and / or B” refers to A alone, B alone, and both A and B. Additionally, the terms “comprising,” “including,” “having,” and “with” are used throughoutAtorney Docket No.: P330621.W0.01 (LM8018) to mean including at least the recited feature(s) such that any greater number of the same features and / or additional types of other features are not precluded. Moreover, as used herein, the phrases “based on,” “depends on,” “as a result of,” and “in response to” shall not be construed as a reference to a closed set of conditions. For example, a step that is described as “based on condition A” may be based on both condition A and condition B without departing from the scope of the present disclosure. In other words, as used herein, the phrase “based on” shall be construed in the same manner as the phrase “based at least in part on” or the phrase “based at least partially on.”

[0051] Reference herein to “one embodiment,” “an embodiment,” “some embodiments” or similar formulations means that a particular feature, structure, operation, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present technology. Thus, the appearances of such phrases or formulations herein are not necessarily all referring to the same embodiment. Furthermore, various particular features, structures, operations, or characteristics may be combined in any suitable manner in one or more embodiments.

[0052] Unless otherwise indicated, all numbers expressing numerical values used in the specification and claims, are to be understood as being modified in all instances by the term “about.” Accordingly, unless indicated to the contrary, the numerical parameters set forth in the specification and attached claims are approximations that may vary depending upon the desired properties sought to be obtained by the present technology. At the very least, and not as an attempt to limit the application of the doctrine of equivalents to the scope of the claims, each numerical parameter should at least be construed in light of the number of reported significant digits and by applying ordinary rounding techniques. Additionally, all ranges disclosed herein are to be understood to encompass any and all subranges subsumed therein. For example, a range of “1 to 10” includes any and all subranges between (and including) the minimum value of 1 and the maximum value of 10 (e.g., any and all subranges having a minimum value of equal to or greater than 1 and a maximum value of equal to or less than 10, such as 5.5 to 10).

[0053] The disclosure set forth above is not to be interpreted as reflecting an intention that any claim or example requires more features than those expressly recited in that claim or example. Rather, as the preceding examples and the following claims reflect, inventive aspects lie in a combination of fewer than all features of any single foregoing disclosed embodiment. Thus, the preceding examples and the following claims are hereby expressly incorporated intoAtorney Docket No.: P330621.W0.01 (LM8018) the Detailed Description, with each claim standing on its own as a separate embodiment. This disclosure includes all permutations of the independent claims with their dependent claims.

Claims

Atorney Docket No.: P330621.W0.01 (LM8018)CLAIMSI / We claim:

1. A control unit, comprising: a first input operably coupled to an endoscope and configured to receive a real-time video feed of a surgical site captured using the endoscope; a second input operably coupled to a non-robotic, handheld surgical device (surgical device) and configured to collect real-time data associated with the surgical device, wherein the surgical device is usable to perform a medical procedure at the surgical site; and a processor operably coupled to the first and second inputs and configured to generate a graphical user interface (GUI) displayable on a screen, wherein the GUI includes: the real-time video feed of the surgical site, and one or more overlays, wherein the overlays are configured to display, in text or graphic form, the real-time data associated with the surgical device.

2. The control unit of claim 1 wherein the one or more overlays include an overlay configured to indicate whether the first input is properly coupled to the endoscope, whether the second input is properly coupled to the surgical device, or a combination thereof.

3. The control unit of claim 2 wherein the overlay includes an indicator configured display a first color to indicate that the first input is properly coupled to the endoscope and / or the second input is properly coupled to the surgical device, and display a second color different from the first color to indicate that the first input and / or the second input is not properly coupled to the endoscope or the surgical device, respectively.

4. The control unit of claim 1 wherein the one or more overlays include an overlay configured to indicate a reload and handle state of the surgical device.Atorney Docket No.: P330621.W0.01 (LM8018)5. The control unit of claim 4 wherein the reload and handle state of the non- robotic, handheld surgical device is selected from the following: a reload assembly of the surgical device is ready to fire; the reload assembly is loaded with at least one surgical fastener; the reload assembly is running low on surgical fasteners; the reload assembly has previously been fired; the reload assembly is currently firing a surgical fastener; and the reload assembly was successful in firing the surgical fastener.

6. The control unit of claim 1 wherein the one or more overlays include an overlay configured to display information about (a) surgical fasteners currently loaded in a reload assembly of the surgical device and / or (b) one or more characteristics of the reload assembly.

7. The control unit of claim 6 wherein the one or more characteristics of the reload assembly include a stroke length and / or a tip type of the reload assembly.

8. The control unit of claim 1 wherein the one or more overlays include an overlay configured to display a message, wherein the message includes at least one of workflow instructions, reload recommendations, or warnings.

9. The control unit of claim 1 wherein the one or more overlays include an overlay configured to indicate a firing progress of a reload assembly of the surgical device.

10. The control unit of claim 9 wherein the overlay includes a firing bar and a marker, wherein a relative position of the marker within the firing bar indicates the firing progress of the reload assembly.

11. The control unit of claim 10 wherein the overlay is further configured to display an indicator to indicate that an anomalous event occurred, wherein a position of the indicator relative to the firing bar indicates where the anomalous event occurred along the firing progress of the reload assembly.Atorney Docket No.: P330621.W0.01 (LM8018)12. The control unit of claim 9 wherein the overlay is configured to display at least one of a percentage or a firing distance indicative of the firing progress of the reload assembly.

13. The control unit of claim 1 wherein the one or more overlays include an overlay configured to indicate a force capacity of a motor of the surgical device.

14. The control unit of claim 13 wherein the overlay includes (i) a color gradient bar or a dial and (ii) a marker, and wherein a position of the marker relative to the color gradient bar or the dial indicates the force capacity of the motor.

15. The control unit of claim 13 wherein the force capacity of the motor corresponds to resistive feedback from tissue clamped by jaws of the surgical device.

16. The control unit of claim 1 wherein the one or more overlays include a combined firing progress and force capacity bar (bar) and a marker, wherein a position of the marker relative to the bar indicates a firing progress of a reload assembly of the surgical device, wherein the bar is configured to, as the marker moves relative to the bar, display one or more colors at one or more points along a length of the bar, and wherein each color indicates a force capacity of a motor of the surgical device when the marker was at the corresponding point along the length of the bar.

17. A method for providing real-time information associated with a non-robotic, handheld surgical device (surgical device), the method comprising: receiving, from an endoscope, a real-time video feed of a surgical site; collecting, from the surgical device, real-time data associated with the surgical device, wherein the surgical device is usable to perform a medical procedure at the surgical site; and generating a graphical user interface (GUI), wherein generating the GUI comprises generating one or more overlays around and / or over the real-time video feed of the surgical site, wherein the one or more overlays are configured to display, in text or graphic form, the real-time data associated with the surgical device.Atorney Docket No.: P330621.W0.01 (LM8018)18. The method of claim 17, further comprising transmitting instructions for causing the GUI to be presented on a display.

19. The method of claim 17, further comprising displaying the GUI on a display.

20. The method of claim 17 wherein generating the one or more overlays includes: during a first operation stage of the medical procedure in which jaws of the surgical device are clamping tissue, displaying information about a thickness of the tissue; and during a second operation stage of the medical procedure in which the surgical device is firing a surgical fastener into the tissue, displaying information associated with a force capacity of a motor of the surgical device.

21. The method of any of claim 17, further comprising: receiving a toggling input; and showing or hiding at least a portion of the one or more overlays in response to the received toggling input.

Images