Intraoperative video assist device

Abstract

A method includes identifying one or more surgically-related characteristics and correlating the one or more surgically-related characteristics with a surgical instrument layout stored in a database and based on a surgical procedure to be performed. The surgical instrument layout includes a type and quantity of one or more surgical instruments one or more physical dimensions of the one or more surgical instruments, and orientations of the one or more surgical instruments. The method further includes displaying on a display screen the surgical instrument layout; and projecting the surgical instrument layout onto a surgical table.

Description

INTRAOPERATIVE VIDEO ASSIST DEVICECROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of and priority to U.S. Provisional Application No. 63 / 661 ,406, filed on June 18, 2024. The disclosure of the prior application is hereby incorporated by reference herein in its entirety.TECHNICAL FIELD

[0002] The present disclosure relates generally to an intraoperative video assist device and methods for operating the same. More specifically, systems associated with the intraoperative video assist device are used, for example, during surgical procedures.BACKGROUND

[0003] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.

[0004] During surgical procedures, a surgeon is typically assisted by a number of medical personnel, such as a scrub technician or surgical technician, who are responsible for organizing and providing surgical instruments used during the surgical procedure. These technicians often handle the preparation of surgical instruments prior to the surgical procedure, which typically involves removing the surgical instruments from a tray and arranging them on a table, such as a Mayo stand or back table, for easy access during the surgical procedure.

[0005] The job requirements of a surgical technician (and other medical personnel) can include challenging tasks, one of which is identifying various surgical instruments. The identification of surgical instruments can be difficult due to factors such as the sheer number of instruments (e.g., thousands) and the existence of numerous variants of the same surgical instrument. For example, a Westcott scissor used by an eye surgeon may come in different configurations, such as a straight blade, a curved blade, a pointed tip, a blunt tip, or a combination of these features. Additionally, the challenge of identifying surgical instruments can be compounded for new surgical technicians or for experienced technicians who are unfamiliar with a particular surgical field. For instance, a surgical technician well-versed in one field maynot be familiar with the surgical instruments used in another, yet surgical technicians are often required to assist in unfamiliar fields.

[0006] An additional challenge in identifying surgical instruments is their varieties of sizes, as many can appear similar when viewed without magnification, such as surgical instruments used in microsurgery. For example, a Sinskey Hook, a Kuglen Hook, and a Graether Collar Button may look indistinguishable to the naked eye.

[0007] In many procedures, especially in microsurgery, operating room lights must be turned off to enhance the surgeon’s view through a surgical microscope. However, this presents a challenge because surgical technicians, who do not use microscopes, typically rely on ceiling-mounted surgical lamps to illuminate the surgical instrument table. This additional lighting can distract the surgeon and, at the very least, reduce the quality of vision through the microscope.

[0008] Another challenge, arising from the typical use of operating room lights and the lack of a magnified view, is that surgical technicians often have difficulty preparing surgical instruments for the surgeon’s use during the procedure. For example, microsurgical needles with attached suture material, both of which are often thinner than a human hair, must be grasped with a needle holder and handed to the surgeon at specific times. Improper grasping of the needle is thus not uncommon due to its small size.

[0009] Additionally, the visibility challenges in an operating room (e.g., insufficient lighting) make properly grasping the needle even more difficult. As a result, the surgical technician may inadvertently break the suture material off of the needle while attempting to grasp it or remove it from its packaging. Often, it is only when the surgeon views the needle holder through the microscope that they realize the needle has been improperly grasped. If the needle is identified as improperly grasped, the surgeon must pause the procedure to correct the grasp before suturing can proceed, thereby prolonging the surgery.

[0010] Another challenge faced by surgical technicians is the difficulty in detecting damage to microsurgical instruments, which is typically only identified by the surgeon when viewed through the microscope. Such damage often occurs during cleaning, sterilization, or packaging before the procedure. Once detected, the damaged instrument must be replaced, leading to a prolonged surgery and causing frustration for the surgeon.

[0011] The present disclosure addresses these and other issues related to providing instruments, particularly in a surgical environment.SUMMARY

[0012] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key factors or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0013] The present disclosure provides a method comprising: receiving a magnification of a surface from a camera associated with a multi-functional component; displaying, on a display screen, the magnified surface; identifying one or more surgical instruments based on one or more surgically-related characteristics; and causing the one or more surgical instruments to be projected onto the surface; wherein the one or more surgically-related characteristics include a name associated with each of the one or more surgical instruments, one or more preference cards associated with one or more surgeons, an instrument profile, a surgical specialty, a surgeon, a surgical procedure, or a combination thereof; wherein the display screen is a touch screen interactive with a user; wherein an electronic display associated with the display screen includes a microprocessor, and wherein the microprocessor includes a database that stores the one or more surgically-related characteristics; further comprising: counting one or more actual surgical instruments prior to commencement of a surgery; and counting the one or more actual surgical instruments after completion of the surgery; further comprising: generating an alarm based on the count of the one or more actual surgical instruments prior to the commencement of the surgery not matching the count of the one or more actual surgical instruments after completion of the surgery; further comprising: determining an error based on the projected one or more surgical instruments not matching one or more actual surgical instruments; wherein the projection of the one or more surgical instruments on the surface further comprises: causing the one or more surgically-related characteristics to be projected onto the surface; and further comprising: reducing the received magnification of the surface.

[0014] The present disclosure provides an apparatus comprising: a first pivot arm coupled to an electronic display, wherein the first pivot arm is configured to variably position the electronic display about a horizontal axis; and a second pivot arm coupled to the electronic display and a multi-functional component that includes at least one light, at least one camera, at least one microphone, at least one projector, or a combination thereof, wherein: the second pivot arm is configured to variably position the multi-functional component about the horizontal axis, the at least one light is configured to illuminate a surface, the at least one camera is configured to magnify the surface, and the electronic display is configured to display the magnified surface on a display screen; wherein the at least one microphone is configured to receive one or more voice commands associated with an operation of the display screen; wherein the electronic display is further configured to: receive a magnification of a surface from a camera associated with a multi-functional component; display, on a display screen, the magnified surface; identify one or more surgical instruments based on one or more surgically-related characteristics; and cause the one or more surgical instruments to be projected onto the surface; wherein the electronic display is further configured to: count one or more actual surgical instruments prior to commencement of a surgery; and count the one or more actual surgical instruments after completion of the surgery; wherein the electronic display is further configured to: generate an alarm based on the count of the one or more actual surgical instruments prior to the commencement of the surgery not matching the count of the one or more actual surgical instruments after completion of the surgery; wherein the electronic display is further configured to: determine an error based on the projected one or more surgical instruments not matching one or more actual surgical instruments; wherein the one or more surgically- related characteristics include a name associated with each of the one or more surgical instruments, one or more preference cards associated with one or more surgeons, an instrument profile, a surgical specialty, a surgeon, a surgical procedure, or a combination thereof; wherein the display screen is a touch screen interactive with a user; wherein the electronic display associated with the display screen includes a microprocessor, and wherein the microprocessor includes a database that stores the one or more surgically-related characteristics; wherein the at least one projector is configured to project the one or more surgical instruments on the surface, the one or more surgically-related characteristics on the surface, or a combination thereof; wherein the display screen is further configured to: display the one or more preferencecards; and wherein the display screen is further configured to: reduce or enlarge the received magnification of the surface.

[0015] Further areas of applicability will become apparent from the description provided herein. It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure.BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order that the disclosure may be well understood, there will now be described various forms thereof, given by way of example, reference being made to the accompanying drawings, in which:

[0017] FIG. 1 is a diagram illustrating a system including an intraoperative video assist device in accordance with one or more embodiments of the present disclosure.

[0018] FIG. 2 is a surgical table containing a plurality of surgical instruments in accordance with one or more embodiments of the present disclosure.

[0019] FIG. 3 is an illustrative embodiment of a surgical instrument layout shown on a display screen in accordance with one or more embodiments of the present disclosure.

[0020] FIG. 4 is an illustrative embodiment of a projected surgical instrument layout on a surgical table in accordance with one or more embodiments of the present disclosure.

[0021] FIG. 5A is an image of unmagnified surgical instruments in accordance with one or more embodiments of the present disclosure.

[0022] FIG. 5B is an image of magnified portions of surgical instruments in accordance with one or more embodiments of the present disclosure.

[0023] FIG. 6 is an unmagnified image of a surgical table illustrating a surgical technician an unmagnified surgical instrument and a superimposed illustrativeexample of the display screen containing a magnification of the surgical table in accordance with one or more embodiments of the present disclosure.

[0024] FIG. 7 is a displaying showing a surgical procedure video on a display screen in accordance with one or more embodiments of the present disclosure.

[0025] FIG. 8 is a flowchart illustrating an example method for identifying and displaying a surgical instrument layout in accordance with one or more embodiments of the present disclosure.

[0026] FIG. 9 is a block diagram illustrating an example computer system in accordance with one or more embodiments of the present disclosure.

[0027] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way.DETAILED DESCRIPTION

[0028] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, or uses. It should be understood that throughout the disclosure, reference numerals indicate like or corresponding parts and features. The herein described implementations are not necessarily limited to the particular embodiment or embodiments disclosed and are provided merely as examples.

[0029] To the extent one or more advantages are disclosed herein, the advantage(s) does not in any way suggest that any one of the embodiments necessarily provides all the described advantages or that all the embodiments necessarily provide any one of the described advantages.

[0030] One or more herein described examples of the present disclosure include systems and methods providing an intraoperative video assistant (IOVA), as illustrated in FIGS. 1-9, that provides at least visual assistance during preparation, execution, and conclusion of a surgery.

[0031] FIG. 1 is a diagram illustrating a system 100 configured as an intraoperative video assist device (herein referred to as “the system”). The system 100 can include a first pivot arm 102, an electronic display 104, a second pivot arm 106, and a multi-functional component 108. Additionally, the system 100 and correspondingmethods can project a surgical instrument layout 110 as described in more detail herein.

[0032] The first and second pivot arms 102, 106 can be a mechanical component that facilitates rotational movement around a fixed point, which can be called a pivot. The first and second pivot arms 102, 106 in some implementations consist of a lever-like structure (i.e., a bar) connected to the pivot, allowing application of force to produce rotational motion around the pivot. Rotational motion is often used to move a tool or load attached to the end of the pivot arm. The first and second pivot arms 102, 106 additionally consist of two ends, a first end and a second end. Either end can be attached to a mounting point and facilitate rotation with respect to the mounting point. The first end of the first pivot arm 102 can be attached to a loadbearing portion of a ceiling, a mechanical structure built around the system 100 or may be coupled using any other method or system for attaching to and supporting the first pivot arm 102. For example, the first end of the first pivot arm 102 can be mechanically coupled to mechanical scaffolding surrounding the system 100. In another example, the first end can be mechanically coupled to a ceiling mount. The second end of the first pivot arm 102 can be mechanically coupled to the electronic display 104. Additionally, a pivot arm can include two or more pivots, where each pivot can allow rotation in a different axis. For example, the first pivot arm 102 can include a first pivot which facilitates rotation around a horizontal axis and a second pivot which facilitates rotation around a vertical axis. The second pivot arm 106 can similarly include two pivots.

[0033] The first pivot arm 102 can be configured to facilitate variable positioning of the electronic display 104, the second pivot arm 106, and the multifunctional component 108 around a horizontal axis relative to the mounting point of the first end of the first pivot arm 102. In other words, the first pivot arm 102 can allow the electronic display 104, the second pivot arm 106, and the multi-functional component 108 to rotate or spin while maintaining a constant height.

[0034] The electronic display 104 can be any device that, for example, receives user input, performs computational processes of the disclosed methods, and visually presents information using electronic technologies. The electronic display 104 can include a display screen and one or more microprocessors in communication with a database 928 (described in more detail herein with reference to FIG. 9). The electronic display 104 can receive user input from one or more sources, including acomputer mouse, a keyboard, interaction with tactile buttons within the electronic display 104, a microphone receiving voice commands, or any other method. In various implementations, the electronic display 104 can be an interactive touch screen which can receive and process tactile input as user input.

[0035] In some embodiments, the database 928 can be located external to the system 100. The system 100 can electronically communicate with the database 928 via a network 926. In some embodiments, the database 928 can be included in the electronic display 104. The database 928 can store surgical instrument layouts, which can include a type and quantity of surgical instruments, physical dimensions of the surgical instruments, and sizes and orientations of the surgical instruments, among other information as described in more detail herein.

[0036] The electronic display 104 can be mechanically coupled to the second pivot arm 106, which in turn can be mechanically coupled to the multifunctional component 108. A first end of the second pivot arm 106 can be attached to a rotating mount or coupled using any other method or system for attaching to and supporting the second pivot arm 106 with the electronic display 104. The attachment of the first end of the second pivot arm 106 can facilitate vertical and / or horizontal rotation of the second pivot arm 106 and the multi-functional component 108. In other words, in some embodiments, the second pivot arm 106 and multi-functional component 108 can be rotated vertically and / or horizontally.

[0037] The second pivot arm 106 can be configured to facilitate variable rotation of the multi-functional component 108 around a horizontal axis relative to the first end of the second pivot arm 106. In other words, the second pivot arm 106 can allow the multi-functional component 108 to rotate or spin while maintaining a constant height.

[0038] The multi-functional component 108 in various embodiments can include a light source, a camera, a microphone, a projector, or a combination thereof. The light source can be configured to illuminate a surgical table positioned beneath the multi-functional component. The camera can be configured to generate an image of the surface of a surgical table and transmit the image for further processing, including but not limited to counting actual surgical instruments on the surgical table. In some embodiments, the camera can transmit the image to the electronic display 104. The microphone can be configured to receive voice commands associated with an operation of the electronic display or display screen. Correspondingly, theelectronic display can be configured to receive the voice commands and perform a corresponding task. The projector can be configured to project a surgical instrument layout 110 onto the surgical table.

[0039] FIG. 2 is a surgical table 200 containing a plurality of surgical instruments. The surgical table 200 can be a specialized table used in operating rooms to hold and organize surgical instruments and supplies during a surgical procedure. The surgical table 200 can ensure that all necessary surgical instruments are easily accessible to the surgical team in a sterile, organized manner.

[0040] In use, organizing and tracking for surgical instruments on a surgical table 200 can include multiple difficulties, including but not limited to: visually distinguishing similar-looking surgical instruments; properly preparing small, dexterous surgical instruments; and accounting for all necessary surgical instruments, both before and after surgical procedures. The disclosed systems and methods can address these and other difficulties by identifying surgically-related characteristics, correlating the surgically-related characteristics with a surgical instrument layout, displaying the surgical instrument layout on the display screen, and projecting the surgical instrument layout onto the surgical table as described in more detail herein. In some embodiments, the disclosed systems and methods additionally facilitate counting actual surgical instruments prior to commencement of a surgical procedure and after completion of a surgical procedure. It should be appreciated that the systems and method described herein, including the system 100 can be incorporated or provided in combination with different types of operating room equipment. For example, the systems and method described herein be provided as an integral part of a phacoemulsfication console used for cataract surgery or an arthroscopy / video tower used in orthopedics. Moreover, in some embodiments, the systems and method described herein are provided as a freestanding device or a non-freestanding device.

[0041] FIG. 3 is an illustrative embodiment of a surgical instrument layout shown on a display screen 300. The display screen 300 can be a user interface that includes interactive user interface elements 302, 304, and 306 corresponding to surgically-related characteristics or information (e.g., instrument profile including specialty, surgeon, procedure to be performed, etc.). The display screen 300 can additionally include interactive user interface elements corresponding to selectable and performable operations, including user interface elements 308, 310, 312, 314,316, and 318 (e.g., to initiate an instrument count, to end an instrument count, to perform magnification, to acquire an image or video, to enable a light, to turn on a projector, etc.). The display screen 300 can additionally include visual user interface elements 320 corresponding to a surgical instrument layout. That is, the visual user interface elements 320 are representative of one or more surgical instruments as described in more detail herein.

[0042] One or more embodiments use surgically-related characteristics as part of the systems and methods described herein, such as to facilitate identification of instruments, positioning of instruments, organization of instruments, etc. In various embodiments, the surgically-related characteristics are qualities, descriptions, features, or considerations of an object, person, material, device, surgical instrument, or surgical procedure used in or applicable to a surgical setting. Surgically-related characteristics can include surgical instrument names, preference cards (e.g., detailed documents which outline specific surgical instruments, supplies, techniques, and setup for different surgical procedures associated with one or more surgeons), instrument profiles, surgical specialties, surgeon names, surgical procedure names, or a combination thereof, among other. Surgically-related characteristics, when obtained or identified, can be used by the system 100 to correlate and display a surgical instrument layout as described in more detail herein. Surgically-related characteristics can be identified via user input or other means, including one or more of a plurality of methods for interacting with one or more of the interactive user interface elements of the display screen 300 as described herein.

[0043] In use, for example, the interactive user interface element 302 can be used to select and display a surgical specialty, which can include head & neck, heart, orthopedic, brain, eye, gastrointestinal, colon & and rectal, plastic, and bariatric, among others. The interactive user interface element 304 can be used to select and display a surgeon’s name. A surgeon’s name can include a first name, last name, middle name, middle name initial, or combination thereof. The interactive user interface element 306 can be used to select and display a surgical procedure name. Surgical procedure names can include thyroidectomy, appendectomy, hysterectomy, cholecystectomy, hernia repair, prostatectomy, mastectomy, among others.

[0044] Upon identifying surgically-related characteristics (via interaction with interactive user interface elements 302, 304, and / or 306, or any other method), asurgical instrument layout can be correlated and obtained from the database 928. Correlating the surgical instrument layout can include using the identified surgically- related characteristics as criteria to search the database 928 and identify a corresponding surgical instrument layout. For example, identified surgically-related characteristics can include head & neck, Dr. Smith, and Thyroidectomy. The identified surgically-related characteristics can be used to search through the database 928 and identify a surgical instrument layout that corresponds to all of the surgically-related characteristics.

[0045] In various embodiments, the interactive user interface elements 308, 310, 312, 314, 316, and 318 can be interacted with to cause performance of one or more operations of the disclosed system 100. For example, the interactive user interface element 308 can be selected to initiate a count of the actual surgical instruments on the surgical table before commencement of a surgical procedure. Actual surgical instruments can include surgical instruments positioned on the surgical table. The interactive user interface element 310 can be selected to end a count of the actual surgical instruments on the surgical table after commencement of the surgical procedure. That is, the area, region, count process, etc. can be controlled using one or more interactive user interface elements.

[0046] Counting actual surgical instruments in various embodiments can include utilizing the camera of the multi-functional component 108 to receive an image of actual surgical instruments placed on the surgical table and can generate a count. The image can be provided to and obtained from the electronic display 104 and applied to one or more image processing methods to generate a count of actual surgical instruments. The one or more image processing methods can include identifying one or more surgically-related characteristics in the image and correlating the identified one or more surgically-related characteristics with one or more surgical instruments (e.g., an object matching process). The count of actual surgical instruments can include a type and quantity of actual surgical instruments. In other words, counting actual surgical instruments can include determining a quantity and type of actual surgical instruments positioned on the surgical table.

[0047] In some embodiments, the image processing methods can include a machine learning model configured to receive the image and generate a count of surgical instruments included in the image. The machine learning model can include an artificial neural network, a support vector machine, a decision tree, arandom forest, among other machine learning models and a combination thereof. The image processing methods can identify one or more actual surgical instruments based upon surgically-related characteristics identified in the image.

[0048] The image processing methods can generate a count of actual surgical instruments which can be compared against the corresponding surgical instrument layout. The system 100 can determine an error based on a degree to which the surgical instrument layout matches the actual surgical instruments.

[0049] Comparing the count of actual surgical instruments and the corresponding surgical instrument layout can include contrasting the type and quantity of actual surgical instruments of the actual surgical instrument count and the type and quantity of surgical instruments of the surgical instrument layout.

[0050] If the count of actual surgical instruments (the “count”) does not match the corresponding surgical instrument layout, an alarm can be generated. The alarm can be visual and / or auditory and broadcast via the electronic display 104 or any ancillary system connected to the electronic display 104. The alarm can alert system users that the count of actual surgical instruments does not match the surgical instrument layout (e.g., a desired layout), the arrangement or positioning of the instruments does not match the surgical instrument layout, the types of surgical instruments does not match the surgical instrument layout, etc.

[0051] In some embodiments, the count can be determined, and the corresponding comparison with surgical instrument layout performed, before commencement of a surgical procedure. In some embodiments, the count can be determined, and the corresponding comparison with surgical instrument layout performed, after completion of a surgical procedure.

[0052] Additionally, an error of other difference can be determined based on a degree to which the count of actual surgical instruments matches the surgical instrument layout. For example, while engaging in counting and identifying the actual surgical instruments present, the surgical instrument layout may or may not be actively projected. The error or difference can be presented in a plurality of formats or using different information, including for example, indicating the name of the surgical instrument that is missing and the surgical instrument that is needed, indicating that more than one of the same surgical instrument is missing (e.g., two Mosquito clamps are missing), indicating one or more damaged surgical instruments, etc. The differenceor error can be presented on the display screen 300, stored in the database 928, or stored or presented in any other medium.

[0053] It should be noted that one or more embodiments can be used to identify or determine other issues with the surgical instruments. For example, detecting damage to one or more of the surgical instruments can be performed using the systems and methods described herein. This can be performed separate from or in combination with the matching operations. The various embodiments can use different object matching techniques, etc. to facilitate detection of a number of surgical instruments, a condition of the surgical instrument, etc., such as based on defined or known layouts, templates, etc.

[0054] The interactive user interface element 312 can be selected to cause increased or decreased magnification of one or more actual surgical instruments on the surgical table. When an increase in magnification is requested (via selection of interactive user interface element 312 or another method), the system 100 can receive an image of actual surgical instruments and cause magnification of at least a portion of the image. As described with reference to FIG. 6, providing magnification can include receiving a magnification of the surface of the surgical table and displaying a magnified surface on the display screen 300. Interactive user interface element 312 can include two sub-interactive user interface elements, one for increasing magnification and one for decreasing magnification. When a decrease in magnification is requested, the system can reduce a magnification level applied to a currently magnified image being displayed on the display screen 300. Additionally, if a minimal level of magnification is being applied, upon request of a decrease in magnification, the system 100 can stop displaying the magnified image on the display screen 300. It should be appreciated that in various embodiments a magnification of a surface can be received from a single camera or from a plurality of cameras.

[0055] The interactive user interface element 314 can be selected to cause an image of the surface of the surgical table to be obtained and displayed on the display screen 300. The displayed image can be manipulated by interactions with the display screen 300 in a plurality of ways, including translation, rotation, scaling, shearing, and / or cropping, among others. In some implementations, the displayed image can later be magnified via interaction with interactive user interface element

[0056] The interactive user interface element 316 can be selected to cause illumination of the surgical table via activation of the light source included in the multi-functional component 108. The light source can include any light-emitting device or source. For example, the light source can be a light-emitting diode (LED), florescent light, halogen lamp, incandescent lamp, among others.

[0057] The interactive user interface element 318 can be selected to cause projection of the surgical layout onto the surgical table via the projector included in the multi-functional component 108. When the surgical layout is being projected onto the surgical table, interaction with interactive user interface element 318 can cause projection of the surgical layout to cease. Projection of the surgical instrument layout is described in more detail with reference to FIG. 4.

[0058] The visual user interface elements 320 can include one or more visual representations of surgical instruments included in the surgical instrument layout. Additionally, names for the surgical instruments can be included in combination with the visual user interface elements 320. In the illustrated example, the visual user interface elements 320 include visual representations and names of a muscle hook, needle holder, vannas scissor, smooth forceps, toothed forceps, and / or syringe, among others. However, any type of visual representation of different objects can be provided. The visual user interface elements 320 can include all of or a portion of the surgical instruments included in the surgical instrument layout.

[0059] FIG. 4 is an illustrative embodiment of a projected surgical instrument layout 400 on a surgical table. The projected surgical instrument layout 400, similar to the surgical instrument layout displayed on the display screen 300, can include visual user interface elements 402 corresponding to surgical instruments included in the projected surgical instrument layout 400. Additionally, names for the surgical instrument can be included in combination with the visual user interface elements 402. For example, the visual user interface elements 402 can include visual representations and names of a muscle hook, needle holder, vannas scissor, smooth forceps, toothed forceps, and / or syringe, among others. As is shown, the projected surgical instrument layout 400 includes surgical instruments in specific orientations and positions. These orientations and positions can be altered or modified by one or more methods to match desired orientations.

[0060] For example, a preference card can include preferential orientations and / or positions for surgical instruments. When the preference card isused to correlate a surgical instrument layout, a corresponding projected surgical instrument layout can present the surgical instruments in a corresponding orientation. It should be noted that the preference card in various embodiments corresponds to information for a particular layout stored in the database 928. It should be noted that in some embodiments the preference card correspond to a generic or non-customized surgical instrument layout instead of a surgeon’s preferred surgical instrument layout.

[0061] Projection of surgical instrument layouts can enable surgical procedure preparation to be easier and less prone to error by providing a visual guide for medical personnel preparing a surgical environment. In other words, when preparing a set of surgical instruments, the projected surgical instrument layout can make it easier to visually confirm all necessary surgical instruments are accounted for.

[0062] It should appreciate that variations and modifications are contemplated. For example, in one or more embodiments, the surgical instrument layout 400 is displayed on the surgical table itself. That is, in one or more embodiments, the surgical table is a video screen or display, or include or has integrated therewith a video screen or display, configured to display the surgical instrument layout 400 along with other information (e.g., display the instrument images and names on the surface of the table without using a projector or otherwise having to project image onto the table). In some embodiments, the surgical table is covered by a clear, sterile plastic so that the instruments can be seen on the display.

[0063] FIG. 5A is an image 500 of unmagnified surgical instruments 502. As is shown, the unmagnified surgical instruments 502 can be difficult to visually distinguish, wherein the tip of both unmagnified surgical instruments 502 are visually similar. The difficulty distinguishing surgical instruments can lead to surgical delays and mistakes. However, by providing magnification, such surgical instruments can be more easily distinguished.

[0064] FIG. 5B is an image 504 of magnified surgical instruments 506, 508 corresponding to the unmagnified surgical instruments 502. As is shown, with magnification, the surgical instruments 506, 508 are more easily distinguished, i.e., the tip of surgical instrument 506 can be readily seen as being a point while the tip of surgical instrument 508 can be readily seen as having a key-shaped tip. Providing magnification via the system 100 can allow easier identification and distinguishing of these and other surgical instruments.

[0065] Similarly, magnification can allow easier preparation of surgical instruments. FIG. 6 is an unmagnified image 602 of a surgical table where a surgical technician is grasping an unmagnified surgical instrument 604, and a superimposed illustrative example of the display screen containing a magnified image 606 of the surgical table. The magnification includes a magnified surgical instrument 608.

[0066] The surgical instrument shown in both the unmagnified image 602 and the magnified image 606 is a needle meant to be grasped by a needle holder at its middle portion. However, as is shown in the unmagnified image 602, the unmagnified surgical instrument 604 is relatively small and difficult to view with a high level of detail. Thus, properly grasping the surgical instrument can be difficult to perform and to confirm. However, in the magnified image 606, the positioning of the magnified surgical instrument 608 can be more easily confirmed to be improper, i.e., not at the middle portion. Therefore, the magnification functionality of the system 100 advantageously allows for increased accuracy when preparing surgical instruments.

[0067] The system 100 can additionally display videos of surgical procedures (e.g., surgical procedure videos) for educational and referential purposes. FIG. 7 is an illustrative embodiment of a surgical procedure video 702 on a display screen 700. The user interface on the display screen 700 can include an option to display videos of surgical procedures (e.g., previous procedures performed by one or more surgeons that are downloaded from the Internet or from a database stored locally within the system 100). Once selected, the video 702 can include guidance for, or an example of, performing or completing the surgical procedure or a portion of the surgical procedure. For example, the video 702 can include an ophthalmic surgery (eye surgery). In some embodiments, videos can be coordinated with the surgical procedure selected with interactive user interface element 306. In other words, by selecting a particular surgical procedure with interactive user interface element 306, a selection to display a video can cause display a video showing an example of the particular surgical procedure. For example, if the interactive user interface element 306 has been selected to be a thyroidectomy, choosing to display a video can play a corresponding video of a thyroidectomy, without the need for specific video selection. It should be appreciated that the video and other data can be obtained from different sources. For example, the surgical videos or patient records can be obtained from the Internet, cloud storage, local storage, or a combination thereof, among other sources.

[0068] FIG. 8 is a flowchart illustrating an example method 800 for identifying and displaying a surgical instrument layout. The method 800 begins at operation 802, when the system 100 is activated upon receiving a user input. The user input can be received by selecting an appropriate interactive user interface element on the display screen or providing user input with another manner (e.g., a mouse and keyboard, voice commands, or any other method) as described in more detail herein.

[0069] At operation 804, the one or more surgically-related characteristics can be identified by the system 100. The surgically-related characteristics can be identified from a set of surgically-related characteristics received via user interaction with the system 100, e.g., by user interaction with one or more interactive user interface elements 302, 304, and 306, and / or another method, as described in more detail herein.

[0070] At operation 806, the surgically-related characteristics can be correlated with a surgical instrument layout stored in the database 928. Correlating a surgical instrument layout can include using the surgically-related characteristics as search criteria to locate a corresponding surgical instrument layout as described in more detail herein.

[0071] At operation 808, the surgical instrument layout can be displayed on the display screen as described in more detail herein. The display surgical instrument layout can include one or more surgical instruments orientations and names.

[0072] At operation 810, upon receipt of further user input, the surgical instrument layout can be projected onto a surgical table. The projected surgical instrument layout can include the same information as the surgical instrument layout displayed on the display screen as described in more detail herein.

[0073] At step operation, the method ends.

[0074] In various implementations, the methods described herein can be performed via voice commands. The microphone of the multi-functional component can be configured to receive voice commands and cause performance of corresponding operations. Voice commands can include spoken words or phrases which the system 100 recognizes as user input that corresponds to an operation of the display screen (e.g., related to one or more of the interactive user interface elements 302, 304, 306, 308, 310, 312, 314, 316, 318) or any other method the system 100 can perform or cause to be performed. The electronic display 104 can beadditionally configured to receive the voice commands from the microphone and perform a corresponding task. For example, a voice command can be “Turn on the light.” Upon receipt of the voice command, the electronic display 104 can turn on the light included in the multi-functional component 108, similar to a user interacting with the interactive user element 316. In another example, a voice command can be “Magnify the surgical table.” Upon receipt of the voice command, the electronic display 104 can display a magnified image of the surgical table on the display screen 300.

[0075] Receiving voice commands and causing performance of corresponding operations can include providing the received voice commands to a machine learning model configured to receive voice commands and generate computer-executable instructions. In some embodiments, the machine learning model can be a large language model (“LLM”). In some embodiments, the LLM (or any machine learning model used) can be stored at a remote location and communicate with the system 100 via a network 926 (described in more detail with reference to FIG. 9).

[0076] The generated computer-executable instructions can be received by the electronic display 104 and used to perform one or more corresponding operations, including the operations associated with the interactive user interface elements 302, 304, 306, 308, 310, 312, 314, 316, and 318, among other operations.

[0077] FIG. 9 illustrates an operating environment, such as a computer system, that facilitates the performance of the one or more systems and methods described herein. More specifically, the systems and methods described herein can be implemented using a computing device 902. For example, the computing device 902 can be a personal computer, a desktop, a laptop, a tablet, a hand-held computer, a server, a workstation, a mainframe, a wearable computer, a supercomputer, or a combination thereof. However, it is understood that the aforementioned examples of the computing device 902 is non-exhaustive and the computing device 902 can be any type of processing or computing device. Additionally, in some embodiments, the computing device 902 can be or be included in the electronic display 104. The computing device 902 generally includes a processor 904, a display adapter 908, one or more input / output port(s) 910, one or more input / output component(s) 912, a network adapter 914, a power supply 916, and a memory 906. However, it is understood that the computing device 902 can include any additional components therein and is not required to include any of the listed components (e.g., the processor904, the display adapter 908, the one or more input / output port(s) 910, the one or more input / output component(s) 912, the network adapter 914, the power supply 916, and the memory 906).

[0078] The processor 904 is configured to provide instructions to the computing device 902 so that the computing device 902 can process one or more tasks including the implementation of a software program to perform one or more operations as described in more detail herein. It is also understood that the computing device 902 may include any number or processors 904 therein. The display adapter 908 can be a graphics card or a video board that provides the computing device 902 with a capability to display content on a display device 930. For example, the display device 930 can be any screen, monitor, and / or light-emitting component associated with any of the personal computer, the desktop, the laptop, the tablet, the hand-held computer, the server, the workstation, the mainframe, the wearable computer, the supercomputer, or a combination thereof. However, it is understood that the aforementioned examples of the display device 930 is non-exhaustive and that the display device 930 can be any type of device capable of providing a visual display. In various implementations, the display device can be or be included in the electronic display 104.

[0079] The input / output port(s) 910 provide a number of interfaces (e.g., sockets) for one or more cables to connect to the computing device 902. It is understood that there may be any number of input / output port(s) 910 on the computing device 902. For example, the input / output port(s) 910 provides a means for the computing device 902 to receive signals and / or data from an external device connected to the computing device 902 via the one or more cables. As another example, the input / output port(s) 910 provide a means for the computing device 902 to send signals and / or data to an external device connected to the computing device 902 via the one or more cables. The input / output component(s) 912 can include one or more components that support the input / output port(s) 910 such as, but not limited to, a switch, a computer mouse, a computer keyboard, a microphone, a push button, a pressure mat, a float switch, a keypad, a radio receive, or a combination thereof.

[0080] The network adapter 914 can be any type of network interface controller that is configured to provide a means for communicating over a network 926 with another computing device, such as a remote computing device 932. For example, the remote computing device 932 can be a user device such as a cellular-phone, asmartphone, a tablet, a laptop, or a combination thereof. The network 926 can include a system that enables the exchange of data and resource sharing between multiple computing devices using interconnected transmission channels and communication protocols.

[0081] The power supply 916 is configured to convert alternating high voltage current (e.g., AC) into direct current (e.g., DC) to provide power to the other components (e.g., the processor 904, the display adapter 908, the one or more input / output port(s) 910, the one or more input / output component(s) 912, the network adapter 914, and the memory 906) of the computing device 902.

[0082] Additionally, the memory 906 can be a mass storage device and / or a system memory such as a hard disk drive, a memory card, a solid-state drive, random access memory (RAM), or a combination thereof. The memory 906 is configured to provide storage for instructions and data associated with the operation of the computing device 902. The memory 906 can generally include an operating system 918, image and voice recognition software 920, and procedure video data 922. For example, the operating system 918 is configured to manage and / or process any of the data and / or instructions associated with the image and voice recognition software 920 and / or procedure video data 922, as described in more detail herein, such as to manage communication-related traffic within the marshaling environment.

[0083] The image and voice recognition software 920 can include computer-executable instructions which perform or cause to be performed the portions of the herein described methods associated with recognizing actual surgical instruments in images and performing voice commands. The procedure video data 922 can include recorded surgical videos accessible for display on the display screen 300.

[0084] Furthermore, a system bus 924 is also included within the computing device 902 that is configured to couple each of the various components (e.g., the processor 904, the display adapter 908, the one or more input / output port(s) 910, the one or more input / output component(s) 912, the network adapter 914, the power supply 916, and the memory 906) of the computing device 902. It is also understood that each of the components of the computing device 902, and the functionality associated with each of the components of the computing device 902, may be implemented within the remote computing device 932. While the operating environment illustrated within FIG. 9 depicts a particular configuration associated withat least the computing device 902, the network 926, and the remote computing device 932, it is understood that the operating environment may be configured in any way.

[0085] Additionally, a database 928 can be in communication with the computing device 902 and can include stored and accessible surgically-related characteristics and surgical instrument layouts corresponding to a plurality of surgically-related characteristics as described in more detail herein. In some embodiments, the database 928, or the contents thereof, can be included in the memory 906. In some embodiments, the procedure video data 922 can be included in an externally located database 928.

[0086] Unless otherwise expressly indicated herein, all numerical values indicating mechanical / thermal properties, compositional percentages, dimensions and / or tolerances, or other characteristics are to be understood as modified by the word “about” or "approximately" in describing the scope of the present disclosure. This modification is desired for various reasons including industrial practice, material, manufacturing, and assembly tolerances, and testing capability.

[0087] As used herein, the phrase at least one of A, B, and C should be construed to mean a logical (A OR B OR C), using a non-exclusive logical OR, and should not be construed to mean “at least one of A, at least one of B, and at least one of C.”

[0088] In this application, the term “controller” and / or “module” may refer to, be part of, or include: an Application Specific Integrated Circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor circuit (shared, dedicated, or group) that executes code (e.g., processor-executable instructions); a memory circuit (shared, dedicated, or group) that stores code executed by the processor circuit; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, such as in a system-on-chip.

[0089] The term memory is a subset of the term computer-readable medium. The term computer-readable medium, as used herein, does not encompass transitory electrical or electromagnetic signals propagating through a medium (such as on a carrier wave); the term computer-readable medium may therefore be considered tangible and non-transitory. Non-limiting examples of a non-transitory, tangible computer-readable medium are nonvolatile memory circuits (such as a flashmemory circuit, an erasable programmable read-only memory circuit, or a mask readonly circuit), volatile memory circuits (such as a static random access memory circuit or a dynamic random access memory circuit), magnetic storage media (such as an analog or digital magnetic tape or a hard disk drive), and optical storage media (such as a CD, a DVD, or a Blu-ray Disc).

[0090] The apparatuses and methods described in this application may be partially or fully implemented by a special purpose computer created by configuring a general-purpose computer to execute one or more particular functions embodied in computer programs. The functional blocks, flowchart components, and other elements described above serve as software specifications, which can be translated into the computer programs by the routine work of a skilled technician or programmer.

[0091] The description of the disclosure is merely exemplary in nature and, thus, variations that do not depart from the substance of the disclosure are intended to be within the scope of the disclosure. Such variations are not to be regarded as a departure from the spirit and scope of the disclosure.

Claims

CLAIMSWhat is claimed is:1 . A method comprising: identifying one or more surgically-related characteristics; correlating the one or more surgically-related characteristics with a surgical instrument layout stored in a database and based on a surgical procedure to be performed, wherein the surgical instrument layout comprises: a type and quantity of one or more surgical instruments, one or more physical dimensions of the one or more surgical instruments, and orientations of the one or more surgical instruments; displaying on a display screen the surgical instrument layout; and projecting the surgical instrument layout onto a surgical table.

2. The method of claim 1 , wherein the one or more surgically-related characteristics include a name associated with one or more surgical instruments, one or more preference cards associated with one or more surgeons, an instrument profile, a surgical specialty, a surgeon’s name, a surgical procedure, or a combination thereof.

3. The method of claim 1 , wherein the database comprises a plurality of surgical instrument layouts corresponding to the one or more surgically-related characteristics.

4. The method of claim 1 , wherein the display screen is a touch screen interactive with a user.

5. The method of claim 1 , additionally comprising: determining a count of one or more actual surgical instruments prior to commencement of a surgery to identify one or more surgical instruments positioned on the surgical table before a corresponding surgical procedure begins; anddetermining a count of the one or more actual surgical instruments after completion of the surgery to identify one or more surgical instruments positioned on the surgical table before a corresponding surgical procedure begins.

6. The method of claim 5, wherein determining the count of the one or more actual surgical instruments comprises: identifying one or more surgical instruments based on one or more surgically-related characteristics.

7. The method of claim 5, further comprising: generating an alarm based on the determined count of the one or more actual surgical instruments prior to the commencement of the surgery not matching the count of the one or more actual surgical instruments after completion of the surgery.

8. The method of claim 4, further comprising: determining an error based on a degree to which the projected surgical instrument layout matches one or more actual surgical instruments.

9. The method of claim 1 , further comprising: receiving a magnification of a surface from a camera associated with a multi-functional component; and displaying, on the display screen, a magnified surface.

10. An apparatus comprising: a first pivot arm coupled to an electronic display, wherein the first pivot arm is configured to variably position the electronic display about a horizontal axis, wherein the electronic display includes a microprocessor, and wherein the microprocessor is in communication with a database that stores at least one or more surgically-related characteristics; anda second pivot arm coupled to the electronic display and a multifunctional component that includes at least one light source, at least one camera, at least one microphone, at least one projector, or a combination thereof, wherein: the second pivot arm is configured to variably position the multi-functional component about the horizontal axis, the at least one light is configured to illuminate a surface, and the at least one camera is configured to generate an image of the surface and provide the image to the electronic display.

11. The apparatus of claim 10, wherein the one or more surgically-related characteristics include a name associated with one or more surgical instruments, one or more preference cards associated with one or more surgeons, an instrument profile, a surgical specialty, a surgeon’s name, a surgical procedure, or a combination thereof.

12. The apparatus of claim 10, wherein the electronic display is configured to: obtain the image of the surface; display the image of the surface on a display screen, and display a magnified surface on the display screen.

13. The apparatus of claim 12, wherein the electronic display is additionally configured to: obtain a user input corresponding to a selection of a magnification level; and display a magnified surface on the display screen at the magnification level.

14. The apparatus of claim 10, wherein the electronic display is additionally configured to: identify one or more surgically-related characteristics; correlate the one or more surgically-related characteristics with a surgical instrument layout stored in the database and based on a surgical procedure to be performed, wherein the surgical instrument layout comprises:a quantity of one or more surgical instruments, physical dimensions of the one or more surgical instruments, and orientations of the one or more surgical instruments; and determine a count of one or more actual surgical instruments prior to commencement of a surgery, and determine a count of one or more actual surgical instruments after completion of the surgery.

15. The apparatus of claim 14, wherein the electronic display is further configured to generate an alarm based on the count of the one or more actual surgical instruments not matching the surgical instrument layout16. The apparatus of claim 14, wherein the electronic display is further configured to determine an error based on the count of the one or more actual surgical instruments not matching the surgical instrument layout17. The apparatus of claim 10, wherein the at least one microphone is configured to receive one or more voice commands associated with an operation of a display screen, and wherein the electronic display is additionally configured to receive the voice commands from the at least one microphone and perform a corresponding task.

18. The apparatus of 17, wherein the display screen is a touch screen interactive with a user.

19. The apparatus of claim 11 , wherein the display screen is further configured to: display the one or more preference cards; receive a selection of a surgical procedure; and display one or more surgical procedure videos.

Images