Remote operating room broadcasting system for medical device representatives

Abstract

Providing an electronic operating room collaboration session includes preconfiguring the session, starting the session and connecting the devices as participant devices in the session, where the devices include Smart Glasses devices, equipment computer system(s) that present equipment video feed(s), and a remote participant computer system, receiving the equipment video feed(s) and presenting them as participant(s) in the session, receiving a video stream from a corresponding Smart Glasses device and presenting the video stream to the remote participant computer system as part of the session, and remotely controlling participation of the Smart Glasses devices in the session, the remotely controlling including issuing a control directive to a Smart Glasses device to change an audio / video (A / V) function of the Smart Glasses device, where the change of the A / V function alters at least one of audio and video associated with the Smart Glasses device.

Description

REMOTE OPERATING ROOM BROADCASTING SYSTEM FOR MEDICAL DEVICE REPRESENTATIVES BACKGROUND

[0001] Orthopedics is a highly coveted specialty amongst medical doctors. Their field covers spine, joints, sports, trauma, and extremities. During this work they use aerospace grade implants to correct deformities, replace joints and tendons and piece together bones that have been fractured during accidents.

[0002] With the increase in innovation of devices, costs have also increased, and therefore, stakeholders have a deep seeded interest in ensuring that each procedure is done as correctly and efficiently as possible. Medical device representatives support the surgeon in selecting the correct implants and ensuring that they are installed correctly the first time as to not create waste or added time to the surgery. Currently this is done with the medical device representative physically in the operating room. SUMMARY

[0003] Shortcomings of the prior art are overcome and additional advantages are provided through a computer-implemented method. The method includes preconfiguring an electronic operating room collaboration session based on session preconfiguration data received from a scheduling component, the preconfiguring setting roles for participation of a plurality of devices in the electronic operating room collaboration session; starting the electronic operating room collaboration session and connecting the plurality of devices as participant devices in the electronic operating room collaboration session, the plurality of devices including: an operating room computer system; a plurality of smart glasses devices, wherein the plurality of smart glasses devices are connected as participants in the electronic operating room collaboration session based on authentication of the plurality of smart glasses devices to join the electronic operating room collaboration session as dictated by the preconfiguring; at least one equipment computer system presenting at least one equipment video feed graphically depicting information of a corresponding at least one operating room equipment; and a remote participant computer system connected as a participant in the electronic operating room collaboration session; receiving, from the at least one equipment computer system, the at least one equipment video feed and presenting the at least one equipment video feed as one or more additional participants in the electronic operating room 6516.001AWO 1collaboration session; receiving at least one video stream from a corresponding at least one smart glasses device of the plurality of smart glasses devices, and presenting the at least one video stream to the remote participant computer system as part of the electronic operating room collaboration session; and remotely controlling participation of the plurality of smart glasses devices in the electronic operating room collaboration session, the remotely controlling including issuing a control directive to a smart glasses device of the plurality of smart glasses devices to change an audio / video (A / V) function of the smart glasses device, wherein the change of the A / V function alters at least one of audio and video associated with the smart glasses device.

[0004] Additionally, or alternatively, in one or more embodiments, the remotely controlling is based on a control request to change the A / V function. In one or more embodiments the remotely controlling provides remote control to the remote participant computer system by accepting control requests from the remote participant computer system, and the method further includes receiving from the remote participant computer system the control request to change the A / V function based on invocation of the control request by a remote participant user.

[0005] Additionally, or alternatively, in one or more embodiments, the A / V function includes a microphone operating state of a microphone of the smart glasses device, wherein the control request to change the A / V function includes a request to mute the microphone, and wherein the control directive directs the smart glasses device to mute the microphone.

[0006] Additionally, or alternatively, in one or more embodiments, the A / V function includes an audio output volume of the smart glasses device, wherein the control request to change the A / V function includes a request to adjust the audio output volume, and wherein the control directive directs the smart glasses device to adjust the audio output volume.

[0007] Additionally, or alternatively, in one or more embodiments, the A / V function includes a video capture or output state of the smart glasses devices, wherein the control request to change the A / V function includes a request to enable or disable video capture or output from the smart glasses device, and wherein the control directive directs the smart glasses device to enable or disable the video capture or output from the smart glasses device.

[0008] Additionally, or alternatively, in one or more embodiments, the A / V function includes a video display state of the smart glasses device, wherein the control request to 6516.001AWO 2change the A / V function includes a request that selects which video stream of the at least one video stream to present on a display of the smart glasses device, and wherein the control directive directs the smart glasses device to present the selected video stream on the display.

[0009] Additionally, or alternatively, in one or more embodiments, the remotely controlling includes selectively controlling audio output volume levels of participants in the electronic operating room collaboration session.

[0010] Additionally, or alternatively, in one or more embodiments, the remotely controlling includes selectively isolating communication between the remote participant computer system and a smart glasses device by issuing control directives to mute a microphone and audio output of one or more participants, other than the remote participant computer system and the smart glasses device, in the electronic operating room collaboration session.

[0011] Additionally, or alternatively, in one or more embodiments, each smart glasses device of the plurality of smart glasses devices performs agentless participation as a participant device in the electronic operating room collaboration session, in which the smart glasses device is configured with a Wi-Fi configuration that includes identification of a Wi-Fi network for connection to the Internet and a starting uniform resource locator (URL), wherein the smart glasses device is configured to, on startup of the smart glasses device, connect to the Wi-Fi network, launch a browser that automatically navigates to the URL, and retrieve and execute code in the browser for performing functions as part of the participation of the smart glasses device as a participant device in the electronic operating room collaboration session.

[0012] In accordance with one or more aspects, each of the embodiments is separable and optional from one another. Further, embodiments may be combined with one another.

[0013] Computer-implemented methods, computer systems, and computer program products relating to one or more aspects are described and claimed herein. Each of the embodiments of the computer program product may be embodiments of each computer system and / or each computer-implemented method and vice-versa. Further, each of the embodiments is separable and optional from one another. Moreover, embodiments may be combined with one another. Each of the embodiments of the computer program product may 6516.001AWO 3be combinable with aspects and / or embodiments of each computer system and / or computer- implemented method, and vice-versa.

[0014] The present summary is not intended to illustrate each aspect of, every implementation of, and / or every embodiment of the present disclosure. Additional features and advantages are realized through the concepts described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Aspects described herein are particularly pointed out and distinctly claimed as examples in the claims at the conclusion of the specification. The foregoing and other objects, features, and advantages of the disclosure are apparent from the following detailed description taken in conjunction with the accompanying drawings in which:

[0016] FIG.1 provides an example conceptual depiction of automated connection of a smart glasses device to an electronic operating room collaboration session, in accordance with aspects described herein;

[0017] FIG.2 depicts an example conceptual diagram of remote control of a smart glasses device in an electronic operating room collaboration session, in accordance with aspects described herein;

[0018] FIG.3 depicts an example conceptual diagram of hardware agnostic agentless multi-smart glass device participation in close proximity, in accordance with aspects described herein;

[0019] FIG.4 depicts an example interface of a remote participant computer system, for instance a computer system being used by a remote operator, in accordance with aspects described herein;

[0020] FIG.5 depicts an example process for providing an electronic operating room collaboration session, in accordance with aspects described herein; and

[0021] FIG.6 depicts an example computer system to incorporate, use, and / or perform aspects described herein. DETAILED DESCRIPTION 6516.001AWO 4

[0022] Aspects described herein provide and facilitate various processes and efficiencies around surgical procedures. With respect to hospitals and surgical centers, scheduling cases in the operating room demands the attendance and availability of all participants, and in orthopedic cases, medical device representatives play a critical role. Further, focusing on the patient points to removing unnecessary people from the operating room, and at the same time ensuring that critical knowledge is readily available.

[0023] With respect to medical device companies and medical device representatives (“reps”), the companies are motivated to provide their products to as many Cases as legitimately possible, and medical device representatives earn their revenue by having the products they represent implanted or used during surgeries. Restrictions of geography or scheduling conflicts (due to travel and weather) are roadblocks towards the objectives of both.

[0024] With respect to doctors, they demand that their surgeries be scheduled when they need them. They also drive to use the best devices available for their patient needs and need consultants to be available “on demand.” “On Time Starts” is an important aspect of the surgical theatre. The doctor demands that all equipment and facilities are ready to start on time. The medical device representative plays an important role in consulting with the scrub tech to ensure everything is prepared and ready. Aspects described herein contribute to efficient “On Time Starts”.

[0025] With respect to healthcare insurance, one of the cost drivers of medical devices is the cost of medical device representatives attending operating rooms in person. Aspects of the present disclosure provide relief in this regard to the healthcare insurance providers, from the medical device companies, for cases that utilize remote medical device representative systems as described herein.

[0026] Aspects described herein also benefit patients through provision of better access to cutting edge technologies with the technical support of medical device representatives regardless of their geography, or other uncontrollable circumstances, for instance weather delays, traffic delays, sicknesses, or other circumstances that prevent representatives from attending cases in-person. Patients can also benefit from lower chances for infections in the operating room because of reductions in physical traffic and door swings through facilities provided herein. 6516.001AWO 5

[0027] The advent of smart glasses (which are interchangeably referred to herein as ‘smart glasses devices’) and collaborative surgical operations has been brought into the field of medicine. To date, these techniques have served great purposes, however, aspects of this disclosure recognize the high costs associated with having costly doctors on either end of one collaboration surgery.

[0028] Accordingly, aspects offer expert consulting in the operating room, with greater availability of the experts, at a reduced cost and without medical device representatives in the hospital.

[0029] In an example embodiment, systems and processes described herein enable a remote medical device representative to direct certain procedures, offer counseling, and provide explanations to operating room staff during an operating room procedure with better results than if the medical device representative were in-person. This is accomplished in part by leveraging smart glasses and their integration into a session system controller / server in the provision of an electronic operating room collaboration session, which may also be referred to interchangeably herein as a ‘live operating room session’, ‘collaboration session’, operating room session’, surgery session’ or just ‘session’. The system can coordinate and manage these sessions and may be implemented by computer systems, such as computer systems provided in a cloud environment as a non-limiting example; the computer systems could instead be provided in any desired environment.

[0030] Some example, non-limiting features that could be provided in embodiments by aspects described herein are as follows:

[0031] 1) The ability to operate with multiple wearers of smart glasses in the operating room and blend the respective outputs of the smart glasses to the remote device representative and blend displayed session images and prompts on screen to the smart glass wearers.

[0032] 2) The ability to have the remote medical device representative remotely control functional features of the operating room staff’s smart glasses. Example functional features include zoom, audio mute, image parameters, and camera power, such as resolution and / or quality. In some non-limiting examples, the control includes control over zoom, audio volume, image parameters such as white balance, and / or various audio / video functions (including microphone, audio output, camera, display, and / or other aspects) of the devices. In a specific example, the microphone and audio output (e.g., earpiece / speaker) on / off and 6516.001AWO 6volume controls of a smart glasses device may be controlled selectively and independently of each other, as well as separately and independently from those of each of the other involved smart glasses and other devices in the operating room and participating in the session.

[0033] 3) The ability to receive and broadcast multiple simultaneous smart glasses camera feeds to provide the remote medical device representative with a points of view of the surgeon and / or the scrub tech, and screen images from operating room equipment. Image output from equipment such as the fluoroscopes, x-ray, scopes and other devices may also be provided to the remote medical device representative, for instance by converting signal and / or interface types. For instance, in an embodiment, a High-Definition Multimedia Interface (HDMI) and / or other output signal from the operating room equipment is converted to that of another protocol and / or for provision though another interface type, such as Universal Serial Bus (USB). A USB cable providing the converted USB protocol signal can be plugged into another computer system, such as a tablet or other computer that recognizes the input as another video / camera image feed. This live screen feed is then used to join a live operating room session where the remote medical device representative may add that equipment video image feed as another window for monitoring and consulting on remote surgeries.

[0034] 4) Service Provider integration, for instance integration with SurgiCloud™ (offered by Surgicloud LLC of Columbus, Ohio), offers the ability to schedule surgical activities, and capture information such as details, quantities, and part numbers of the implants and equipment used in a surgery session, and provide account billing for them and billing and replacement into the surgical trays.

[0035] In embodiments, an example system of this disclosure enhances the delivery of critical expertise of medical device representatives (MDR) in operating rooms (OR) by enabling a remote medical device representative delivery of service.

[0036] Furthermore, an example system of this disclosure integrates the surgical “boarding” or e- scheduling function, to coordinate all stakeholders, inside of a singular environment, controlled by the doctor and hospital.

[0037] An example disclosed system provides e-documentation of consumed implants during surgery and has a facility to capture / tabulate objective data from surgeries in real time or over history. 6516.001AWO 7

[0038] Examples described herein are presented in the context of orthopedic surgery and entities in the orthopedic surgery community. Example users of systems described herein are hospitals and surgery centers with operating rooms.

[0039] Stakeholders affected and benefitting from aspects described herein include hospitals, orthopedic surgeons, patients, medical device companies, medical device representatives and distributors.

[0040] In embodiments, aspects enable expert consulting to be provided by remote medical device representatives, regardless of geographic locations and logistical conflicts in scheduling.

[0041] Systems described herein offer immediate benefits to orthopedic operating room environments Functional capabilities include remote collaboration services and the use of multiple smart glasses before, during and after surgery. Example systems are designed to serve agnostic to medical device companies. Relevant service and equipment suppliers and device representatives may be given access to the system, albeit access managed through the control of the hospital host.

[0042] Some example capabilities remove the medical device representative from the operating theatre, reducing “door swings” and unwarranted solicitations, while improving the representative’s communication with the surgeon and technicians, for instance a scrub tech, via 1-to-1 audio and improving the medical device representative’s field of view as described herein. This is advantageous in comparison to the typical situation of the medical device representatives being restricted to a back table with their vantage point to the surgical site and corresponding electronic monitoring equipment often being significantly obstructed.

[0043] Remote capabilities as described herein provide improvements in On Time Starts (OTS) and Turn Over Times (TOT), while providing opportunities for hospitals to perform surgeries requiring medical device representatives that otherwise could not be available due to geography and / or time constraints, for instance, and also opportunities to offer additional surgeries per day, per operating room, based on improved scheduling confidence.

[0044] Medical device companies / representatives can access a system of this disclosure when invited into the session, which in some examples is only by a proper hospital host of the session. 6516.001AWO 8

[0045] In addition, some example capabilities of a system of this disclosure can expand and include interoperability with hospital and doctor scheduling systems, inventory management of medical devices from each device company and tracking the devices that have been used during the operation, Enterprise Resource Planning (ERP) systems for billing, training for medical device representatives, scrub techs, and doctors, and repositories of operating room video recordings, controlled by the hospital.

[0046] As a specific example, e-documentation can record the implants consumed during surgeries and provide a repository of implant serial numbers consumed during each surgical case matched to each unique case number.

[0047] Additional capabilities of a system of this disclosure includes gathering of data. As permitted by the hospitals, surgery information can be gathered to accurately assess important information about operating room efficiencies, time requirements, equipment usage, scrub tech efficiencies and more. DETAILED DESCRIPTION

[0048] Initially, the following presents aspects of example, and non-limiting, systems and process provided herein.

[0049] DOCTOR’S OFFICE (DO)

[0050] Provider Interface with patient - Provider fills out handwritten or electronically generated surgical procedure(s) for individual patient care (Surgical Order (SO)). The provider (surgeon) dictates the surgical procedure(s) and their preferences regarding the equipment, devices, medical device reps and support staff required. The provider determines which, if any, medical device representatives should be in attendance in person or attend remotely through the system of this disclosure.

[0051] Physician’s Office Schedulers (POS) - Physician’s Office Scheduler (POS) requests operating room time at surgical facility (hospital or ambulatory surgical center (ASC)). The scheduler often uses surgeon’s block operating time to schedule procedure. POS coordinates / specifies which facility, equipment, implants and devices, medical staff support, and estimated surgical procedure duration based on surgeon’s preferences / availability and often uses block time already established. POS notifies Medical Device Rep (MDR) of 6516.001AWO 9pending surgical order. POS notifies the medical device representative if they must be physically present at the surgery, or whether medical device representative has the option or has been requested to provide remote assistance, using a system of this disclosure.

[0052] SURGICAL FACILITY (HOSPITAL OR AMBULATORY SURGICAL CENTER) ROLES

[0053] Surgeon - Surgeon arrives to the operating room with the expectation that the scheduled procedure shall run smoothly. This means the operating room is equipped and staffed to proceed in a timely efficient fashion. In the event that the surgical order calls for the use of a Remote Medical Device Rep (RMDR), the surgeon has the option to wear one pair of the smart glasses (preferred) or wireless Bluetooth earbuds.

[0054] Surgical Facility Surgery Scheduler (SFSS) - Facility receives requests for surgery arriving from various physician’s offices and the SFSS organizes / prioritizes them. The SFSS creates scheduling time slots for each procedure according to the availability and staffing for individual operating rooms called boarding. SFSS shall schedule surgeries using RMDR into designated operating rooms equipped with dedicated Wi-Fi connections and medical monitoring equipment compatible with Monitoring Systems (MS). SFSS makes plans according to the available surgeon’s schedule / block time and surgical facility availability.

[0055] Charge Nurse (CN) - Charge Nurse (CN) oversees the surgical operations and plans for scheduled procedures in their operating room. CN or designee calls for required equipment, as specified in the surgical order. CN or designee confirms with surgeon. CN or designee notifies medical device representatives to confirm. CN or designee ensures that the Implant / Device Trays (DT) are at the hospital, that they have gone through sterilization, as required, and assembled on the Surgical Carts (SC) as required for the day of surgery. CN or designee ensures operating room staff are scheduled to work in the operating room, as required. CN or designee will create a session using scheduling software, and invite required parties, to secure session (to occur in the future, according to the hospital boarding). Using a website, as a client portal participants may be registered and verified as HIPPA compliant participants, who have permission to participate in a live operating session. The same website may be used to launch a surgical collaboration session with the approved participants and may also determine paid up subscriptions for session participants. 6516.001AWO 10

[0056] When a Remote Medical Device Representative is scheduled, the CN or designee is to ensure that Smart glasses (equipment) are available / charged, the operating room is Wi-Fi enabled and working, the medical device representative has been notified that they will work remotely, and that medical device representative has system privileges in good standing, the designated surgeon and medical scrub tech have been oriented and competent in the use of the relevant equipment of a system of this disclosure, and accessory monitoring devices (such as the fluoroscope, arthroscopic monitor, anesthesia monitors) pertinent to the procedure and the medical device representative have been equipped with monitoring systems.

[0057] Circulating Nurse (CirN) - The circulating nurse is hands on in the operating room. This person is available to move things, retrieve things, help with things, provide coverage for breaks. The circulating nurse is not scrubbed in sterile during surgery.

[0058] The CirN can use a browser to access a website of a system of this disclosure, locate the specific surgical case that was initiated by the CN, and login / launch the scheduled session, as host.

[0059] The CirN can use the browser and login to the website of a system of this disclosure as the surgeon (logging out as CirN and logging in as surgeon).

[0060] The CirN accesses the Scheduled Session (now as the surgeon), and chose, “wearables” (meaning, join with smart glasses). A QR code appears on the screen. The CirN will plug the surgeon’s smart glasses into its battery pack. When the glasses have “powered up”, point the camera of the surgeon’s smart glasses at the QR code on the screen, and the smart glasses will join that specific surgical session. Click “join session”, and again confirm “join session”, by touching a rear button on the top of the smart glasses, where there are, in examples, three buttons in a row.

[0061] The CirN can use the browser and again login to the website of a system of this disclosure as the scrub tech (log out as surgeon and log in as the scrub tech). The CirN will access the Scheduled Session (now as the scrub tech), and chose, “wearables” (meaning, join with smart glasses). A QR code will appear on the screen. The CirN will plug the scrub tech smart glasses into its battery pack, and when the glasses have “powered up”, point the camera of the scrub tech’s smart glasses at the QR code on the screen, and the smart glasses will join that specific surgical session. Click “join session”, and again confirm “join session”, by 6516.001AWO 11touching a rear button on the top of the smart glasses, where there are, in examples, three buttons in a row.

[0062] The smart glasses, after joining the session, can then be transmitting audio and / or video of the session as described herein.

[0063] In examples, video images from the joined smart glasses shall appear on a host session collaboration screen of the CirN.

[0064] The audio within a session can begin to “screech” if the glasses are close to each other. The CirN can adjust the audio controls of the various joined glasses to reduce the speaker volumes and / or the microphone sensitivity, as examples. The CirN can use this opportunity to verify that the (audio and video zoom) controls are functioning acceptably.

[0065] The smart glasses may also include a flashlight, laser pointer, or focused spotlight / light beam, for instance, to assist the wearer (e.g., surgeon / scrub tech) in knowing exactly where the camera of the wearer’s smart glasses is pointing when broadcasting a video feed from the smart glasses.

[0066] The CirN can check that the Remote Medical Device Rep (MDR) has also accessed the surgical session and verify that the MDR’s audio is functioning.

[0067] At this point, the CirN might chose to be the controller of the volume and zoom (as examples) for this session. The surgeon, scrub tech, and remote medical device representative can provide feedback on the volume of audio they hear from the session and the CirN can adjust these controls, as communicated by these three people. It is possible that these volume levels could be set once and not adjusted again for the duration of the surgery, or, alternatively, adjustment might be requested from time to time.

[0068] As another example option, some or all the smart glass controls can appear on the co-host screen. The CirN and / or the remote medical device representative could control all (or just some of) volume and zoom features, from the host and co-cost screens, and optionally without logging in as the surgeon or scrub tech, separately. In examples, a host and co-host (MDR) could control audio on the microphone and earpieces / speakers on the smart glasses devices independently. 6516.001AWO 12

[0069] The CirN can collaborate with the medical device representative to document the implants and medical devices consumed during surgery, and information about such implants / devices. The system of this disclosure can provide this information in the form of a digital record of items consumed, their serial numbers, part numbers, and cost, and can associate digital records to each specific surgical case.

[0070] The CirN may have an option to wear wireless (for instance Bluetooth-based) earbuds to access audio of the surgical case and allow one way or two-way communications.

[0071] Case Dependent: In some examples, the CirN can connect into surgical session accessory monitoring devices (such as the fluoroscope, arthroscopic monitor, anesthesia monitors) pertinent to the procedure with a tablet or other devices, for instance a Bluetooth- enabled device.

[0072] Surgical Scrub Technicians - Surgical Scrub Technicians (SST) are operating room staff members that handle the device trays, and “open them” and prepare the implants, tools and fixtures to be ready for surgery. Device trays are complicated, so the surgical scrub technician is responsible for extracting only the required items from the trays and having them ready to be handed to the surgeon, on demand. The SST stands within arm’s reach of the patient, the surgeon, and the First Assist (FA), to hand items to the surgeon, and to retrieve and move items out-of-the-way, during surgery.

[0073] The SST puts the medical device trays back together after surgery. The SST works with the CirN to account for the items consumed during surgery, such as implants installed, sponges used & removed, etc. The SST works closely with the Medical Device Rep (MDR), who will provide active consultation on what items are required for this specific surgery, and what items shall be demanded by the surgeon in the next few minutes. The SST is scrubbed in sterile to the surgery. The SST wears smart glasses with functionality described herein during cases employing remote medical device representatives. Two-way communication between the medical device representative and the SST enables a smooth-running surgery. The SST and the remote medical device representative can have “private” communications about medical devices. Whereas communication with the surgeon is required from time to time, communication strictly between the medical device representative and SST occurs much more frequently, and the surgeon does not want to be distracted with their dialogue. 6516.001AWO 13

[0074] Medical Device Company (MDC) - The medical device company provides implants, tools and fixtures designed for the medical field, and uses distributors and representatives to promote, sell, explain and support their products to medical professionals.

[0075] Medical Device Rep - The Medical Device Rep (MDR) is the subject matter expert trained in the usage of the many items contained in the medical device manufacturer’s device trays. The medical device representative may be trained in medical procedures. Whereas the medical device representative is often not a doctor, they become very familiar with how devices are used during surgery and provide critical consultation to the operating room staff, such as the Medical Scrub Tech (MST) and often the surgeon as well.

[0076] The Remote Medical Device Rep (RMDR) provides consultation through collaboration methods described herein of a system of this disclosure. Surgeons and Surgical Scrub Techs (SST) can wear smart glasses during surgery and described herein, and the remote medical device representative can monitor surgical cases and communicate with the operating room staff live, and in real time.

[0077] The remote medical device representative is authenticated into the operating room session and may be given limited control of one or more of the Session features. Example such features could include one or more of the following:

[0078] Control of the sound volume in the individual “earpiece”, for instance device speaker or analogous audio output device such as a Bluetooth connection to earbuds or other headset, for each smart glass wearer (for example, can reduce the distracting sound heard in the surgeon’s smart glasses, but increase sensitivity level for the scrub tech).

[0079] Control of the microphone sensitivity for each smart glass wearer (for example, the SST can whisper their questions and still be heard loud and clear, whereas the surgeon’s voice is heard, but does not override all other communications.

[0080] Control of the zoom level of the smart glasses video feed, to enable a focused look at object details. (for example, zooming in to see the details of a small screw in the device tray, but zoom out on the surgeon’s glasses, to view the entire surgical field).

[0081] Control of the white-balance level of the video from the smart glasses, to adjust for whitewash blinding and glare, from light sources. 6516.001AWO 14

[0082] The following provides example devices and methods around a day of surgery, in accordance with aspects described herein.

[0083] Operating Room Equipment and Room Readiness

[0084] Wi-Fi connectivity - Steady and reliable Wi-Fi connectivity may be important to maximize functionality described herein. A dedicated Wi-Fi network may be used to achieve this, in some examples using a Wi-Fi booster / extender to ensure that involved devices local to the surgical environment have strong connection to the Wi-Fi network.

[0085] Smart Glasses: Aspects described herein leverage smart glass solutions, for instance those provided by VUZIX, such as the model M400 smart glasses. In examples, there are multiple (i.e., two or more) sets of smart glasses for each operating room. In examples, the surgeon wears one set and the surgical scrub teach wears another set.

[0086] Smart glasses device configuration: In embodiments, an example system of this disclosure employs a live connection with the smart glasses.

[0087] Software – Though not required, in some embodiments the smart glasses could have proprietary software loaded that automatically launches software when powered up, and seeks to read in a QR code, specifically coded for the surgical case that the circulating nurse has invited doctor and surgical scrub tech to join. The smart glasses operating software and registered devices are managed by the system of this disclosure, for clients.

[0088] Batteries - Similar to orthopedic surgical helmets, smart glasses are often powered by rechargeable batteries. The batteries need to be charged prior to the day of surgery. A fully charged battery pack will provide sufficient battery charge for an eight (8) hour day of surgery. Battery packs are to be clipped on to pant scrub or kept in back scrub pocket. The smart glasses and battery pack are connected between a three (3) foot or four (4) foot power cord (depending on the height of the wearer).

[0089] Monitoring Systems (MS): operating room monitored equipment. The remote medical device representative can monitor critical operating room equipment to follow along with the surgery and provide appropriate consultation. For instance, a fluoroscope, arthroscopic monitor, ultrasound, and an anesthesia tower are example operating room devices that, when used during a surgical session, could be monitored and included in a 6516.001AWO 15collaboration session of a system of this disclosure. Video feed is extracted from these machines using the proprietary Monitoring System (MS). In some examples, a connection from such operating room devices duplicates the image(s) from computer monitors of these machines and presents these machines as “participants” in the sessions, depicting the images therefrom.

[0090] Chargers - A charging unit, such as a multi-port 5-volt charging unit, is provided that provides sufficient power to simultaneously charge the batteries involved for an operating room.

[0091] Case - A case can be provided that houses the equipment for equipment of an example system of this disclosure. The case may be equipped with an LED, ultraviolet (UV) strip to help sterilize the equipment while inside the case. The case may be designed to transport the equipment, and be present in the operating room, or to be left outside the operating room, during surgery. The case may be designed to be easy to transport, and robust enough to protect the electronic equipment inside, while offering a combination lock for security. The case can have a standard plug input (for instance a two-prong plug input) to the outside of the case to receive a power input, for instance a110 volt input. The case can provide a compartment or other area to store “how to” instructions.

[0092] By way of a specific example, physical devices / items to form an example case as described can include:

[0093] 1 Carrying Case;

[0094] 2 sets of Vuzix™ M400 Smart Glasses offered by Vuzix Corporation of Rochester, New York; or 2 sets of Realwear Nav500, offered by Realwear Corp.

[0095] 4 sets of 10k milliampere-hour (mAh) battery packs;

[0096] 2 sets of three (3) foot power cords;

[0097] 2 sets of four (4) foot power cords;

[0098] 2 sets of headbands (to mount the smart glasses);

[0099] 1 tablet device; 6516.001AWO 16

[0100] 1 Monitoring System (MS);

[0101] HDMI and USB-C connection cords;

[0102] 1 UV sterilizing LED Strip;

[0103] 1 Multiport, 5 Volt charging station; and

[0104] 2 sets of Headband Name Plates.

[0105] In embodiments, a client can designate one or more team members to refresh the authentication mechanism on each pair of smart glasses devices and video feed tablets / computers periodically or periodically. In examples, this is to be refreshed once every 30 days or other time-based interval. In examples, this could be done by powering on each device, logging out of the software, and logging back into the software.

[0106] In embodiments, a client can designate a team member that is granted the scheduler role in the software. The scheduler can open the software in advance of the medical procedure and schedule a planned session. A planned session can include, in examples, a planned start date / time and planned end date / time. A planned session can include, in example, invitee definitions for themselves, a smart glasses device for a surgeon, a smart glasses device for the scrub tech, and each of one or more video feed tablets or computers for medical devices, such as scopes.

[0107] Example roles and responsibilities of personnel on day of surgery

[0108] Circulating Nurse (CirN) - CirN opens the collaboration session on an operating room computer. CirN powers on the smart glasses for the surgeon and verifies they automatically connect to the session. CirN utilizes the on screen automatically generated QR codes if the glasses fail to automatically connect. CirN powers on the smart glasses for the surgical scrub tech (SST) for that case and verifies they automatically connect to the session. CirN utilizes the on screen automatically generated QR codes if the glasses fail to automatically connect. Whereas smart glasses are worn by the surgeon and SST, the information displayed on operating room equipment, will be displayed in the collaboration session, using the monitoring system (MS), which enables the remote medical device representative to monitor the video feeds and the smart glass camera feeds. The system picks 6516.001AWO 17up the digital display signal(s) from the various operating room medical equipment, connected to the tablet(s), using the MS.

[0109] CirN powers on all invitee video feed tablets connected to scopes such as a fluoroscope or arthroscopy monitor, as examples and ensures they automatically connect to the session or manually authenticates if needed. The settings of the tablet(s) are configured to pass the medical room equipment camera video feed to the collaboration session. Software restricts video feed profiles to only allow sending of video. Video feed profiles may not receive video or audio and may not otherwise participate in the session.

[0110] The CirN will power on the smart glasses for the SST and the surgeon. The CirN verifies that they are connected (and transmitting audio and / or video) into the session. Should the smart glasses fail to automatically connect, the software provides on screen QR codes as an alternative method to connect to the session, utilizing the smart glass camera, which translates the QR code into a URL, unique to each device profile.

[0111] In some embodiments, the CirN uses browser windows of the smart glasses to control the audio on the glasses being output to the wearer. For instance, slider bars may be available on these browser windows to control audio and possibly camera controls (e.g., zoom) for individual smart glasses. Audio control can be important, as smart glasses in close enough proximity to each other can generate loud feedback loops. The CirN or other entity (such as a remote medical device representative) can turn the sensitivity down by lowering the output volume of one or more of the joined smart glasses. The smart glasses are ready to be worn at that point and can be placed appropriately on a table in the operating room waiting to be worn by the operating room staff.

[0112] Surgical Scrub Tech (SST) - On the day of surgery, the SST will wear the corresponding set of smart glasses. The smart glasses are waiting for the SST in the operating room as prepared by the CirN. The SST has preparation duties such as setting up a back table in the sterile environment prior to the surgical case beginning and therefore the smart glasses may or may not be worn during this set up time. In any case, once the surgical procedure is ready to begin, the SST puts on the smart glasses. If desired, the corresponding battery pack for the set of smart glasses is placed into the SST’s back pant pocket or clipped onto surgical pant, as examples. The SST can adjust the positioning of the camera on the smart glasses to point towards the SST’s hands while working in the operating room. In examples, this 6516.001AWO 18adjustment is done by performing the “thumb check” (a visual alignment of the camera field of view, by looking at the wearer’s thumbs as they hold them at a desired height, for instance tabletop height.). Assuming the remote medical device representative is already joined to the collaboration session, the SST can communicate with the remote medical device representative to perform a sound check. The remote medical device representative can verify the audio and video. The remote medical device representative can also adjust the sound levels, as appropriate, using controls of the system of this disclosure, as is explained further herein.

[0113] Surgeon - On the day of surgery, the surgeon will wear the corresponding set of smart glasses. The smart glasses are waiting for the surgeon in the operating room as prepared by the CirN. The surgeon or CirN will place the battery pack into the surgeon’s back pant pocket or clipped onto the surgical pant, as example. The surgeon can adjust the positioning of the camera on the smart glasses to point towards the surgeon’s hands while working in the operating room. In examples, this adjustment is done by performing the thumb check. The surgeon can communicate with the remote medical device representative to perform a sound check. The remote medical device representative can verify the audio and video. The remote medical device representative can also adjust the sound levels, as appropriate, using controls of a system of this disclosure, as is explained further herein.

[0114] Remote Medical Device Representative (MDR) - During the surgery, the MDR will be “remote” from the surgical theater, i.e., the operating room. In examples, the medical device representative can verify ahead of time that the required device trays are present at the surgical facility and ready for use in the operating room. Activity of the Medical Device Representative during the collaboration session can include, in addition to observing the surgical procedure and contributing via dialog with the surgical team in the collaboration session, invoking control of various audio / video properties of participants in the session, for instance audio output volume levels, microphone levels, and other properties.

[0115] To this point, the Charge Nurse (CN) has scheduled the sessions for the day of surgeries, the Circulating Nurse (CirN) has launched the specific session for a specific case, the CirN has connected the smart glasses for the surgeon and SST to that specific case, the CirN has connected the tablet that monitors the fluoroscope (or other relevant OR equipment) to that specific case and the CirN has invited the remote medical device representative to that specific case. The MDR can open the system session by way of an invitation that is provided 6516.001AWO 19to the MDR, and position the session on a display, for instance on a large screen. A large screen can enable multiple video feeds (such as those from the involved smart glasses and the medial equipment fees) can all be clearly seen. The remote medical device representative may be able to, in another browser or otherwise, log in to the session as the surgeon. Here, the MDR can choose “join session with glasses”, which can display volume and zoom controls specifically for the surgeon’s glasses. The remote medical device representative can also log in to the session similarly as the SST, choose “join session with glasses”, which can display volume and zoom controls specifically for the SST’s glasses. The remote medical device representative can also perform a sound check to ensure that the operating room staff can hear the medical device representative’s voice. The volume controls can now be adjusted by the medical device representative, as desired by the operating room staff.

[0116] In specific examples, the smart glasses speaker that plays session audio into the surgeon’s ear can be turned off completely, or reduced in volume by the MDT. The smart glasses microphone picking up the surgeon’s voice can be muted by the MDT as to avoid feedback or echo from multiple microphones and speakers in close proximity and to provide cleaner audio to the SST.

[0117] In specific examples, the smart glasses speaker that plays session audio into the SSTs ear can be increased in volume by the MDT. The smart glasses microphone picking up the SST’s voice can be enabled by the MDT to allow isolated communication.

[0118] During the session, the medical device representative can counsel the SST, for instance counsel about what trays to open, how to open them, and how to present the devices for this surgery. The trays can be very complex; the medical device representative can counsel the SST on what items are needed and when.

[0119] In embodiments, a participant such as the medical device representative or the CirN can record the system session for documentation.

[0120] During surgery, the remote medical device representative may be actively speaking to the SST and / or other participants, monitoring the surgeon’s progress, reviewing the results shown on the OR equipment such as the fluoroscope, and actively interacting with the SST. In examples, the medical device representative advises what to pull out from the trays, and when and where to present these devices for the surgeon. The remote medical device representative can anticipate the next steps of a surgery and give the SST advance 6516.001AWO 20notice on what to prepare, and therefore be immediately ready when the surgeon calls for a specific item. The remote medical device representative can also assist the SST and CirN with how to repackage the medical device trays and how to closeup the devices after surgery.

[0121] A system of this disclosure can have a digital record component to record the items consumed during surgery. The records of these items may be documented by the remote medical device representative and reconciled with the CirN. The remote medical device representative can generate a document with the inventory of items consumed during surgery and this document can be transmitted to the operating room / hospital via the collaboration session.

[0122] The remote medical device representative assists the SST and CirN with how to repackage the medical device trays and how to closeup the devices after surgery.

[0123] Accordingly, one aspect described herein is a system that includes components to be utilized in a sterile space such as medical operating room. In this context, and in an effort to maintain sterility, smart glasses and related devices can remain hands-free when in operative use during a surgical procedure. To enable multiple audio devices in close proximity (for instance different smart glasses devices being worn by a doctor and scrub- technician), microphone and speaker controls are to be controlled to reduce and / or eliminate audio feedback without interaction from medical personnel engaged with a patient.

[0124] In an aspect, hardware-agnostic agentless automated participation of smart glasses devices in an electronic operating room collaboration session is provided. To facilitate an identified industry need, software can implement various automation without the need for a preinstalled software service agent. In some examples, preconfiguration of a smart glasses device enables medical personnel to use the smart glasses device with the only interaction being to turn the device on and adorn the device properly. The smart glasses device can perform agentless joining and participation as a participant device in the electronic operating room collaboration session. For instance, the smart glasses device is configured (preconfigured, prior to the surgery) with a proper Wi-Fi network configuration including an identification of the Wi-Fi network to which to connect and optionally credentials for the connection, which provides Internet connectivity to join the electronic operating room collaboration session. In this regard, the smart glasses device can also be configured to launch a browser on startup of the device, the browser having a home or start page specifically 6516.001AWO 21set / preconfigured to a desired uniform resource locator (URL) address to connect to the session via the establish network connection over the Wi-Fi network. In examples, no agent software is installed on the smart glasses device. On startup, for instance upon boot of the smart glasses device and connection via the browser, software can be served to the device dynamically via the Internet, and the device joins the scheduled session. For example, as noted, the smart glasses can launch a browser that automatically navigates to the URL, and proceed to retrieve and execute code in the browser for performing functions as part of the participation of the smart glasses device as a participant device in the electronic operating room collaboration session, as is described herein. For instance, the smart glasses device can begin transmitting / receiving video and / or audio automatically depending on tis preconfiguration or on control directives that it receives over its connection to the session. In the event of temporary loss of internet connectivity, the smart glasses can attempt to automatically reconnect to the session once internet connectivity restored. Automation in this regard can be performed entirely without a preinstalled software service agent to handle the session connection, thereby providing a hardware-agnostic solution.

[0125] FIG.1 provides an example conceptual depiction of automated connection of a smart glasses device to an electronic operating room collaboration session, in accordance with aspects described herein. Here, the smart glasses device 102 connects to various cloud services 106 available on / over a network – Internet 104 in this example. Initially, the smart glasses device 102 is preconfigured with details for connecting to the Internet 104 and joining a session – for instance the device 102 is configured with Wi-Fi information / credentials so that on boot, the device connects to the Wi-Fi network. A user presses a button of smart glasses device 102 for instance to power on (1) the device. The device 102 joins the Wi-Fi network and connects (2) to the internet through the Wi-Fi network. The device 102 continues by interacting with cloud services 106, one or more of which can be provided as described herein by computer systems existing in a cloud environment, for example. Service interactions may require, for functionality and / or security reasons, the smart glasses device to initiate outbound connections to the services. In examples, this is performed by way of a browser that the smart glasses device 102 is configured to automatically launch to navigate to a preconfigured / pre-set URL for interacting with a system providing cloud services in accordance with aspects described herein. 6516.001AWO 22

[0126] The smart glasses device 102 loads (3) a user interface (UI) from web service 108. In an example, an entire user interface is downloaded to the glasses device directly using any desired method, for instance a code-chunking method. The smart glasses device can include a screen / display that presents the UI. The UI could show status and various capabilities. In some examples, video stream(s), as selected by a remote participant, of other participating devices are shown in the UI, though in other examples the UI displays only minimal or no information at all.

[0127] The smart glasses device 102, if not already authenticated / authorized with the system by authentication service 110, authenticates and authorizes (4) as such. Medical staff may be required to interact and authenticate the smart glasses device with the system manually once every selected number of days based on session timeout for instance. This authentication refresh can be performed outside the critical path of the scheduled session. Once authenticated / authorized, then until timeout after that refresh, the smart glasses can utilize cached credentials handled by the browser to authenticate to the system.

[0128] The smart glasses device 102 loads (5) a scheduled collaboration session from data service 112, and then connects (6) to the session being hosted by the audio / video service 114.

[0129] In an aspect, hardware-agnostic agentless remote control of audio and video on smart glasses devices is provided as part of the electronic operating room collaboration session. In this regard, software executing (e.g., in the browser of a smart glasses device) implements control logic that traverses the Internet and allows a remote operator (by way of an internet-enabled remote participant computer system connected as a participant in the session) to control audio and / or video aspects of the smart glasses devices in use within the medical facility and collaboration session. Remote control can include the ability to, for instance, mute the microphone of a remote smart glasses device, mute and control the volume of audio of the session played through the smart glasses device, enable / disable transmission of video from the smart glasses device into the session, and remotely control what video feed, if any, of the session is shown on video display(s) of the smart glasses device.

[0130] FIG.2 depicts an example conceptual diagram of remote control of a smart glasses device in an electronic operating room collaboration session, in accordance with aspects described herein. In this example, a remote operator 202, for instance a remote 6516.001AWO 23medical device representative, as a remote participant uses a remote participant computer system 203 in the remote control of smart glasses 206 being word by a medical professional during surgery to remotely unmute / activate the smart glasses device 206 so that the medical professional can contribute audio into the session by way of the smart glasses 206. The user 202 watches the medical procedure via a live video feed from a camera of the smart glasses 206 being worn by the medial professional. Initially, the microphone of the device 206 is muted / disabled and thus no audio is captured from that device 206 or, still captured, is provided into the session, as indicated by the crossed-out microphone interface element icon shown in an interface of the computer system 203. The remote operator 202 wishes to enable the microphone to unmute the medical professional, and issues a user action (1), for instance selects that interface element or another interface element shown on the interface. This prompts a conveyance of control logic (2) across the internet 204 to the device 206. In examples, the control logic and status logic passed are handled by one or more cloud service(s) coordinating the session, which could be cloud services that provide features and aspects described herein. The control logic results in application (3) of the setting of the device 206 to enable / unmute its microphone. Feedback in the form of status logic (4), for instance to confirm that the setting was applied, is provided to the system 203, and the system update (5) the UI of system 203 to indicate that the smart glasses device 206 is now unmuted. The remote operator may be free to interact with any other user interface controls relating to the medical professional of any other participant in the session. For instance, the remote operator might use control logic to turn off the speakers of the scrub technician’s smart glasses and speakers of any other participant in the session, other than the medical professional, which would enable the remote operator to hold have an isolated conversation with the medical professional, as an example.

[0131] Code is downloaded by the browser and is executed within the browser, which IS on the RO's computer (and on the Surgeon's smart glasses). Upon connecting to the session, each endpoint (PC / smart glasses / other device) initiate a web socket connection to the cloud service. The RO clicks a button, the code running in the browser makes a call to the web service endpoint in the cloud via a web-socket. The cloud service relays that request to the intended surgeon smart glasses endpoint. The Smart glasses code (running in its browser) then checks the request to verify it is proper, and then runs the request LOCALLY on the smart glasses; such as “Enable The Microphone”. Once complete, a status message is sent by the smart glasses back up through the web-socket connection to the cloud service, which is 6516.001AWO 24relayed to ALL attendees, so they can update their UI, showing that the surgeon's smart glasses microphone is enabled. The messages sent are control requests, not executable code (as that would be a BIG security risk).

[0132] FIG.3 depicts an example conceptual diagram of hardware agnostic agentless multi-smart glass device participation in close proximity, in accordance with aspects described herein. In various embodiments, after a smart glasses device initializes and boots to load the browser and connect to the URL, but before joining the session, code running in the browser can download an initial configuration from cloud data storage of a cloud service and apply the settings before connecting to the session. Example such settings to automatically apply can include configuring the microphone, speakers, and / or any camera / video settings or feed of the smart glasses device. The settings can be those that are configured to best suit a multi-device environment. More generally, this initialization and application of settings to best suit a multi-device environment can be for any types of devices – smartphone or any other mobile devices or computer systems, video feeds of equipment connecting to the session, etc. - to best suit the multi-device environment.

[0133] As noted above, a remote operator may selectively isolate communication with one smart glasses device by manipulating both microphone and speakers of other participating devices via provided controls, reducing and / or eliminating audio feedback from close proximity of other devices. FIG.3 depicts three scenarios (310A, 310b, 310c) in which the remote operator 302 has interface controls via the operator’s computer system 303 to control (e.g., reduce) session volume to each smart glasses participant to further reduce and / or eliminate audio feedback. In the scenarios 310a, 310b, and 310c, remote operator 302 participates in a session via remote participant computer system 303. Participating individuals via respective smart glasses devices also include scrub tech 304 and surgeon 306. Microphone and speaker icons proximate each participant indicate status of the microphone and audio output / speaker volume associated with the participant’s device.

[0134] In scenario 310a, the microphones of the devices used by the three participants 302, 304, 306 are unmuted, and the audio output volume of the devices are all set to 100% volume. Such a situation may not be optimal from an audio standpoint because audio feedback loops can result. For instance, a microphone of one of the smart glasses devices could pick up audio being played through the speaker of the other smart glasses device and 6516.001AWO 25result in an audio loop that renders audio communication in the session impossible. This feedback could result from multiple devices (both sets of smart glasses) in this example.

[0135] In scenario 310b, the speaker output (volume) level for the remote operator 302 has been reduced through control directives as part of preconfiguration or remote operator to 75%. The microphones of the remote operator and the scrub tech remain unmuted, but the microphone of the surgeon has been muted. Meanwhile, speaker output levels for the scrub tech and the surgeon have been reduced to 55% and 65%, respectively. In examples, a cloud service controlling session configuration can automatically configure the participating devices as they enter the session to configure the session audio to eliminate potential audio feedback. As an example, the automatic configuration can mute ‘low priority’ participant microphones and set speaker output levels to appropriate volumes. This can eliminate audio feedback loops that could otherwise occur due to multiple participating devices being within sufficiently close proximity to one another.

[0136] Scenario 310c shows a configuration where the remote operator 302 desires to have an isolated audio conversation with a selected participant – the scrub tech 304 here. Relative to scenario 310b, the remote operator 302 has muted the speaker output of the surgeon smart glasses device. This enables the remote operator 303 to carry-on a conversation with the scrub tech without disturbing the surgeon, as the surgeon would not be able to hear or participate in the conversation.

[0137] Some smart glasses are configured to listen for voice commands and invoke certain functions based thereon. Consequently, random sounds could occasionally trigger inadvertent actions on the smart glasses and therefore, in some embodiments, any voice command feature of a smart glasses device can be configured with a wake word or phrase that the device is less likely to interpret from ambient sounds or be disabled entirely to avoid inadvertent invocation.

[0138] FIG.4 depicts an example interface of a remote participant computer system, for instance a computer system being used by a remote operator, in accordance with aspects described herein. Interface 400 may be presented on a display of the remote operator’s computer system and enable the remote operator to see and hear communications taking place in the electronic operating room collaboration session, and select interface elements to remotely control participation of the various session participating devices in the session, for 6516.001AWO 26instance via issued control directives. Interface 400 includes two general areas – sidebar area 402 and primary viewing area 406.

[0139] Sidebar area 402 shows participant-specific interface areas corresponding to each of one or more session participants, with each participant-specific interface area showing condensed A / V controls and a video feed, if applicable, attributed to that participant. In the example of FIG.4, sidebar area 402 shows participant-specific interface area 404a for a surgeon participant wearing a smart glasses device. The video stream from the surgeon’s smart glasses is on in this example. Sidebar area 402 also shows participant-specific interface area 404b for a video feed, for instance an equipment video feed of information output by an equipment computer system of operating room equipment. Here, the equipment is a fluoroscopic equipment showing an x-ray of patient anatomy. Sidebar area 402 also shows participant-specific interface areas 404c and 404d for the remote operator and charge nurse, respectively, with no video currently being shared by either. Each participant-specific interface area also includes A / V controls in the form of icons that are explained below.

[0140] The primary viewing area 406 shows participant-specific interface areas as well for one or more session participants, though in a larger presentation than the condensed presentation of the participant-specific interface areas shown in the sidebar area 402. Here, another participant in the session is the scrub tech. Primary area 406 shows a participant- specific interface area 408 corresponding to the scrub tech and showing A / V controls 410 and shared video 412 from a camera of a smart glasses device being worn by the scrub tech. The video stream from the scrub tech’s smart glasses is on in this example. The sidebar area 402 does not in this example show a separate participant-specific interface area corresponding to the scrub tech on account that the scrub tech’s interface area has been selected for inclusion in the primary viewing area 406, though the participant-specific interface area of the scrub tech could be included redundantly in the sidebar area 402, if desired. Additionally, the primary viewing area 406 is not limited to display of just one participant-specific interface area. Any one or more participant-specific interface areas could be placed in the primary viewing area 406. In examples, any participant-specific interface area can be moved by the remote operator between the sidebar area 402 and the primary viewing area 406 by interacting with (selecting, clicking, dragging, etc. elements within) the interface. In the event that multiple participant- specific interface areas are placed in the primary viewing area 406, those participant-specific interface areas could be presented in any desired manner, for instance stacked, side-by-side, 6516.001AWO 27or in a scrollable window in which the user scrolls to see lower participant-specific interface area(s). Similarly, the sidebar area 402 could be made scrollable if there is not sufficient room to fit all participant-specific interface areas to be displayed therein.

[0141] The participant-specific interface area 408 corresponding to the scrub tech shows a livestream, as video feed 412, from a camera of the smart glasses device being worn by the scrub tech. A / V controls 410 enable the remote participant / operator to remotely control the participation of the scrub tech’s device in the electronic operating room collaboration session, for instance to control the issuance of control directives to the smart glasses device to change A / V function(s) of the smart glasses device. Such a change can alter at least one of audio and video associated with the smart glasses device. It is noted that a control directive imposes a control on the device to cause a change in one or more settings of the device, as opposed to merely controlling how audio / video received from the device is mixed into the collaboration session and presented to the other participants. Thus, this differs from, for example, a scenario where the participant device provides audio / video to an intermediary device which then selects an appropriate mix of that audio / video into the audio / video of the session.

[0142] The A / V controls 410 are implemented as interactive interface elements in this example. One A / V function is an audio output volume of the smart glasses device. A control request to change this A / V function includes a request to adjust the audio output volume and the control directive directs the smart glasses device to adjust the audio output volume. Elements 414 and 418 can be selected to invoke a control request to either lower or raise, as the case may be, the speaker volume of the session audio being played through the speaker of the smart glasses device. Element 416 can be selected to invoke a control request to mute the session audio being played through the speaker of the smart glasses device. Another A / V function is a video capture or output state of the smart glasses devices. A control request to change this A / V function includes a request to enable or disable video capture or output from the smart glasses device, and the control directive directs the smart glasses device to enable or disable the capture or output of video of the smart glasses device. Element 420 can be selected to invoke a control request to toggle between enabling and disabling video capture by the camera of the smart glasses device and / or output of captured images from the smart glasses device, and therefore can effectively enable or disable a video feed from the device in the session. Yet another A / V function is a microphone operating state of a microphone of the smart glasses device. A control request to change this A / V function includes a request to 6516.001AWO 28mute or unmute the microphone of the smart glasses device, and the control directive directs the smart glasses device to mute (or unmute) the microphone. Element 422 can be selected to invoke such a control request. Element 424 can be selected to kick the participant from the session entirely. This could be useful in a situation where a participant is an unauthorized participant, or where the participant device is causing technical problems in the session, as examples.

[0143] Another example A / V function that could be controlled is a video display state of the smart glasses device that is used to set which video stream(s) presented in the session are to be streamed to the smart glasses device. A control request to change this A / V function includes a request that selects which video stream(s) of the various video stream(s) in the session to present on a display of the smart glasses device, and the control directive directs the smart glasses device to present the selected video stream(s) on the display.

[0144] The following presents an example flow for a specific surgical case to incorporate aspects described herein. Initially, as part of the preparation for the surgical case, a doctor’s office requests the surgery and writes a surgical request for a patient. As part of this, the doctor can specify the treatment, location, equipment, and personnel required. In cases where the doctor decides that a remote medical device representative is an acceptable service level (or perhaps this is the only option to attain the expertise required), the doctor calls for a remote medical device representative ‘case’.

[0145] In some examples, the hospital or ambulatory surgery center (ASC) would schedule their operation rooms equipped with technology described herein for electronic operating room collaboration sessions.

[0146] An example system sets up the hospital / ASC accordingly. For instance, the system provides the subject operating room with dedicated Wi-Fi, equips the operating room with sets (e.g., two or more) of smart glasses devices that are programmed to auto-accept collaboration sessions, and equips the equipment computer system(s) associated with operating room equipment (such as a fluoroscope) to auto-accept to collaboration sessions. This enables the smart glasses devices or equipment computer systems to automatically on power-up join the session to which they were invited. In some embodiments, the device on power-up launches into a browser session configured to automatically load (e.g., by way of a URL or other resource locator) into the collaboration session. Launched sessions can be 6516.001AWO 29opened and have invited that device to the session. device automatically connects into that session.

[0147] Training with the system can be provided to medical device representatives, doctors, surgeons, scrub techs, schedulers, charge nurses, circulating nurses, and hospital information technology staff.

[0148] The hospital scheduler can provide preconfiguration data for use in preconfiguring the collaboration session. The scheduler can book rooms and equipment, coordinate with the surgeon, invite personnel and give role permissions, as examples. The SCHEDULER can be a role defined in the software (e.g., of a cloud service providing aspects described herein) that is granted to a user’s profile. Only those with the SCHEDULER role may be allowed to schedule a session in the software, while others (not assigned the SCHEDULER role) may not be able to view the interface, nor call the related application programming interfaces to preconfigure sessions. This role's permissions can be configured including data model level access / restriction.

[0149] For a particular case when ready to perform the surgery, the charge nurse starts the collaboration session, for instance by invoking a cloud service to start the session. Invites and permissions have already been established / preconfigured for the session. The charge nurse (or circulating nurse) can launch the session on an operating room computer system. Timing can be fluid, and, in examples, the operating room nurse controls the traffic and start time. In this regard, the scheduler can choose, as part of the preconfiguration when the session is scheduled, which participants to invite, and preconfigure which participants can send video / audio, for instance preconfigure by limiting or enabling such activity. This can be done by granting defined roles, such as OBSERVER, REMOTE OPERATOR, or SMARTGLASS, as examples, which can be associated with different pre-set A / V function settings for their devices. The scheduler can also determine when the meeting starts, even if it is before or after the planned start time of the meeting. This can be important to ensure that the plan is executed only when the patient is prepared, the surgeon is scrubbed-in, and other necessary preparation has taken place. The scheduler can therefore set up the cases, while the charge or circulating nurse can in the operating room coordinating ‘traffic’ therein. Example of this traffic coordination includes determining when the X-ray techs are to start setting up, determining whether the room is clean and ready from a prior surgery, ensuring that all personnel are ready and at their stations, determining whether the surgical tools and implants 6516.001AWO 30are ready and prepared and that the correct implant trays arrived from the sterilization equipment, determining when the doctor is ready, and determining when the wheel-m the patient in. When ready, the charge / circulating nurse can launch the session to cause the equipment to join as run time participant devices in the session. The nurse can also perform activity to determine whether the remote device representative is connected and operational, and whether the equipment and participant devices (including the smart glasses devices) are running properly.

[0150] During the session, the remote representative, as a remote participant in the session, monitors the procedure and controls audio and video feeds of each participant by controlling the A / V functions of the devices. The remote representative can be granted access to do this by the scheduler. When the nurse starts the session, the representative can see and hear all other non-equipment computer system participants.

[0151] smart glasses devices, as participant devices, can be registered devices that auto- launch into the session. Generally speaking, the surgical environment is sterile and therefore the construction, cleanliness, and use of the smart glasses can be conducive to maintaining as clean an environment as possible. In examples, a smart glasses device worn by a user does not need to be (and should not be) touched after the user powers and places it on until the surgery concludes. On power-up, the device can auto-launch the specific session that it has been configured to attend. Because sound in the operating room can be chaotic, voice commands may not be practical and can be disabled.

[0152] An equipment viewer may be or include an equipment computer system. Registered operating room equipment can auto-launch into the session, for instance when powered on. An equipment computer system may be a computer system that is part of, or associated with, operating room equipment, such as a fluoroscopy machine as an example. In the event that the operating room equipment does not have a computer system to leverage for connecting to the session, then an equipment computer system can be associated with the operating room equipment for joining the session. For instance, a tablet or laptop could be connected to a standard video output port of the operating room equipment, for instance a medical video imaging device (such as a fluoroscope), where the tablet / laptop joins the session as the equipment computer system. In this regard, there could be a participant role defined as a VIDEO FEED, having restrictions applied to it in its default configuration (subject to revision) that restrict the ability to receive any video / audio. In a particular 6516.001AWO 31example, a device is used to convert a signal (such as an HDMI signal) from the operating room equipment into an output signal, such as one for passing via a USB-C connection into an equipment computer system, such as a tablet or laptop as an input. That input can act like a camera of the equipment computer system. The feed from that ‘camera’ can be provided as another video feed in the session for the remote representative to see what the operating room equipment (X-ray, fluoroscope, etc.) is showing. As described above, a VIDEO FEED role can be designated for these types of devices. The equipment computer system can automatically launch a web browser and access a destination resource that launched and joins the collaboration session that has invited that Video Feed device. In examples, the tablet / laptop is initially turned off but preconfigured to launch into a collaboration session. The tablet / laptop can be powered on after the input from the operating room equipment is attached, and the table / laptop can auto-join the prescribed session.

[0153] Observer (‘view and hear’) participants can be invited to view and hear the collaboration session. Participants can be invited by the scheduler with a role enabling them to view and hear the session. Generally, these participants can neither affect nor communicate into the session. The remote representative can mute and / or turn off their feeds. This role could typically be used for training future medical experts.

[0154] In accordance with various aspects described herein, agentless participating devices can pass audio and / or video data through secure channels into a collaboration session that may be hosted / coordinated by cloud services, for instance. This enables A / V to be passed between the participating devices in the operating room and the remote participant computer system. This provides the remote participant, e.g., a remote medical device representative, with an up-close and detailed view on the representative’s display by way of the participating devices in the operating room. The views can show the surgical procedure, the implant or other instrument table, and operating room equipment.

[0155] Aspects can advantageously address situations often encountered by medical device representatives and other personnel in operating rooms where their vantage points are obstructed and sounds in the operating room can be distracting. When kept out of the sterile field of surgery, vantage points to the patient, selected tables, carts, and the like can be very poor. This can be problematic because often the representatives are relied upon for supporting selection and / or use of objects on implant trays, which can be complicated. Drills, gages, fixtures, screws, and other equipment / devices may be present and the scrub techs often rely 6516.001AWO 32on the representative to help navigate these devices. The doctors, meanwhile, typically know these devices very well but expect the scrub tech to identify and provide the doctor with the correct item on each exchange, and without hesitation. Medical device representative often provide valuable assistance in this regard.

[0156] The views offered by live video feeds from wearable devices can be far superior to what the individual might otherwise see if directly visually observing in the operating room. In examples, the surgeon and scrub technician wear smart glasses devices that provide their respective video feeds of their respective points of view into the collaboration session. Video feed(s) corresponding to equipment such as a fluoroscope is / are also sent into the session. These collective video feeds along with audio capture by the devices can be provide through the Wi-Fi connection in the operating room, over network(s), such as a local network and / or the Internet, to cloud services and the remote representative. Audio and / or video from the remote participant (e.g., remote representative) can similarly be provided into the session.

[0157] Table 1 below presents a glossary of words and phrases that may be encountered in this document together with non-limiting, example descriptions or definitions of the words and phrases.6516.001AWO 336516.001AWO 346516.001AWO 356516.001AWO 366516.001AWO 376516.001AWO 386516.001AWO 39Table 1

[0158] FIG.5 depicts an example process for providing an electronic operating room collaboration session, in accordance with aspects described herein. The process may be executed, in one or more examples, by a processor or processing circuitry of one or more computers / computer systems, such as those described herein, which could be or include one of more computer system provided in a cloud computing environment. In examples, code or instructions implementing the process(es) of FIG.5 are part of one or more code modules and / or one or more code sub-modules of other code modules. Various options are available.

[0159] The process of FIG.5 includes preconfiguring (502) an electronic operating room collaboration session based on session preconfiguration data received from a scheduling component. Preconfiguration can include the establishment of collaboration session information prior to commencement of the collaboration session, which facilitates the automated nature of the authenticating / joining of participant devices. The preconfiguring can, for example, set roles for participation of a plurality of devices in the electronic operating room collaboration session. In examples, the scheduling component is a computer system utilized by a scheduler user.

[0160] The process continues by starting (504) the electronic operating room collaboration session and connecting the plurality of devices as participant devices in the electronic operating room collaboration session. The plurality of devices includes an operating room computer system, for instance one used by a charge or circulating nurse. The plurality of devices also includes a plurality of smart glasses devices. The connecting of the plurality of smart glasses devices connects the devices as participants in the electronic operating room collaboration session, and this can be based on authentication of the plurality of smart glasses device to join the electronic operating room collaboration session. Such authentication can be dictated and facilitated by the preconfiguring of the session. 6516.001AWO 40

[0161] The plurality of devices also includes at least one equipment computer system that presents at least one equipment video feed graphically depicting information of a corresponding at least one operating room equipment. In examples, the operating room equipment interfaces with the equipment computer system through a dedicated communication path, for instance a physical cable, such as an HDMI or USB cable, or a wireless communication path, for instance using a near-field communication protocol such as Bluetooth as one example. The equipment computer system could be a tablet or other mobile device into which the operating room equipment plugs and provides output such as images / video. the equipment computer system could be a dedicated computer system that accepts feed(s) from one or more operating room equipment, though in typical examples there is a dedicated equipment computer system for each individual piece of operating room equipment for which to include a video stream / equipment video feed in the session.

[0162] The plurality of devices also includes a remote participant computer system connected as a participant in the electronic operating room collaboration session.

[0163] The process of FIG.1 also includes receiving (506), from the at least one equipment computer system, the at least one equipment video feed and presenting the at least one equipment video feed as one or more additional participants in the electronic operating room collaboration session, and further includes receiving (508) at least one video stream from a corresponding at least one smart glasses device of the plurality of smart glasses devices, and presenting the at least one video stream to the remote participant computer system as part of the electronic operating room collaboration session. It may be that the process presents all or just a subset (one or more) of the video streams received from the various smart glasses devices, as the video capture might have been disabled on one or more of the smart glasses devices.

[0164] The process of FIG.1 additionally includes remotely controlling (510) participation of the plurality of smart glasses devices in the electronic operating room collaboration session. The remotely controlling includes issuing a control directive, such as a command, invocation, or the like, to a smart glasses device of the plurality of smart glasses devices to change an audio / video (A / V) function of a smart glasses device. The change of the A / V function alters at least one of audio and video associated with the smart glasses device. 6516.001AWO 41

[0165] In some embodiments, the remotely controlling is based on a control request to change the A / V function. In some further embodiments, the remotely controlling provides remote control to the remote participant computer system by accepting control requests from the remote participant computer system. In these scenarios, the method can further include receiving from the remote participant computer system the control request to change the A / V function based on invocation of the control request by a remote participant user.

[0166] In embodiments, the A / V function includes a microphone operating state of a microphone of the smart glasses device, where the control request to change the A / V function includes a request to mute the microphone, and where the control directive directs the smart glasses device to mute the microphone.

[0167] In an additional or alternative embodiment, the A / V function includes an audio output volume of the smart glasses device, where the control request to change the A / V function includes a request to adjust the audio output volume, and where the control directive directs the smart glasses device to adjust the audio output volume. Note that this encompasses a situation to mute the output, which is just a special case of output volume adjustment.

[0168] In an additional or alternative embodiment, the A / V function includes a video capture or output state of the smart glasses devices, where the control request to change the A / V function includes a request to enable or disable video capture or output from the smart glasses device, and where the control directive directs the smart glasses device to enable or disable the video capture or output from the smart glasses device. There are multiple possibilities for video-related state control dealing with camera / video disabling. Disabling video capture, for instance, could disable the device’s camera so that image capture is not enabled. Alternatively, disabling video output from the device could mean that images are captured by the camera, but the device can effectively drop the captured frames or otherwise refrain from processing and sending the image data into the collaboration session.

[0169] In an additional or alternative embodiment, the A / V function includes a video display state of the smart glasses device, meaning video that the smart glasses are configured to show, where the control request to change the A / V function includes a request that selects which video stream of the at least one video stream to present on a display of the smart glasses device, and where the control directive directs the smart glasses device to present the selected video stream on the display. 6516.001AWO 42

[0170] Provided in accordance with some embodiments is a selective control aspect of the output volumes of session participants, whereby the remotely controlling includes selectively controlling audio output volume levels of participants in the electronic operating room collaboration session. Additionally, this can be used to provide isolated communication, in which the remotely controlling includes selectively isolating communication between the remote participant computer system and a smart glasses device by issuing control directives to mute a microphone and audio output of one or more participants, other than the remote participant computer system and the smart glasses device, in the electronic operating room collaboration session.

[0171] In some embodiments, each smart glasses device of the plurality of smart glasses devices performs agentless participation as a participant device in the electronic operating room collaboration session, in which the smart glasses device is configured with a Wi-Fi configuration that includes identification of a Wi-Fi network for connection to the Internet and a starting uniform resource locator (URL), and the smart glasses device is configured to, on startup of the smart glasses device, connect to the Wi-Fi network, launch a browser that automatically navigates to the URL, and retrieve and execute code in the browser for performing functions as part of the participation of the smart glasses device as a participant device in the electronic operating room collaboration session.

[0172] Processes described herein may be performed singly or collectively by one or more computer systems, such as one or more computer systems discussed herein. FIG.6 depicts one example of such a computer system and associated devices to incorporate, use and / or perform aspects described herein. A computer system may also be referred to herein as a data processing device / system, computing device / system / node, or simply a computer. The computer system may be based on one or more of various system architectures and / or instruction set architectures, such as those offered by International Business Machines Corporation (Armonk, New York, USA), Intel Corporation (Santa Clara, California, USA) or ARM Holdings plc (Cambridge, England, United Kingdom), as examples.

[0173] FIG.6 shows a computer system 600 in communication with external device(s) 612. Computer system 600 includes one or more processor(s) 602, for instance central processing unit(s) (CPUs). A processor can include functional components used in the execution of instructions, such as functional components to fetch program instructions from locations such as cache or main memory, decode program instructions, and execute program 6516.001AWO 43instructions, access memory for instruction execution, and write results of the executed instructions. A processor 602 can also include register(s) to be used by one or more of the functional components. Computer system 600 also includes memory 604, input / output (I / O) devices 608, and I / O interfaces 610, which may be coupled to processor(s) 602 and each other via one or more buses and / or other connections. Bus connections represent one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include the Industry Standard Architecture (ISA), the Micro Channel Architecture (MCA), the Enhanced ISA (EISA), the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI).

[0174] Memory 604 can be or include main or system memory (e.g. Random Access Memory) used in the execution of program instructions, storage device(s) such as hard drive(s), flash media, or optical media as examples, and / or cache memory, as examples. Memory 604 can include, for instance, a cache, such as a shared cache, which may be coupled to local caches (examples include L1 cache, L2 cache, etc.) of processor(s) 602. Additionally, memory 604 may be or include at least one computer program product having a set (e.g., at least one) of program modules, instructions, code or the like that is / are configured to carry out functions of embodiments described herein when executed by one or more processors.

[0175] Memory 604 can store an operating system 605 and other computer programs 606, such as one or more computer programs / applications that execute to perform aspects described herein. Specifically, programs / applications can include computer readable program instructions that may be configured to carry out functions of embodiments of aspects described herein.

[0176] Examples of I / O devices 608 include but are not limited to microphones, speakers, Global Positioning System (GPS) devices, cameras, lights, accelerometers, gyroscopes, magnetometers, sensor devices configured to sense light, proximity, heart rate, body and / or ambient temperature, blood pressure, and / or skin resistance, and activity monitors. An I / O device may be incorporated into the computer system as shown, though in some embodiments an I / O device may be regarded as an external device (612) coupled to the computer system through one or more I / O interfaces 610. 6516.001AWO 44

[0177] Computer system 600 may communicate with one or more external devices 612 via one or more I / O interfaces 610. Example external devices include a keyboard, a pointing device, a display, and / or any other devices that enable a user to interact with computer system 600. Other example external devices include any device that enables computer system 600 to communicate with one or more other computing systems or peripheral devices such as a printer. A network interface / adapter is an example I / O interface that enables computer system 600 to communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet), providing communication with other computing devices or systems, storage devices, or the like. Ethernet-based (such as Wi-Fi) interfaces and Bluetooth® adapters are just examples of the currently available types of network adapters used in computer systems (BLUETOOTH is a registered trademark of Bluetooth SIG, Inc., Kirkland, Washington, U.S.A.).

[0178] The communication between I / O interfaces 610 and external devices 612 can occur across wired and / or wireless communications link(s) 611, such as Ethernet-based wired or wireless connections. Example wireless connections include cellular, Wi-Fi, Bluetooth®, proximity-based, near-field, or other types of wireless connections. More generally, communications link(s) 611 may be any appropriate wireless and / or wired communication link(s) for communicating data.

[0179] Particular external devices 612 can be operating room equipment. Other particular external device(s) 612 may include one or more data storage devices, which may store one or more programs, one or more computer readable program instructions, and / or data, etc. Computer system 600 may include and / or be coupled to and in communication with (e.g. as an external device of the computer system) removable / non-removable, volatile / non-volatile computer system storage media. For example, it may include and / or be coupled to a non- removable, non-volatile magnetic media (typically called a "hard drive"), a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk"), and / or an optical disk drive for reading from or writing to a removable, non-volatile optical disk, such as a CD-ROM, DVD-ROM or other optical media.

[0180] Computer system 600 may be operational with numerous other general purpose or special purpose computing system environments or configurations. Computer system 600 may take any of various forms, well-known examples of which include, but are not limited to, personal computer 6516.001AWO 45(PC) system(s), server computer system(s), such as messaging server(s), thin client(s), thick client(s), workstation(s), laptop(s), handheld device(s), mobile device(s) / computer(s) such as smartphone(s), tablet(s), and wearable device(s), multiprocessor system(s), microprocessor- based system(s), telephony device(s), network appliance(s) (such as edge appliance(s)), virtualization device(s), storage controller(s), set top box(es), programmable consumer electronic(s), network PC(s), minicomputer system(s), mainframe computer system(s), and distributed cloud computing environment(s) that include any of the above systems or devices, and the like.

[0181] A particular example of a computer system is an eyewear based wearable device, for instance a wearable smart glasses device. A smart glasses device can include many of the same types of components included in a computer system described above. The smart glasses device can be configured to be wearable on the head of the device user. The device can optionally include a display that is positioned in a peripheral vision line of sight of the user when the device is in operative position on the user’s head. Suitable displays can utilize LCD, CRT, or OLED display technologies, as examples. Lenses, if existent, of the smart wearable device may optionally include active translucent displays, in which inner and / or outer surface(s) of the lenses are capable of displaying images and other content. This provides the ability to impose this content directly into the line of sight of the user, overlaying at least part of the user’s view to the environment through the lenses. In particular embodiments described herein, graphical elements presented on the lens displays are AR elements overlaying a view to the environment, though a display of a smart glasses device could in examples display video stream(s) / images not as AR elements.

[0182] A smart glasses device could include a touch input portion enabling users to input touch-gestures in order to control functions of the device, though it might be desired for reasons noted above to disable or prevent / limit touch-based controls. In examples, the power switch / button is provided as a touch-based control.

[0183] Various other input devices of a smart glasses device may be provided, such as camera which can be used to capture images or video. The camera could be used by the device to obtain image(s) / video of a view of the wearer’s environment to use in, for instance, capturing images / videos of a scene, such as a surgical procedure and / or instruments around the procedure.

[0184] One or more microphones, proximity sensors, light sensors, accelerometers, speakers, GPS devices, and / or other input devices (not labeled) may be additionally provided, 6516.001AWO 46for instance within a housing of the smart glasses device. The housing can also include other electronic components, such as electronic circuitry, including processor(s), memory, and / or communications devices, such as cellular, short-range wireless (e.g., Bluetooth), or Wi-Fi circuitry for connection to remote devices. The housing can further include a power source, such as a battery to power components of the device. Additionally or alternatively, any such circuitry or battery can be included in an enlarged end, which may be enlarged to accommodate such components. An enlarged end, or any other portion of the device, can also include physical port(s) used to connect the device to a power source (to recharge a battery) and / or any other external device, such as a computer. Such physical ports can be of any standardized or proprietary type, such as Universal Serial Bus (USB).

[0185] Aspects disclosed herein may be a system, a method, and / or a computer program product, any of which may be configured to perform or facilitate aspects described herein. Computer systems configured to perform these and other methods, and computer program products that include a computer readable storage medium storing instructions for execution to perform these and other methods are also provided.

[0186] In some embodiments, aspects of the present invention may take the form of a computer program product, which may be embodied as computer readable medium(s). A computer readable medium may be a tangible storage device / medium having computer readable program code / instructions stored thereon. Example computer readable medium(s) include, but are not limited to, electronic, magnetic, optical, or semiconductor storage devices or systems, or any combination of the foregoing. Example embodiments of a computer readable medium include a hard drive or other mass-storage device, an electrical connection having wires, random access memory (RAM), read-only memory (ROM), erasable- programmable read-only memory such as EPROM or flash memory, an optical fiber, a portable computer disk / diskette, such as a compact disc read-only memory (CD-ROM) or Digital Versatile Disc (DVD), an optical storage device, a magnetic storage device, or any combination of the foregoing. The computer readable medium may be readable by a processor, processing unit, or the like, to obtain data (e.g. instructions) from the medium for execution. In a particular example, a computer program product is or includes one or more computer readable media that includes / stores computer readable program code to provide and facilitate one or more aspects described herein. 6516.001AWO 47

[0187] As noted, program instruction contained or stored in / on a computer readable medium can be obtained and executed by any of various suitable components such as a processor of a computer system to cause the computer system to behave and function in a particular manner. Such program instructions for carrying out operations to perform, achieve, or facilitate aspects described herein may be written in, or compiled from code written in, any desired programming language. In some embodiments, such programming language includes object-oriented and / or procedural programming languages such as C, C++, C#, Java, etc.

[0188] Program code can include one or more program instructions obtained for execution by one or more processors. Computer program instructions may be provided to one or more processors of, e.g., one or more computer systems, to produce a machine, such that the program instructions, when executed by the one or more processors, perform, achieve, or facilitate aspects of the present invention, such as actions or functions described in flowcharts and / or block diagrams described herein. Thus, each block, or combinations of blocks, of the flowchart illustrations and / or block diagrams depicted and described herein can be implemented, in some embodiments, by computer program instructions.

[0189] Although various embodiments are described above, these are only examples.

[0190] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and / or “comprising”, when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0191] The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below, if any, are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The

[0192] description of one or more embodiments has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the 6516.001AWO 48art. The embodiment was chosen and described in order to best explain various aspects and the practical application, and to enable others of ordinary skill in the art to understand various embodiments with various modifications as are suited to the particular use contemplated. 6516.001AWO 49

Claims

CLAIMS What is claimed is:

1. A computer-implemented method including: preconfiguring an electronic operating room collaboration session based on session preconfiguration data received from a scheduling component, the preconfiguring setting roles for participation of a plurality of devices in the electronic operating room collaboration session; starting the electronic operating room collaboration session and connecting the plurality of devices as participant devices in the electronic operating room collaboration session, the plurality of devices including: an operating room computer system; a plurality of smart glasses devices, wherein the plurality of smart glasses devices are connected as participants in the electronic operating room collaboration session based on authentication of the plurality of smart glasses devices to join the electronic operating room collaboration session as dictated by the preconfiguring; at least one equipment computer system presenting at least one equipment video feed graphically depicting information of a corresponding at least one operating room equipment; and a remote participant computer system connected as a participant in the electronic operating room collaboration session; receiving, from the at least one equipment computer system, the at least one equipment video feed and presenting the at least one equipment video feed as one or more additional participants in the electronic operating room collaboration session; receiving at least one video stream from a corresponding at least one smart glasses device of the plurality of smart glasses devices, and presenting the at least one video stream to the remote participant computer system as part of the electronic operating room collaboration session; and 6516.001AWO 50remotely controlling participation of the plurality of smart glasses devices in the electronic operating room collaboration session, the remotely controlling including issuing a control directive to a smart glasses device of the plurality of smart glasses devices to change an audio / video (A / V) function of the smart glasses device, wherein the change of the A / V function alters at least one of audio and video associated with the smart glasses device.

2. The method of claim 1, wherein the remotely controlling is based on a control request to change the A / V function. The method of claim 2, wherein the remotely controlling provides remote control to the remote participant computer system by accepting control requests from the remote participant computer system, and wherein the method further includes receiving from the remote participant computer system the control request to change the A / V function based on invocation of the control request by a remote participant user.

4. The method of claim 2, or 3, wherein the A / V function includes a microphone operating state of a microphone of the smart glasses device, wherein the control request to change the A / V function includes a request to mute the microphone, and wherein the control directive directs the smart glasses device to mute the microphone.

5. The method of claim 2, or 3, wherein the A / V function includes an audio output volume of the smart glasses device, wherein the control request to change the A / V function includes a request to adjust the audio output volume, and wherein the control directive directs the smart glasses device to adjust the audio output volume.

6. The method of claim 2, or 3, wherein the A / V function includes a video capture or output state of the smart glasses devices, wherein the control request to change the A / V function includes a request to enable or disable video capture or output from the smart glasses device, and wherein the control directive directs the smart glasses device to enable or disable the video capture or output from the smart glasses device. The method of claim 2, or 3, wherein the A / V function includes a video display state of the smart glasses device, wherein the control request to change the A / V function includes a request that selects which video stream of the at least one video stream to 6516.001AWO 51present on a display of the smart glasses device, and wherein the control directive directs the smart glasses device to present the selected video stream on the display.

8. The method of claim 1, 2, or 3, wherein the remotely controlling includes selectively controlling audio output volume levels of participants in the electronic operating room collaboration session. The method of claim 1, 2, or 3, wherein the remotely controlling includes selectively isolating communication between the remote participant computer system and a smart glasses device by issuing control directives to mute a microphone and audio output of one or more participants, other than the remote participant computer system and the smart glasses device, in the electronic operating room collaboration session.

10. The method of claim 1, 2, or 3, wherein each smart glasses device of the plurality of smart glasses devices performs agentless participation as a participant device in the electronic operating room collaboration session, in which the smart glasses device is configured with a Wi-Fi configuration that includes identification of a Wi-Fi network for connection to the Internet and a starting uniform resource locator (URL), wherein the smart glasses device is configured to, on startup of the smart glasses device, connect to the Wi-Fi network, launch a browser that automatically navigates to the URL, and retrieve and execute code in the browser for performing functions as part of the participation of the smart glasses device as a participant device in the electronic operating room collaboration session.

11. A computer system including: a memory; and a processor in communication with the memory, wherein the computer system is configured to perform a method according to any of claims 1 through 10.

12. A computer program product including: a computer readable storage medium storing instructions for execution to perform a method according to any of claims 1 through 10. 6516.001AWO 52

Images