Evaluating the performance of medical procedures by healthcare professionals through an online collaborative medical platform

Abstract

A collaborative medical platform facilitates remote collaboration related to medical procedures during the pre-procedure, intra-procedure, and post-procedure phases of a medical case. The collaborative medical platform receives a request for evaluation from a healthcare professional, along with segments of data describing the selection of an evaluation framework and the performance of the medical procedure for evaluation. The collaborative medical platform can transmit the evaluation request, containing the selected framework and selected segments, to a selected evaluation healthcare professional who provides the evaluation results. Alternatively, the collaborative medical platform automatically generates evaluation results by applying an evaluation model to the selected framework and selected segments. The evaluation results can be used to select educational content items for healthcare professionals.

Description

Technology Field

[0001] Cross-reference of related applications

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 605,879 filed December 4, 2023; U.S. Provisional Patent Application No. 63 / 641,754 filed May 2, 2024; U.S. Provisional Patent Application No. 63 / 661,015 filed June 17, 2024; U.S. Provisional Patent Application No. 63 / 661,858 filed June 19, 2024; U.S. Provisional Patent Application No. 63 / 717,950 filed November 8, 2024; U.S. Provisional Patent Application No. 63 / 718,000 filed November 8, 2024; and U.S. Provisional Patent Application No. 63 / 719,015 filed November 11, 2024, the contents of which are respectively incorporated herein by reference It is included.

[0003] background

[0004] Technology field

[0005] The described embodiment relates to a system and method for evaluating the performance of a medical professional in performing a medical procedure based on data describing the medical procedure captured by a collaborative medical platform. Background Technology

[0006] The different skills employed by healthcare professionals when performing medical procedures influence patient outcomes. To maintain or improve patient outcomes, data describing the medical procedure is reviewed after its completion to identify the skills employed during the procedure that can be improved or refined by the healthcare professional. In many conventional settings, evaluators are instructed to assess a healthcare professional's performance against one or more frameworks that specify standards for the skills or competencies used during the procedure. For example, to evaluate a healthcare professional's performance, an evaluator compares the performance of the medical procedure by the healthcare professional to standards specified by the framework. This evaluation can be used to select training content for the healthcare professional to improve one or more competencies or skills. Similarly, evaluations can be used to assess changes over time in the performance of one or more modalities of the medical procedure by the healthcare professional. However, evaluators are instructed to evaluate the performance of the medical procedure after it has been performed. Therefore, to evaluate the healthcare professional, the evaluator must rely on their own memory of the medical procedure and the healthcare professional's performance. This reliance on the medical professional's memory in evaluating medical procedures increases the difficulty for the evaluating medical professional to identify specific areas requiring improvement. Brief explanation of the drawing

[0007] FIG. 1 is an exemplary embodiment of a computing environment for electronically assisted medical procedures. Figure 2 is a block diagram of an exemplary architecture for a collaborative medical platform. FIG. 3a illustrates a first view of an exemplary worker dashboard associated with a collaborative medical platform. FIG. 3b illustrates a second view of an exemplary worker dashboard associated with a collaborative medical platform. Figure 4 illustrates an exemplary worker dashboard that displays educational content items to medical workers associated with a collaborative medical platform. FIG. 5 is an exemplary embodiment of a case sharing interface associated with sharing medical cases in a collaborative medical platform. FIG. 6 is an exemplary embodiment of a case dashboard associated with a case set in a collaborative medical platform. Figure 7 is an exemplary telepresence interface associated with a collaborative medical platform. Figure 8 is another example of a telepresence interface associated with a collaborative medical platform. Figure 9 is an exemplary analysis dashboard associated with a collaborative medical platform. Figure 10 is an exemplary video interface associated with a collaborative medical platform. Figure 11 is an exemplary evaluation interface associated with a collaborative medical platform. Figure 12 is an exemplary framework interface associated with a collaborative medical platform. Figure 13 is an exemplary evaluator selection interface associated with a collaborative medical platform. Figure 14 is an exemplary segment selection interface associated with a collaborative medical platform. Figure 15 is an exemplary evaluation result interface associated with a collaborative medical platform. FIG. 16 is an exemplary embodiment of a process in which a medical professional obtains an evaluation of a medical procedure performed by an evaluation medical professional through a collaborative medical platform. Specific details for implementing the invention

[0008] The drawings and the following description describe specific embodiments merely by way of example. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be adopted without departing from the principles described herein. Hereinafter, reference will be made to the various embodiments illustrated in the accompanying drawings. Where feasible, similar or identical reference numerals may be used in the drawings and may indicate similar or identical functions.

[0009] A collaborative healthcare platform facilitates the exchange of data between telemedicines during the pre-procedure, intra-procedure, and post-procedure phases in relation to a medical case. The collaborative healthcare platform stores or enables access to patient records, imaging data, video data, telemetry data from medical equipment, biosensor data from patients, and other medical information that may be acquired before, during, or after a medical procedure is performed. Based on data describing the performance of a medical procedure, which may include video data or telemetry data acquired during the performance of the medical procedure, the collaborative healthcare platform facilitates the evaluation of a healthcare professional's performance of the medical procedure. For example, a healthcare professional selects a medical procedure, a framework for evaluating the healthcare professional, an evaluating healthcare professional, and a segment of data describing the performance of the medical procedure. The collaborative healthcare platform provides the segment of data describing the performance of the medical procedure and the framework to the evaluating healthcare professional, who generates the evaluation results for the healthcare professional that the collaborative healthcare platform presents to the healthcare professional. Alternatively, the collaborative medical platform applies one or more evaluation models to a combination of data segments and frameworks describing the performance of medical procedures, and the evaluation models generate a predictive portion of the evaluation results based on the data segments and frameworks describing the performance of the medical platform. Furthermore, the collaborative medical platform can select educational content for medical professionals based on the evaluation results regarding the medical professionals.

[0010] FIG. 1 illustrates an exemplary embodiment of a computing environment (100) for a collaborative medical platform (140). The collaborative medical platform (140) may include one or more servers coupled by a network (130) to a client device (150) associated with a user (155) of the collaborative medical platform (140), medical equipment (160), and various third-party servers (170). The collaborative medical platform (140) facilitates the exchange of collaborative data between medical professionals, patients, administrators, or other users (155) through the client device (150) to support the pre-procedure, in-procedure, and post-procedure phases of a medical case. The collaborative medical platform (140) may also facilitate access to telemetry data (e.g., real-time video, images, biometric detection data, equipment control and / or status signals, etc.) from medical equipment (160) that can be utilized in conjunction with performing medical procedures and managing patient cases. Furthermore, the collaborative medical platform (140) can facilitate access to various third-party servers (170) that provide external services such as electronic healthcare record (EHR) services, medical telepresence services, operating room scheduling, and data analysis services.

[0011] To further support the pre-procedure phase of a medical case, the collaborative medical platform (140) may select one or more reference content items to present to a medical professional before performing a medical procedure. In various embodiments, the collaborative medical platform (140) maintains a repository or library of reference content items from which reference content items for the medical professional are selected. Alternatively or additionally, one or more third-party servers (170) maintain reference content items, and the collaborative medical platform (140) selects one or more reference content items from the third-party servers (170). Reference content items for the medical professional may be retrieved from a combination of one or more third-party servers (170) and the collaborative medical platform (140). The collaborative medical platform (140) selects one or more reference content items for the medical professional by utilizing data in the medical professional's user profile.

[0012] To support the in-procedure phase of a medical case, the collaborative medical platform (140) can facilitate the presentation of various information to support the procedure, such as pre-procedure images, models, patient data, equipment information, or other data. The collaborative medical platform (140) can also facilitate telepresence sessions that allow one or more remote contributors to access videos, images, 3D models, equipment telemetry data, or other data streams captured during the medical procedure in progress. The collaborative medical platform (140) can also enable remote practitioners to provide annotations or other comments related to real-time videos, images, or 3D models associated with the procedure. The collaborative medical platform (140) tracks and stores all data from the procedure (including videos, medical equipment telemetry, and collaborative comments) in association with a case identifier to enable subsequent access.

[0013] During the procedure phase, the collaborative medical platform (140) may present educational content regarding the medical procedure being performed to one or more medical personnel performing the medical procedure. The educational content describes the performance of the medical procedure, such as the techniques to be used, the movement of medical devices or medical equipment, information on the settings for the medical equipment, or other information. During the procedure phase, the collaborative medical platform (140) compares the telemetry data or video data of the medical procedure with a baseline reference associated with the educational content and selects the educational content associated with the baseline reference where a deviation occurs from the telemetry data or video data. The educational content may include commands to modify one or more settings of the medical equipment based on the educational content when executed by the medical equipment (160), thereby simplifying the adjustment of the operation of the medical equipment (160).

[0014] To support the post-procedure phase of a medical procedure, the collaborative medical platform (140) enables medical professionals associated with a case to collaboratively monitor data related to the patient's recovery. For example, the collaborative medical platform (140) can provide an interface for viewing health records related to the patient's recovery and facilitate collaborative exchanges among medical professionals through case-specific content feeds. The collaborative medical platform (140) can also perform various analyses related to the medical procedure performed based on data aggregation. The analyses may be useful for supporting patient recovery, improving future procedures, and tracking the performance of medical professionals.

[0015] Educational content related to medical procedures may be selected by the collaborative medical platform (140) and presented to a medical professional who performed the medical procedure during the post-procedure phase. For example, an indicator or analysis determined for the medical procedure by the collaborative medical platform (140) is compared to a baseline standard for various educational content. In various embodiments, the collaborative medical platform (140) selects educational content associated with the indicator and the baseline standard where deviation occurs and presents the selected educational content to the medical professional. For example, the collaborative medical platform (140) simplifies access to educational information related to medical procedures by including information identifying the selected educational content in one or more interfaces created to present it to the medical professional.

[0016] The collaborative medical platform (140) can intelligently utilize data collected during the pre-procedure, in-procedure, and / or post-procedure phases of a case during different phases of the same case or different cases. For example, annotations of images or 3D models, practitioner comments from content feeds, or other information obtained during the pre-procedure phase may be made available during the in-procedure phase to assist practitioners during the procedure being performed. Analytical data related to post-procedure data may be utilized to generate recommendations for future procedures, such as educational content, to improve efficiency and / or results.

[0017] The collaborative medical platform (140) can also facilitate functions such as managing clinical trials, facilitating training and performance tracking, facilitating the broadcasting of medical-related presentations, and facilitating procedure scheduling. Beneficially, the collaborative medical platform (140) stores a complete record of medical cases (including video and telemetry from procedures) on a centralized, standardized platform that naturally enables collaboration in an online environment where practitioners can interact from different remote locations. The collaborative medical platform (140) can maintain data in a manner that complies with the data privacy and compliance obligations of medical practitioners and organizations.

[0018] The collaborative medical platform (140) may also employ various machine learning techniques to infer recommendations, insights, or other artificially generated contributions based on data collected by the collaborative medical platform (140). For example, the collaborative medical platform (140) may generate recommendations for a medical professional to review educational content related to the medical professional based on data captured by the collaborative medical platform (140) during the performance of a medical procedure. For example, the collaborative medical platform (140) selects educational content for the medical professional based on telemetry data captured during a medical procedure performed by the medical professional. As another example, the collaborative medical platform selects educational content for the medical professional based on video data captured during a medical procedure performed by the medical professional. The educational content selected by the collaborative medical platform may be video, audio, text, or other data describing the performance of the medical procedure. Additionally or alternatively, it may be educational content configuration commands or configuration data for one or more medical devices (160).

[0019] The collaborative medical platform (140) may generate and present other recommendations to the medical worker based on information stored for the medical worker. For example, the collaborative medical platform (140) generates recommendations for the medical worker based on the type of procedure scheduled to be performed by the medical worker. In various embodiments, the recommendations include case records associated with one or more past cases captured in the collaborative medical platform (140) regarding the previous performance of the type of procedure for a patient in a similar situation. If appropriate authorization is granted, the worker may review the entire case record, including pre-procedure information, video or other data from the procedure itself, and post-procedure result data, through the collaborative medical platform (140). As another example, the collaborative medical platform (140) may intelligently generate recommendations inviting a specific medical worker to collaborate on the case based on the relevant worker who possesses relevant expertise, experience, and / or availability. Then, an invitation may be generated to the medical worker to enable access to and collaboration on the case during at least one of the pre-procedure, in-procedure, and post-procedure phases. Additionally, the collaborative medical platform (140) can intelligently identify and present patient risk factors related to the performance, planning, and post-procedure management of procedures. The collaborative medical platform (140) can also intelligently recommend educational content for training medical personnel based on individual tracked performance and various comparative analyses.

[0020] The collaborative medical platform (140) may be implemented using an on-site computing or storage system, a cloud computing or storage system, or a combination thereof, and may be implemented by utilizing a local or cloud-based server that may include a physical machine or a virtual machine or a combination thereof. The cloud-based server may include a private cloud system, a public cloud system, a hybrid public / private cloud system, or a combination thereof. Accordingly, the collaborative medical platform (140) may be local, remote, and / or distributed with respect to the medical environment where the procedure is performed and the client device (150) providing user access. Furthermore, different parts of the collaborative medical platform (140) may run on different remote servers, and various system elements of the collaborative medical platform (140) may be coupled to communicate via a network (130).

[0021] A client device (150) may include, for example, a mobile phone, tablet, laptop or desktop computer, other computing device, or an application running thereon to access the collaborative medical platform (140) via a network (130). The client device (150) may enable access to various user interfaces (which may include a web-based interface accessed via a browser or an application interface accessed via an application) for viewing / viewing or editing information associated with the collaborative medical platform (140). The client device (150) may include conventional computer hardware such as a display, an input device (e.g., a touch screen), memory, a processor, and a non-transient computer-readable storage medium that stores instructions for execution by the processor to perform the functions described herein. Examples of user interfaces are described in more detail below in connection with FIGS. 3 through 15.

[0022] A third-party server (170) can facilitate various services utilized by the collaborative medical platform (140). For example, the third-party server (170) may include various EHR systems for managing patient records, a robot control platform for controlling surgical robots or other medical equipment, a telepresence server for facilitating telepresence services, a patient scheduling system, a hospital information system (HIS), or other servers. As another example, one or more third-party servers (170) may include educational content regarding various medical procedures, such as papers on various medical procedures, audio data related to medical procedures, video data related to medical procedures, settings or configuration details for medical equipment (160) used in medical procedures, or other descriptive information regarding medical procedures. The third-party server (170) may be implemented using a cloud computing or storage system, such as various on-site computing or storage systems, private cloud systems, public cloud systems, hybrid public / private cloud systems, or a combination thereof.

[0023] Medical equipment (160) may include various sensors, such as cameras or other imaging equipment, biometric monitors, or other sensing devices that collect data associated with the medical procedure being performed. Sensor data may include physiological or biological signals (e.g., heart rate, blood pressure, body temperature, etc.), video, electrical signals indicating the status of a medical prescription, or other information. The camera or image sensor may include a still image camera, a video camera, a three-dimensional (3D) imaging device, or a combination thereof. The camera may include a fixed camera within a medical environment (e.g., an operating room) or a camera integrated into a medical device, such as an endoscope camera. The imaging system may include a computed tomography (CT) imaging system, a magnetic resonance imaging (MRI) system, an X-ray system, or other imaging equipment. Medical equipment may also include a robotic device that facilitates robot-assisted medical procedures. The robotic device may include, for example, a robotic arm or other computer-controlled mechanical device that performs or assists in the medical procedure. The robotic device may be pre-programmed to perform a series of specific steps or tasks, or may be manually controlled by an operator. Telemetry data associated with the robotic device may include force data, position data or other sensor data, control signals, error conditions, or other data related to the operation of the robotic device during the procedure. Medical equipment data may be streamed in real time to the collaborative medical platform (140) or stored on a third-party server (170) and then uploaded to the collaborative medical platform (140) later.

[0024] The network (130) includes a communication path for communication between a collaborative medical platform (140), medical equipment (160), a client device (150), and a third-party server (170). The network (130) may include one or more local area networks and / or one or more wide area networks (including the Internet). The network (130) may also include one or more direct wired or wireless connections (e.g., Ethernet, WiFi, cellular protocol, WiFi Direct, Bluetooth, Universal Serial Bus (USB), or other communication links).

[0025] FIG. 2 is a block diagram illustrating an exemplary architecture of one embodiment of a collaborative medical platform (140). In the embodiment of FIG. 2, the collaborative medical platform (140) includes a data collection module (205), an entity management module (210), an interface management module (215), a medical intelligence module (220), a telepresence module (225), an analysis module (230), a staff training module (235), a presentation module (240), an application integration module (245), a video library (250), a connection graph repository (255), a user profile repository (260), and a patient data repository (265). In other embodiments, the collaborative medical platform (140) includes functional blocks that are different from or additional to those shown in FIG. 2. Furthermore, in some embodiments, a single functional block provides the functions of multiple functional blocks shown in FIG. 2.

[0026] In one embodiment, the illustrated function block may be executed entirely within the collaborative medical platform (140), but alternative embodiments may include various modules or individual functions of modules executed by one or more third-party servers (170). Here, the collaborative medical platform (140) may interact with the third-party server (170) via an application programming interface (API) to enable the collaborative medical platform (140) to request and utilize services provided by the third-party server (170) to facilitate any of the functions described herein. For example, in one embodiment, electronic medical records may be provided by the third-party server (170). Here, the collaborative medical platform (140) may query the third-party server (170) for relevant data but does not necessarily store complete patient records locally. Furthermore, the third-party server (170) may facilitate services such as telepresence sessions, presentation creation, access to video resources, 3D model creation, or other aspects of the functions of the collaborative medical platform (140) described herein.

[0027] The data collection module (205) collects various medical data used by the collaborative medical platform (140). The data collection module (205) may be electronically coupled to one or more external servers, databases, or other data sources that supply medical data. The medical data may include, for example, profile data about patients (e.g., demographic information, health records, etc.), medical professionals (e.g., expertise, experience, etc.), or facilities; information about medical conditions, procedures, and medications; information about robotic systems, imaging systems, intervention tools, or other medical equipment; information about post-procedure outcomes; or other medical information discussed herein.

[0028] The data collection module (205) can aggregate data from various input data sources. For example, the data collection module (205) can acquire medical data from a conventional electronic medical record (EHR) system. Here, the data collection module (205) can perform various preprocessing to normalize the data into a standardized format used by the collaborative medical platform (140). For example, the medical record may be composed of a database structure containing values ​​(strings, numeric values, binary values, or other data types) assigned to each of the predefined sets of information fields.

[0029] The data collection module (205) may also interface with one or more imaging systems to collect images, videos, or three-dimensional models associated with the patient before, during, or after the procedure. For example, the data collection module (205) may acquire and store X-ray images, magnetic resonance imaging (MRI) images, computed tomography (CT) scan images, visible light images, near-infrared fluorescence (NIRF) images, or other medical images, videos, or three-dimensional models derived therefrom. Image data may also include image or video data from one or more cameras present in the medical environment where the medical procedure is being performed, such as one or more overhead cameras and / or one or more endoscopic cameras. Imaging data may include telemetry data from one or more medical devices used to perform the medical procedure, annotations or comments associated with the video received from one or more medical personnel associated with the medical procedure, segmentation data or images associated with dividing the video into segments associated with different stages of the procedure, or other information associated with the video data.

[0030] To simplify subsequent search and review of medical procedure videos and associated metadata, the data collection module (205) can perform various preprocessing and indexing on the content and associated metadata. For example, the data collection module (205) indexes the video of a medical procedure with associated metadata to correlate different metadata with different parts of the video, synchronize videos associated with the same medical procedure, or perform various encoding or reformatting of the video data. The video can also be automatically divided and indexed into video segments corresponding to different stages of the procedure.

[0031] The data collection module (205) can also be integrated with various robot platforms or other medical equipment to acquire telemetry data associated with the procedure. For example, the data collection module (205) can acquire various sensor data from sensors utilized during the medical procedure to identify information associated with the medical equipment, control data associated with controlling the robot platform or other medical equipment, or other data generated from the medical equipment in association with the medical procedure performed.

[0032] The data collection module (205) may also provide an interface accessible through a client device (150) to collect data that is directly input into the collaborative medical platform (140). For example, the data collection module (205) may present various forms or free-form input elements to enable the input of medical information related to operation.

[0033] In one embodiment, the data collection module (205) may manage data in a manner consistent with various compliance and privacy policies. For example, the data collection module (205) may enable the removal or correction of parts of the received data to protect patient privacy when the data is used for purposes where patient identification is not required.

[0034] The entity management module (210) manages the presentation of entity pages associated with different entities linked to the collaborative medical platform (140) and manages connections between entities. Entities may include, for example, users (155) (which may include medical professionals, patients, administrators, etc.), medical cases associated with procedures, facilities, medical equipment (160), files or media content, events (e.g., conferences), presentations, training modules, or other data objects. Entity pages may include web pages accessible through a web browser of a client device (150) or pages of a desktop or mobile application installed on the client device (150).

[0035] Each entity page for an entity may allow viewing of information associated with the entity and / or interactions with the entity. For example, each user (155) of the collaborative medical platform may have a dedicated page providing information about the user (155), such as identification information, role (e.g., surgeon, nurse, executive, manager, patient, etc.), profile information (e.g., history, credentials, etc.), assigned cases, procedure history, connections with other users or cases, scheduling information, or other user-specific data. An entity page for a patient (regardless of whether the patient is a user (155) of the collaborative medical platform (140) may include patient profile information, health history, planned procedures, risk factors, or medical information associated with the patient. An entity page for a medical case may include information about the patient associated with the case, descriptive information about the medical procedure associated with the case (e.g., a type of medical procedure), the medical environment in which the medical procedure will be performed, other descriptive information about the medical procedure, the status of the procedure (e.g., pre-procedure, in-procedure, or post-procedure phases), or other information related to the medical case. The page may also include various interactive elements (e.g., content feeds) that allow users to share and interact with data associated with the entity, as described in more detail below.

[0036] The entity management module (210) also organizes pages and associated data received by the collaborative medical platform (140) into a connection graph (stored in the connection graph repository (255)) that captures the relationships between data associated with different entities. Some connections may be configured as default connections, while others may be created based on specific actions by a user (155). For example, a user (155) may be connected by default with another user (155) within the same organization (at least having view permissions). Alternatively, a connection may be created only when the user (155) explicitly invites another user (155) to connect and the receiving user (155) accepts the connection request. Connections between medical practitioners and medical cases may similarly be created by default or in response to an invitation to create a connection. For example, a default connection may be created between an entity for a planned medical procedure and a medical practitioner assigned responsibility for the procedure. Alternatively, all medical practitioners within an organization or related department may be connected to the planned procedure by default. In another scenario, the user may create a connection request to invite other medical professionals to collaborate on the medical case by sharing the medical case with one or more other medical professionals. Then, acceptance of the connection request may create a connection between the invited professionals and the medical case. Additional connections may also be created automatically (e.g., between the procedure owner and the invited contributor). Connections may also be created between the user (155) and individual videos, files, presentations, or other data objects. For example, the user (155) who creates or owns a video may share the video with one or more other users (155) to grant access to the video.

[0037] Connections between entities can have various types and may be governed by different permissions. Generally, pages may only be accessed by users with appropriate access permissions. Different permission levels may dictate distinct levels of access to different pages. For example, depending on user permissions for a specific page, users may be allowed or blocked from accessing data, editing data, commenting on or annotating data, deleting data, or performing other modifications. In one embodiment, a page may have a page owner with the highest level of access permissions. Typically, a medical professional may be the owner of their profile page and the pages for procedures for which they have primary responsibility. Pages associated with facilities, medical equipment, or other entities may be variably owned by assigned medical professionals. Non-owners may have distinct levels of access to pages based on configured permissions. Permissions may be granted by other users with appropriate permissions to assign or grant permissions to the page owner or other users.

[0038] Based on different connections available to different users (155), the collaborative medical platform (140) enables a personalized experience for each user (155). For example, when logging into the collaborative medical platform (140), the user (155) may be presented with a personalized interface related to connections to other users (155), medical cases, videos, presentations, or other content hosted by the collaborative medical platform (140).

[0039] The interface management module (215) manages content associated with various interfaces hosted by the collaborative medical platform (140) and accessible through a client device (150). As described above, the interface management module (215) may manage pages associated with various entities managed by the collaborative medical platform (140), including users (155) (which may include medical professionals, patients, administrators, etc.), medical cases associated with procedures, facilities, medical equipment (160), files or media content, events (e.g., conferences), presentations, training modules, or other data objects. Access to different pages by a specific user (155) may depend on the user's connection and permissions configured in the connection graph repository (255). Additionally, patient data may be pseudonymized for viewing by other specific users (depending on connection type and / or permissions) so that patient data cannot be attributed to a specific individual.

[0040] A medical case page associated with a medical case may contain information organized into pre-procedure, in-procedure, and post-procedure phases. In the pre-procedure phase, the medical case page may contain information about the patient, the procedure being performed, and the medical professional performing the procedure. The interface management module (215) may also provide access to various analytical information (e.g., generated by the analysis module (230) described below), such as risk factors for the patient, the experience / expertise of the medical professional, results for the planned type of procedure, or other data. In the in-procedure phase, the medical case page may provide access to a telepresence session that allows a remote collaborator to collaborate remotely regarding the procedure in progress. In the post-procedure phase, the medical case page may contain information about the patient's treatment plan, risk factors, follow-up visits, or other post-procedure information.

[0041] Some entity pages in the collaborative medical platform (140) may include a content feed to facilitate collaboration among users (155). The content feed may include various content (e.g., posts), such as text-based commentary, images, videos, 3D models, or other multimedia content related to medical cases. Content may be posted directly to pages associated with medical cases, or posts may include links to content stored by the collaborative medical platform (140) or on an external server. Posts may be grouped into conversations that hierarchically track the relationships between posts. For example, posts may be written as original posts (starting a new conversation) or as replies to existing posts (becoming part of a conversation).

[0042] In one exemplary use, a user (155) may invite one or more other users (155) to collaborate on a medical case and access the case page for the medical case. The content feed on the page allows the collaborating users (155) to post to the case page in relation to the medical case. Thus, the content feed may enable discussions on the procedure to be performed, risks, best practices, or other information that may be useful to the practitioner performing the procedure. Additionally, the participating users (155) may post videos or 3D models (or links to content) related to past procedures on patients in similar situations. Furthermore, the participating users (155) may share links to entity pages related to past procedures that may be relevant, allowing the medical practitioner performing the procedure to view past content feeds related to that procedure. Patient data may be optionally pseudonymized when shared with other users (depending on the connection type and / or permissions) so that patient data cannot be attributed to a specific individual.

[0043] The content feed can also be utilized in relation to the procedure in progress during a real-time telepresence session, as discussed in more detail below. Here, the content feed may be presented as a real-time chat window that allows a contributor to comment during the procedure, share videos, images, or other media, provide links to relevant resources, or provide content during the procedure.

[0044] In the post-procedure phase, the content feed may be utilized by contributors to discuss post-procedure treatment, patient recovery, risk management, or other information related to patient recovery. An example of a content feed is provided in Fig. 7, which is described in more detail below.

[0045] The medical intelligence module (220) generates medical intelligence data that may be automatically added to a content feed or otherwise made available in the context of a collaborative medical platform (140). For example, the medical intelligence module (220) may automatically contribute posts to a content feed regarding medical cases that an artificial intelligence agent infers are relevant. Artificially generated posts may mimic posts provided by human contributors and may include text-based commentary, multimedia, links, etc. Medical intelligence data may be generated during the pre-procedure phase, during the procedure, or during the post-procedure phase.

[0046] In one exemplary embodiment, the medical intelligence module (220) may include one or more machine learning models trained to generate content that the model infers is relevant to a specific medical case or, more generally, to the user (155). In one embodiment, the machine learning model generates an embedding for the medical case based on descriptive information about the medical procedure, characteristics of the patient to whom the medical procedure is to be performed, characteristics of the medical professional performing the procedure, posts within the content feed, or other information available on the collaborative medical platform (140). The medical intelligence module (220) determines a measure of similarity (e.g., cosine similarity, inner product) between the embedding for the medical case and the embedding for other content available on the collaborative medical platform (140) and which may be included in the automated posts. Then, the medical intelligence module (220) may generate posts and / or select content for posts based on the similarity of the embeddings. The medical intelligence module (220) may also employ various large-scale language models (LLMs) to analyze text-based content associated with medical cases and artificially generate relevant natural language content for the content feed. The machine learning model may also include one or more neural networks (e.g., convolutional neural networks (CNN), artificial neural networks (ANN), resentential neural networks (ResNet) or recurrent neural networks (RNN)), regression-based models, generative models, or other types of machine learning models capable of achieving the functions described herein.

[0047] In one exemplary use case, the medical intelligence module (220) can identify one or more past medical cases similar to the current medical case and automatically generate a link to a case page for the relevant cases. Then, the medical professional can view videos, models, or other recorded data associated with the relevant medical case to help prepare for the procedure. In another example, the medical intelligence module (220) can automatically respond to questions raised by the user in the content feed. For example, the medical intelligence module (220) can function like a chatbot that intelligently responds to text-based queries. In a further embodiment, the medical intelligence module (220) can generate recommendations inviting specific medical professionals to collaborate on a medical case based on relevant expertise and experience. Then, the user can choose whether to invite the recommended collaborators to collaborate on the medical case based on the artificially generated recommendations.

[0048] The telepresence module (225) facilitates a telepresence session during a procedure. The telepresence session may include one or more collaborators invited to collaborate on a medical case, and allows other users (155) to remotely access video captured during the medical procedure, telemetry data from one or more medical devices, or other real-time data. As described above, a content feed is also displayed in association with the telepresence session, allowing contributors to comment on or share multimedia or links related to the procedure.

[0049] The telepresence module (225) may also enable the contributor to provide real-time annotations on images, videos, 3D models, or other visual content of anatomical structures related to the ongoing procedure. For example, the contributor may mark locations on the visual content in conjunction with provided comments. The telepresence module (225) may also enable the contributor to add overlaid drawings, highlights, or other visual markings during the ongoing telepresence session.

[0050] In one embodiment, the telepresence module (225) may enable a remote contributor to control the medical equipment (160). For example, the remote contributor may access a control interface that provides control elements for controlling the position or orientation of a camera, controlling a robot arm, setting the configuration of a sensing device, or performing other control functions of the medical equipment.

[0051] When the procedure is completed, the telepresence module (225) can store recorded video, telemetry data, content feeds, annotations, and other captured data associated with the procedure. This information may later be accessed by a user (155) of the collaborative medical platform (140) (if appropriate permissions are granted) and / or utilized by the medical intelligence module (220) to further train machine learning models and / or generate inferences.

[0052] The analysis module (230) facilitates the generation of various statistics, indicators, or other analyses associated with information stored in the collaborative medical platform (140). The analysis can generally be generated based on a set of filtering parameters that yield a subset of data records for aggregation, and a combination function that specifies how the filtered data should be combined. The filtering parameters can filter medical procedure data based on data fields such as patient data, medical worker data, facility, type of procedure, medical equipment used, etc. The combination function may include, for example, an average function, a median function, a histogram function, or other functions. A specific analysis function may provide a single output value or a series of values ​​across one or more dimensions. The series of outputs may be visually presented as a table, chart, graph, or other visual output.

[0053] For example, the analysis module (230) may generate an indicator describing the average length of time required to complete a medical procedure for a specific medical worker or group of medical workers. The average time for various procedures performed by the same medical worker or group of medical workers may be presented along with similar indicators for other medical workers for comparison purposes. In another example, the analysis module (230) may generate an indicator describing the number of times a medical worker has performed a specific type of medical procedure in the past. Such counts may be further aggregated to represent a percentage reflecting the number of times a medical worker performed a different type of medical procedure out of the total number of procedures performed.

[0054] In an additional embodiment, the analysis module (230) may generate an analysis based on interactions of medical professionals on the collaborative medical platform (140). For example, statistics may be derived based on the count of posts, comments, or other content contributed by medical professionals to the collaborative medical platform (140). Such an analysis may be expressed in terms of interaction counts, interaction frequency, or other aggregations. Such an analysis may also be aggregated separately based on whether the interaction is related to the pre-procedure, in-procedure, or post-procedure phases.

[0055] In one embodiment, the analysis module (230) may generate an analysis based on a specific filtering and / or combining function specified by a user (155) of the collaborative medical platform (140). Additionally, the analysis module (230) may include various preset analyses that can be generated without necessarily receiving specific user input. Furthermore, in some embodiments, the medical intelligence module (220) may automatically generate an analysis that is inferred to be relevant to a specific user (155).

[0056] In some embodiments, the analysis module (230) may generate an analysis based on any aspect of collective case data including pre-procedure data, telepresence session data (including recorded video, telemetry data, in-session content feed data, etc.), and post-procedure data. The analysis associated with the telepresence session data may include performing various video processing, content recognition, or other advanced image processing techniques to extract useful information from the video. Additionally, the analysis module (230) may generate an analysis by utilizing various medical intelligence data generated from the medical intelligence module (220).

[0057] The practitioner training module (235) manages and stores training data for medical practitioners associated with medical procedures. In various embodiments, the training data includes educational content containing descriptive information about a medical procedure or a part of a medical procedure. Exemplary educational content includes training videos related to performing a medical procedure, papers on performing a medical procedure, papers or videos on using one or more medical devices (160) in a medical procedure, papers or videos on using one or more medical devices in a medical procedure, best practices for a medical procedure, training manuals for a medical procedure, educational materials on one or more medical devices used in a medical procedure, digital training modules, webinars, audio data on a medical procedure (e.g., a podcast about a medical procedure), or other information for training medical practitioners in relation to a medical procedure.

[0058] In various embodiments, the training content also includes configuration data or configuration commands for one or more medical devices (160) used in one or more medical procedures. For example, the training content includes a set of configuration commands for configuring or calibrating a robotic arm or other medical devices (160) for use in a medical procedure. The configuration commands may include one or more settings for the medical devices (160). Exemplary settings include: one or more limit values ​​for the amount of force applied by the medical devices (160), one or more limit values ​​for the range of motion of the medical devices (160), one or more limit values ​​for the amount of energy supplied by the medical devices (160), an operating mode identifier for the medical devices (160), or a value for one or more other settings of the medical devices (160). As another example, the training content includes a set of commands that, when executed by the medical devices (160), cause the medical devices (160) to perform a series of actions for calibration. Specific educational content can be executed by medical equipment (160) to modify one or more setting values ​​of medical equipment (160) or the operating mode of medical equipment (160), thereby enabling automatic modification of one or more settings of medical equipment (160) through educational content items without medical personnel having to manually specify the setting values ​​of medical equipment (160).

[0059] The practitioner training module (235) stores training content as different training content items, each training content item containing individual parts of content such as files. Each training content item has one or more attributes that provide descriptive information about the training content item. For example, the attributes of the training content item identify one or more types of medical procedures associated with the training content item, thereby enabling the identification of training content items corresponding to different types of medical procedures. Other exemplary attributes of the training content item include: one or more medical practitioners associated with the training content item (e.g., a medical practitioner who performed the medical procedure associated with the training content item, a medical practitioner who created the training content item), a location associated with the training content item (e.g., a geographic location, a specific medical facility), a time associated with the training content item (e.g., the time the training content item was created), an identifier of one or more medical equipment (160) associated with the training content item, an identifier of one or more medical devices used in the medical procedure associated with the training content item, the format of the training content item (e.g., audio, video, text), or other information describing the training content item. In various embodiments, educational content items may be stored locally by the collaborative medical platform (140) (e.g., in a video library (250) or other storage device) or retrieved from one or more third-party servers (170).

[0060] One or more educational content items may include a reference case, which is a medical case selected by a medical professional who performed a medical procedure completed in a medical case to make available to other medical professionals. For the reference case, the professional training module (235) stores video data, telemetry data from medical equipment (160), or other data captured by the collaborative medical platform (140) during the performance of the medical procedure completed. In various embodiments, the reference case includes a content feed containing comments or other data obtained from contributors by the collaborative medical platform (140) during the medical procedure completed. For the reference case, the professional training module (235) pseudonyms the patient data to prevent the reference case from containing patient data that may be attributed to a specific patient. In some embodiments, the pseudonymized patient data in the reference case identifies a range for one or more open data types to prevent identification of the specific patient for whom the medical procedure completed was performed, while maintaining relevant information about the patient for whom the medical procedure completed was performed for other medical professionals.

[0061] Each educational content item is associated with one or more baseline criteria. Different baseline criteria specify values ​​for indicators from performing a medical procedure, settings of medical equipment (160) used in the medical procedure, movement patterns of medical equipment (160) during the medical procedure, patterns of telemetry data acquired during the medical procedure, movement patterns of medical personnel during the medical procedure, or other descriptive information regarding the performance of the medical procedure. Baseline criteria specify standardized values ​​for indicators, standardized techniques or approaches used in the medical procedure, or other standardized values ​​or techniques associated with the medical procedure. The practitioner training module (235) maintains one or more baseline criteria for different medical procedures, so that different educational content items correspond to different medical procedures. The attributes of an educational content item include an identifier of a corresponding type of medical procedure to indicate that the educational content item and its associated baseline criteria correspond to a type of procedure. This enables the practitioner training module (235) to identify different baseline criteria for different types of medical procedures.

[0062] In various embodiments, one or more medical professionals input baseline criteria for medical procedures into the professional training module (235). For example, a group of medical professionals reaches an agreement on the value of an indicator, the pattern of telemetry data, the pattern of movement, or the value of other information describing the performance of the medical procedure. The medical professionals in the group input the agreed baseline criteria into the professional training module (235) to store them associated with training content items. The group of medical professionals may be associated with a specific medical facility (e.g., a hospital, a clinic) to provide facility-specific baseline criteria. The professional training module (235) stores the identifier of the medical facility as an attribute of the training content item associated with the facility-specific baseline criteria to indicate the baseline criteria associated with the specific medical facility. Additionally or alternatively, the group of medical professionals who determined the baseline criteria may not be associated with a specific medical facility but may belong to a larger organization or standard body, and thus the baseline criteria for medical procedures may be applicable across various medical facilities. The employee training module (235) may store facility-specific baseline standards and more generally applicable baseline standards as different attributes of training content items in various embodiments. This allows the more generally applicable baseline standards associated with the training content items to be reinforced with facility-specific baseline standards.

[0063] In various embodiments, the practitioner training module (235) applies one or more trained machine learning models to indicators generated by the analysis module (235) for multiple medical cases in which a type of medical procedure is performed, thereby generating one or more baseline criteria associated with educational content items. In various embodiments, one or more trained machine learning models are also applied to telemetry data or video data captured by the telepresence module (225) during the medical cases in which the corresponding type of medical procedure is performed. For example, the machine learning model detects patterns in telemetry data captured during a specific type of medical procedure occurring in a medical case where a specific value of the generated indicator is generated or the value of the generated indicator is within a value range. The specific value of the generated indicator or the value range of the generated indicator may correspond to one or more specific patient outcomes. For example, the specific value or value range identifies successful patient outcomes for the corresponding type of medical procedure. One or more patterns in telemetry data detected at least at a critical frequency in medical procedures occurring in medical cases where the generated indicator has a specific value or a value within a specified range are stored in various embodiments as baseline criteria for educational content items associated with the specific type of medical procedure.

[0064] For example, applying a machine learning model to telemetry data identifies a specific movement sequence of medical equipment (160) detected at least at a critical frequency in a specific type of completed medical procedure performed in a medical case having an indicator corresponding to a positive outcome, which is stored as a baseline reference for an educational content item corresponding to the movement of medical equipment (160) for a specific type of medical procedure. Telemetry data describing the specific movement sequence of medical equipment (160) may be stored in an educational content item to specify limits on the movement of medical equipment (160) during a specific type of medical procedure or limits on the force applied by medical equipment (160) during a specific type of medical procedure. As another example, captured telemetry data includes location data for medical equipment (160) during the occurrence of a corresponding type of medical procedure in a medical case having one or more indicators associated with a positive outcome for a patient. The practitioner training module (235) stores an educational content item associated with the corresponding type of medical procedure having location data in the captured telemetry data as a baseline reference. This simplifies the creation of educational content items for various medical procedures by enabling the worker training module (235) to dynamically generate educational content items and associated baseline criteria for a type of medical procedure based on telemetry data captured during the performance of a type of medical procedure over time.

[0065] In another example of generating an educational content item from telemetry data, telemetry data from medical equipment includes the two-hand manipulation proficiency of a medical worker during a medical procedure, and the two-hand manipulation proficiency information is stored as a baseline reference in the educational content item in response to a determination that the medical procedure has a positive outcome. In various embodiments, the generated educational content item includes data for accessing a simulator for the medical equipment (160) used during the medical procedure (e.g., an identifier of the simulator, one or more exercises or skills to be performed on the identified simulator, etc.), thereby further improving the use of the medical equipment (160) in response to the telemetry data from the medical worker during the medical procedure including two-hand manipulation proficiency information that deviates by at least a threshold amount from the baseline reference of the educational content item. As another example, telemetry data from medical equipment (160) includes tissue tension on a patient during the medical procedure, and the tissue tension is stored as a baseline reference in the educational content item in response to a determination that the medical procedure has a positive outcome by the worker education module (235). The generated educational content item may include data for accessing a simulator for medical equipment (160) used during a medical procedure (e.g., an identifier of the simulator, one or more exercises or skills to be performed on the identified simulator, etc.), thereby further improving the use of the medical equipment (160) in response to telemetry data from a medical worker during a medical procedure containing two-handed operation proficiency information that deviates from the baseline standard of the educational content item by at least a threshold amount.

[0066] Additionally or alternatively, the practitioner training module (235) applies one or more machine learning models to video data captured during the performance of a specific type of medical procedure during a previous medical case to identify different medical equipment (160) used during the specific type of medical procedure, movement of different medical equipment (160) during the specific type of medical procedure, movement of the medical practitioner performing the specific type of medical procedure, or other information regarding the performance of the specific type of medical procedure. As described in more detail above, applying a machine learning model to video data of a previous performance of a specific type of medical procedure detects movement patterns of the medical practitioner or medical equipment (160) during the performance of the specific type of medical procedure. Patterns or movements detected at least at a threshold frequency in video data of a specific type of completed medical procedure performed in a medical case having an indicator corresponding to a positive outcome are stored as baseline criteria for one or more training content items associated with the specific type of medical procedure. Such training content items associated with baseline criteria describing the medical procedure and movement patterns of that type include location data or other data describing the movement or positioning of the medical practitioner or medical equipment (160) during the specific type of medical procedure for subsequent reference. Other information, such as depth perception data, proximity of the medical equipment (160) to the patient's structure, angle of incision of the patient's structure by the medical equipment (160), path length of the medical equipment (160), tissue tension, proficiency of the medical worker's two-hand operation, or other data, can be determined from video data by the worker training module (235) and is stored as a baseline reference in response to being determined from video data of the medical procedure at a threshold frequency or in response to being determined from video data of the medical procedure having an indicator corresponding to a positive result.This allows the worker training module (235) to determine a baseline standard for one type of medical procedure based on video data of one or more medical procedures.

[0067] To select an educational content item for a medical worker, the worker education module (235) compares data describing the performance of a medical procedure performed by a medical worker with baseline criteria associated with various educational content items. In various embodiments, after a medical worker completes a medical procedure, the worker education module (235) identifies an educational content item associated with a type of medical procedure and compares acquired information describing the medical procedure with one or more baseline criteria associated with the identified educational content item. For example, the worker education module (235) compares an indicator generated for the medical procedure by the analysis module (230) from captured telemetry data, video data, or other data with baseline criteria associated with the educational content item associated with that type of medical procedure. In response to the indicator differing by at least a threshold amount from the baseline criteria associated with the educational content item, or failing to meet the baseline criteria, the worker education module (235) selects an educational content item associated with the baseline criteria to present to the medical worker. For example, in response to a determination that the amount of time a medical worker takes to complete a medical procedure exceeds the average time taken to complete that type of medical procedure or exceeds the baseline time taken to complete that type of medical procedure, one or more educational content items are selected that are associated with that type of medical procedure and associated with a baseline standard that specifies the amount of time taken to complete that type of medical procedure. In some embodiments, the worker training module (235) selects one or more educational content items associated with a baseline standard that is associated with a type of medical procedure and where the indicator determined for the medical procedure differs by at least a threshold amount, so that the worker training module (235) can consider a specific amount of difference between the determined indicator and the baseline standard when selecting the educational content items.

[0068] Alternatively or additionally, the practitioner training module (235) compares one or more patterns or data detected within telemetry data captured during the performance of a medical procedure with a baseline reference associated with a training content item. The practitioner training module (235) selects a training content item associated with a baseline reference that specifies a pattern of telemetry data that is associated with the type of medical procedure and differs from the captured telemetry data by at least a threshold amount. For example, the telemetry data includes depth perception data during the medical procedure, and the practitioner training module (235) selects a training content item associated with depth perception data that is associated with the type of medical procedure and differs from the captured depth perception data by at least a threshold amount. In some embodiments, the selected training content item includes information for accessing a simulator for medical equipment (160) to be accessed by a medical practitioner (e.g., an identifier of the simulator, one or more exercises or skills to be performed on the identified simulator, etc.). In another example, telemetry data includes tissue tension data captured during a medical procedure, and the practitioner training module (235) selects a training content item associated with tissue tension data that is associated with the type of medical procedure and differs from the captured tissue tension data by at least a threshold amount. The selected training content item includes information for accessing a simulator for medical equipment (160) to be accessed by a medical practitioner in various embodiments (e.g., an identifier of the simulator, one or more exercises or skills to be performed on the identified simulator, etc.).

[0069] Telemetry data from one or more sensors (e.g., sensors included in medical equipment (160)) may also describe the movement or positioning of medical equipment (160) or medical devices during a medical procedure, and the practitioner training module (235) selects a training content item that includes a baseline criterion in which the movement of medical equipment or the positioning of medical devices in the telemetry data deviates by at least a threshold amount. For example, the telemetry data includes the path length of medical equipment (160) during a medical procedure, and the practitioner training module (235) selects a training content item that includes a baseline criterion in which the path length of medical equipment (160) in the captured telemetry data associated with that type of medical procedure deviates by at least a threshold amount. In another example, telemetry data includes location data of medical equipment (160) during a medical procedure, and the practitioner training module (235) selects an educational content item that includes a baseline criterion specifying the location data of medical equipment (160) in the captured telemetry data associated with the type of medical procedure and the location data of medical equipment (160) that deviates by at least a threshold amount. In one additional example, telemetry data includes two-handed manipulation proficiency data of a medical practitioner during a medical procedure, and the practitioner training module (235) selects an educational content item that includes a baseline criterion specifying the two-handed manipulation proficiency data of medical equipment (160) that deviates by at least a threshold amount from the two-handed manipulation proficiency data in the captured telemetry data associated with the type of medical procedure. In the preceding examples, the educational content item selected based on the telemetry data includes information for accessing a simulator associated with the medical equipment (160) corresponding to the telemetry data (e.g., via a collaborative medical platform (140)), thereby providing the medical practitioner with increased interaction with the medical equipment (160).Furthermore, the training content item selected based on the deviation from the baseline reference of the location data of the medical equipment (160) may include one or more of the following: a training video associated with the medical equipment (160) and explaining the operation of the medical equipment; audio data explaining the operation of the medical equipment (160); and information for accessing a simulator for the medical equipment (160) (e.g., an identifier of the simulator, one or more exercises or skills to be performed on the identified simulator, etc.). In some embodiments, the captured telemetry data describes the usage pattern of the medical equipment (160) during a medical procedure. In response to determining that the usage pattern of the medical equipment (160) deviates from the baseline usage pattern of the medical equipment in the training content item, the practitioner training module (235) selects a training content item that may include benchmarking data describing the cost of the medical procedure based on the usage pattern and information on alternative usage patterns of the medical equipment (160) to reduce costs, or information describing the recommended usage pattern of the medical equipment (160) or the use of alternative medical equipment (160) in that type of medical procedure.

[0070] In various embodiments, the practitioner training module (235) compares one or more data identified from video data captured during the performance of a medical procedure with a baseline reference associated with the training content item to select one or more training content items for a medical practitioner. The practitioner training module (235) selects a training content item associated with a baseline reference that specifies specific data associated with the type of medical procedure and differs by at least a threshold amount from the data identified from the captured video. For example, the practitioner training module (235) obtains depth perception data from video data of the medical procedure during the medical procedure and selects a training content item associated with depth perception data associated with the type of medical procedure and differs by at least a threshold amount from the depth perception data determined from video data of the medical procedure. In some embodiments, the selected training content item includes information for accessing a simulator for medical equipment (160) to be accessed by the medical practitioner (e.g., an identifier of the simulator, one or more exercises or skills to be performed on the identified simulator, etc.). As another example, the practitioner training module (235) determines tissue tension data during a medical procedure and selects an educational content item associated with tissue tension data that is associated with the type of medical procedure and differs from the tissue tension data determined from video data by at least a threshold amount. The selected educational content item includes information for accessing a simulator for medical equipment (160) to be accessed by a medical practitioner in various embodiments (e.g., an identifier of the simulator, one or more exercises or skills to be performed on the identified simulator, etc.).

[0071] The worker training module (235) determines the movement or positioning of medical equipment (160) or medical devices during a medical procedure from video data of a medical procedure through one or more computer vision models or other models in various embodiments. Based on the movement or positioning information obtained from the video data, the worker training module (235) selects a training content item that includes a baseline criterion that deviates by at least a threshold amount from the movement or positioning of medical equipment or medical devices in the telemetry data. For example, the worker training module (235) determines the path length of the medical equipment (160) during a medical procedure from the video data of the medical procedure, and the worker training module (235) selects a training content item that includes a baseline criterion that specifies the path length of the medical equipment (160) associated with the type of medical procedure and deviates by at least a threshold amount from the path length in the video data. In another example, the worker training module (235) determines location data of medical equipment (160) during a medical procedure from a video of a medical procedure, and the worker training module (235) selects a training content item that includes a baseline criterion for specifying location data of medical equipment (160) associated with the type of medical procedure and having a deviation of at least a threshold amount from the location data of medical equipment (160) from the video data. In one additional example, the worker training module (235) determines two-handed operation proficiency data of a medical worker during a medical procedure from video data, and the worker training module (235) selects a training content item that includes a baseline criterion for specifying two-handed operation proficiency data of medical equipment (160) associated with the type of medical procedure and having a deviation of at least a threshold amount from the two-handed operation proficiency data determined from the video data.In the preceding examples, the training content item selected based on telemetry data includes information for accessing a simulator associated with the medical equipment (160) corresponding to the telemetry data (e.g., an identifier of the simulator, one or more exercises or skills to be performed on the identified simulator, etc.), thereby providing the medical worker with increased interaction with the medical equipment. Furthermore, the training content item selected based on the deviation of the location data of the medical equipment (160) from a baseline reference may include one or more of the following: a training video associated with the medical equipment (160) and explaining the operation of the medical equipment, audio data explaining the operation of the medical equipment (160), and information for accessing a simulator for the medical equipment (160) (e.g., an identifier of the simulator, one or more exercises or skills to be performed on the identified simulator, etc.).

[0072] In some embodiments, the practitioner training module (235) determines from video data of a medical procedure the angle at which a patient’s structure (e.g., a patient’s organ) is incised by medical equipment (160) (or medical device) during the medical procedure. The practitioner training module (235) selects an educational content item that includes an angle for incising a patient’s structure associated with the type of medical procedure and which has a deviation of at least a threshold amount from the angle determined from the video data of the medical procedure. The educational content item selected based on the determined angle of incision of the patient’s structure from the baseline angle of incision may include content describing the use of medical equipment (160) (or medical device) for incising the patient’s structure during the medical procedure or content describing the correlation between the angle of incision of the patient’s structure and one or more medical procedure outcomes (e.g., information describing the correlation between a specific angle of incision of the patient’s structure and a positive medical procedure outcome or the angle of incision of the patient’s structure and a negative medical procedure outcome). As another example, the practitioner training module (235) determines the proximity of medical equipment (160) (or medical device) to one or more critical structures of a patient (e.g., organs, bones, arteries) during a medical procedure from video data of a medical procedure. The practitioner training module (235) selects a training content item that includes the proximity of medical equipment (160) (or medical device) from video data associated with the type of medical procedure and baseline proximity of medical equipment (160) (or medical device) to critical structures of the patient that have deviated by at least a threshold amount.In various embodiments, an educational content item having baseline proximity to a patient's vital structures includes content explaining the use of an energy device during a medical procedure, which may include interactive content (e.g., content with questions to be answered by a medical professional), video or audio content explaining the use of an energy device during a medical procedure, or other explanatory information regarding the use of an energy device during a medical procedure.

[0073] Furthermore, the practitioner training module (235) can determine the usage pattern of the medical equipment (160) during the medical procedure from video data of the medical procedure. In response to determining that the usage pattern of the medical equipment (160) deviates from the baseline usage pattern of the medical equipment in the training content item, the practitioner training module (235) selects a training content item. The selected training content item may include benchmarking data describing the cost of the medical procedure based on the usage pattern and information on alternative usage patterns of the medical equipment (160) to reduce costs. As another example, the selected training content item may include information describing the recommended usage pattern of the medical equipment (160) or the use of alternative medical equipment (160) in that type of medical procedure.

[0074] As another example, the worker training module (235) compares a pattern of one or more movements (e.g., movement of medical equipment (160), movement of a part of a medical worker) detected within video data captured during a medical procedure with a baseline reference that includes a movement pattern for that type of medical procedure, and selects an educational content item for a medical worker associated with a baseline reference that specifies a movement pattern in which the detected movement pattern deviates by at least a threshold amount. Thus, the worker training module (235) can determine when to select an educational content item for a medical worker using data captured during the performance of a medical procedure (e.g., video data or telemetry data). Different detected patterns within the telemetry data or video data captured during the performance of a medical procedure can be compared with different educational content items associated with different baseline references. This enables the medical worker training module (235) to select educational content items for medical workers based on a corresponding baseline standard and a specific part of the medical procedure where deviation occurred, based on video data or telemetry data captured during the performance of the medical procedure, thereby enabling customization of the selection of educational content items for a specific part of the medical procedure.

[0075] The worker training module (235) may select one or more training content items to present to the medical worker by applying one or more trained machine learning models to attributes of the training content items, such as data describing the performance of a medical procedure performed by the medical worker and training content items associated with a type of medical procedure. Exemplary attributes of the training content items include: the type of medical procedure associated with the reference content item, one or more medical workers associated with the reference content item, the location where the medical procedure was performed (e.g., geographic location, medical facility identifier), the format of the reference content item (e.g., text data, audio data, video data, etc.), feedback on the reference content item from one or more medical workers (e.g., ranking, amount of positive feedback received for the reference content item, etc.), or other descriptive information. In various embodiments, the worker training module (235) trains one or more machine learning models to select one or more training content items for the medical worker based on the attributes of the training content items and the characteristics of the medical worker. Exemplary machine learning models include regression models, support vector machines, naive Bayes, decision trees, k nearest neighbors, random forests, boosting algorithms, k-means, and hierarchical clustering. Machine learning models may also include neural networks such as perceptrons, multilayer perceptrons, convolutional neural networks, recurrent neural networks, sequence-to-sequence models, generative adversarial networks, or transformers, and other types of machine learning models may be additionally or alternatively trained or applied by the practitioner training module (235) in various embodiments.

[0076] For example, to train a machine learning model to select one or more educational content items, the practitioner training module (235) generates a set of training examples, each training example including data describing the performance of a medical procedure performed by a medical practitioner and attributes of an educational content item, and has a label indicating whether the medical practitioner in the training example accessed the educational content item included in the training example (or whether the medical practitioner in the training example provided positive feedback on the educational content item included in the training example).

[0077] Applying a machine learning model to a training example generates a predictability that a healthcare worker in the training example will access an educational content item in the training example (or a predictability that a healthcare worker in the training example will provide positive feedback on an educational content item included in the training example). For each training example to which the worker training module (235) applies the machine learning model, the worker training module (235) generates a score for the machine learning model that includes an error term based on the label applied to the training example and the predictability that a healthcare worker in the training example will access an educational content item in the training example (or a predictability that a healthcare worker in the training example will provide positive feedback on an educational content item included in the training example). The error term and the corresponding score are larger when the difference between the label applied to the training example and the predictability of the healthcare worker in the training example accessing the educational content item in the training example (or the predictability of the healthcare worker in the training example providing positive feedback on the educational content item included in the training example) is larger, and are smaller when the difference between the label applied to the training example and the predictability of the healthcare worker in the training example accessing the educational content item in the training example (or the predictability of the healthcare worker in the training example providing positive feedback on the educational content item included in the training example) is smaller. In various embodiments, the worker training module (235) generates a score for the machine learning model applied to the training example using a loss function based on the difference between the label applied to the training example and the predictability of the healthcare worker in the training example accessing the educational content item in the training example (or the predictability of the healthcare worker in the training example providing positive feedback on the educational content item included in the training example). Exemplary loss functions include a mean squared error function, a mean absolute error function, a hinge loss function, and a cross-entropy loss function.

[0078] The worker training module (235) updates the set of parameters constituting the machine learning model by backpropagating error terms and stops backpropagating in response to the score or loss function meeting one or more criteria. For example, the worker training module (235) updates the parameters of the machine learning model by backpropagating the score to the machine learning model through the layers of the machine learning model until the score has a value smaller than a threshold. For example, the worker training module (235) updates the set of parameters constituting the machine learning model using gradient descent. The worker training module (235) stores machine learning models trained to apply to data describing the performance of medical procedures performed by medical workers and attributes of one or more training content items. In some embodiments, the worker training module (235) trains and maintains different machine learning models using different combinations of attributes of training content items and data describing the performance of medical procedures performed by medical workers, respectively.

[0079] Alternatively or additionally, one or more machine learning models applied by the practitioner training module (235) to select an educational content item are nearest neighbor models applied to the characteristics of a medical practitioner, including embeddings corresponding to the educational content item and data describing the performance of a medical procedure performed by the medical practitioner. As described in more detail above, exemplary attributes of an educational content item include: the type of medical procedure associated with the reference content item, one or more medical practitioners associated with the reference content item, the location where the medical procedure was performed (e.g., geographic location, medical facility identifier), the format of the reference content item (e.g., text data, audio data, video data, etc.), feedback on the reference content item from one or more medical practitioners (e.g., ranking, amount of positive feedback received for the reference content item, etc.), or other descriptive information. Exemplary characteristics of a medical worker include: the medical worker's specialty, the type of previous medical procedure performed by the medical worker, the medical procedure scheduled to be performed by the medical worker, the location where the medical worker performs the medical procedure (e.g., geographic location, medical facility identifier, etc.), collaborators connected to the medical worker via a connection graph, or other descriptive information about the medical worker and the data described above in more detail regarding the performance of the medical procedure by the medical worker.

[0080] In some embodiments, the practitioner training module (235) applies a nearest neighbor model to the medical practitioner embedding to determine the distance (or similarity measure) in a latent space between the medical practitioner embedding and the embeddings for various training content items. For example, the nearest neighbor model determines the Euclidean distance between the medical practitioner embedding and the embeddings for training content items. Based on the distance, the nearest neighbor model ranks the training content items according to the distance (or similarity measure) of the corresponding embeddings to the medical practitioner embeddings, selects one or more training content items that have a threshold position in the rank, and thus the selected one or more training content items have the embedding closest to the medical practitioner embedding. Alternatively, the nearest neighbor model selects one or more training content items that have a distance less than the threshold distance from the medical practitioner embedding. Alternatively, the practitioner training module (235) generates an embedding for a medical procedure performed by a medical practitioner, as described in more detail above, and selects one or more training content items based on the distance between the embedding for the medical procedure and the embedding for the training content item.

[0081] Furthermore, in some embodiments, the worker training module (235) generates embeddings for different medical workers based on the characteristics of the medical worker, as described in more detail above. The worker training module (235) determines the distance between the embedding for the medical worker and the embedding for additional medical workers. For example, the worker training module (235) determines the Euclidean distance between the embedding for the medical worker and the embeddings for a number of additional medical workers. Based on the distance (or similarity measure), the worker training module (235) selects a set of additional medical workers. For example, the worker training module (235) selects additional medical workers that have embeddings within a threshold distance from the embedding of the medical worker. As another example, the worker training module (235) ranks the additional medical workers based on the distance between the embedding of the additional medical workers and the embedding of the medical worker, and selects additional medical workers that have at least a threshold position in the rank. The worker training module (235) selects to present one or more educational content items to the medical worker, which are presented to one or more of the selected additional medical workers. Such an embodiment allows the worker training module (235) to select educational content items to present to the medical worker by utilizing similarities among various medical workers.

[0082] In various embodiments, the practitioner training module (235) determines one or more baseline criteria for educational content items by applying one or more clustering models to one or more attributes of a medical case in which a specific type of medical procedure was performed. The attributes of the medical case include one or more indicators generated for the medical case by the analysis module (230), telemetry data captured during the performance of the medical procedure in the medical case, video data captured during the performance of the medical procedure in the medical case, or other descriptive information about the medical case. Based on the attributes of the medical case, the practitioner training module (235) generates an embedding for the medical case. The practitioner training module (235) generates different clusters of cases in which a specific type of medical procedure was performed by applying a clustering model to the embeddings for different medical cases in which a specific type of medical procedure was performed. Different clusters are presented as different centroids in a latent space containing the embeddings for the medical cases, and the clusters include medical cases having embeddings within a threshold distance from the centroid of the cluster. In some embodiments, the practitioner training module (235) applies a k-means clustering model to embeddings for different medical cases in which a specific type of medical procedure was performed. Using k-means clustering causes medical cases in which a specific type of medical procedure was performed to be included in clusters based on the distance between the embedding for the medical case and the centroids of different clusters. Medical cases in which a specific type of medical procedure was performed are included in clusters that have centroids with the minimum distance from the embedding for the medical case. The centroids of the clusters are iteratively updated based on the embeddings for medical cases in which a specific type of medical procedure was performed that are included in various clusters until one or more criteria are met. This results in a specific number of clusters, each cluster containing medical cases in which a specific type of medical procedure was performed that have similar embeddings.

[0083] The worker training module (235) can identify baseline criteria based on medical cases included in one or more clusters. For example, a cluster of cases where a specific type of medical procedure was performed corresponds to a positive outcome for the specific type of medical procedure, and an alternative cluster corresponds to a negative outcome for the specific type of medical procedure. Based on telemetry data or video data captured during the performance of the specific type of medical procedure in additional cases, the worker training module (235) generates an embedding for the additional cases and determines the cluster containing the additional cases based on the cluster centroid and the embedding for the additional cases. In response to determining that the additional cases are included in an alternative cluster corresponding to a negative outcome, the worker training module (235) selects one or more training content items to present to the medical worker performing the specific type of medical procedure during the additional cases. The worker training module (235) compares video data or telemetry data captured during the performance of a specific type of medical procedure in additional medical cases with video data or telemetry data associated with a baseline standard of an educational content item associated with a specific type of medical procedure, and selects one or more educational contents having a baseline standard that specifies video data or telemetry data associated with a specific type of medical procedure and differs from video data or telemetry data captured during the performance of a specific type of medical procedure by at least a threshold amount.

[0084] Alternatively, the practitioner training module (235) selects an educational content item for a medical case in response to a determination that the embedding for the medical case is not included in a specific cluster. As an example, the practitioner training module (235) selects an educational content item for a medical case in response to a determination that the embedding for the medical case is greater than a threshold distance from the center point of a specific medical case cluster. This may indicate that the medical case has characteristics that deviate by at least a threshold amount from the characteristics of other medical cases with positive patient outcomes in which that type of medical procedure was performed. As described in more detail above, the practitioner training module (235) may select an educational content item having a baseline criterion that includes video data or telemetry data that differs by at least a threshold amount from the video data or telemetry data captured during the performance of the medical procedure in the medical case, which is associated with a specific type of medical procedure performed in the medical case.

[0085] When creating clusters of medical cases based on corresponding embeddings, the practitioner training module (235) may identify medical cases included in a specific cluster as reference cases for educational content items for the corresponding type of medical procedure. For example, in response to the practitioner training module (235) including a medical case in a specific cluster associated with a positive outcome, the practitioner training module (235) prompts the medical practitioner associated with the medical case to create a reference case based on the medical case. In response to receiving approval from the medical practitioner to create a reference case from the medical case, the practitioner training module (235) pseudonyms the patient data in the medical case and stores the pseudonymized patient data, video data captured during the performance of the medical procedure, telemetry data captured during the performance of the medical procedure, and one or more indicators generated for the medical procedure as educational content items for the corresponding type of medical procedure. One or more patterns determined from the telemetry data or video data, or one or more generated indicators, are stored as baseline criteria associated with the educational content items. This simplifies the creation of educational content items for a type of medical procedure by utilizing data captured by the collaborative medical platform (140) during the performance of the medical procedure to create educational content items for subsequent reference regarding the medical procedure.

[0086] In various embodiments, an educational content item selected for a medical worker based on the performance of a medical procedure by a medical worker is presented to the medical worker during the post-procedure phase. Presenting the educational content item to the medical worker during the post-procedure phase enables the review of the educational content item after the completion of the medical procedure. The worker education module (235) creates one or more interfaces that identify the educational content item selected for the medical worker. For example, the worker education module (235) includes information identifying the educational content item selected on a worker dashboard presented to the medical worker, such as the worker dashboard described in more detail below in relation to FIG. 4. In various embodiments, the information identifying the educational content item selected includes a link to retrieve the educational content item selected for presentation when selected by the medical worker. Alternatively, the worker education module (235) presents the information identifying the educational content item in a different interface or a different format. For example, the worker education module (235) sends a notification message to the medical worker's client device (150) that includes a link to retrieve the educational content item selected for presentation when selected by the medical worker.

[0087] The worker training module (235) may include selected educational content items in one or more interfaces presented to a medical worker when accessing the collaborative medical platform (140) in various embodiments. For example, the worker training module (235) creates an interface containing educational content and presents information describing the selected educational content item through the interface, so that the medical worker can select the information describing the selected educational content item to access the selected educational content item. As another example, the worker training module (235) includes information identifying the selected educational content item on a medical case page created by the interface management module (215) for a medical procedure for which the educational content item is selected. For example, the medical case page includes a section containing notes or feedback for the medical worker regarding the medical case, and one or more educational content items selected by the worker training module (235) are included in the section. Furthermore, the interface management module (215) may generate one or more interfaces containing recommendations for medical personnel based on indicators for medical personnel based on medical procedures, and the recommendation interface includes one or more educational content items selected by the personnel education module for medical personnel based on data describing the performance of one or more medical procedures.

[0088] In some embodiments, the worker training module (235) includes the selected training content item in different interfaces depending on the content of the selected training content item. For example, a training content item describing the use of medical equipment or medical devices is displayed in a recommendations interface. As another example, a training content item containing interactive materials or audio or video data to be presented to a medical worker is presented in a medical case page or a training interface. However, in other embodiments, the worker training module (235) selects an interface for identifying the selected training content item based on other characteristics of the training content item.

[0089] Alternatively or additionally, the worker training module (235) presents selected educational content items to the medical worker during the procedure phase of the medical procedure. This presents selected educational content items to the medical worker while the medical worker is performing the medical procedure. In various embodiments, the worker training module (235) transmits a notification identifying the selected educational content item to a client device (150) or medical equipment (160), which displays or audibly presents the notification to the medical worker. The notification may include specific content from the selected educational content item to simplify the medical worker's access to relevant information from the selected educational content item. In various embodiments, the worker training module (235) transmits a notification identifying the educational content item to the medical equipment (160) associated with the educational content item. For example, the educational content item may include recommended settings for the medical equipment (160) (e.g., force threshold, movement threshold), so transmitting the notification to the medical equipment (160) simplifies the identification of the medical equipment (160) associated with the educational content item. A notification transmitted to the medical equipment (160) may include a link that, when selected by a medical professional, causes the medical equipment (160) to execute one or more commands to modify one or more settings based on the educational content item. Similarly, information identifying an educational content item associated with the medical equipment (160) presented by the client device (150) may include a command that, when selected, transmits a command to modify one or more settings of the medical equipment (160). This simplifies the modification of settings of the medical equipment (160) based on the selected educational content item by reducing the amount of interaction with the medical equipment (160) by the medical professional. Alternatively, the professional training module (235) includes information identifying the selected educational content item in an interface presented to the medical professional through the client device (150).

[0090] In some embodiments, a medical worker authorizes the worker training module (235) to automatically modify one or more settings of medical equipment (160) based on training content items selected for the medical worker. Such authorization may be limited to one or more specific medical equipment (160) that is specific to a specific medical procedure or used during a specific medical procedure. When the medical worker authorizes the worker training module (235) to automatically modify one or more settings of medical equipment (160), the worker training module (235) sends a notification to the medical equipment (160) used for the medical procedure that includes one or more commands corresponding to the selected training content items. The medical equipment (160) executes one or more commands to modify one or more settings of the medical equipment (160) based on the selected training content items. In various embodiments, the medical equipment (160) displays a notification or informs the medical worker that one or more settings have been modified or specified based on the selected training content items. An indication that one or more settings will be modified based on selected educational content items may be presented to the medical worker by the medical equipment (160) or client device (150) to warn the medical worker that one or more settings of the medical equipment (160) are being automatically updated and to provide the medical worker with an option to prevent the modification of one or more settings. Alternatively, the worker training module (235) automatically modifies one or more settings of the medical equipment (160) based on the educational content items selected for the medical worker, as described in more detail above, unless the medical worker indicates that the worker training module (235) has not authorized the medical worker to automatically modify one or more settings of the medical equipment (160).This allows different embodiments to enable a medical worker to opt in to the worker training module (235) automatically modifying one or more settings of the medical equipment (160) or to opt out to the worker training module (235) automatically modifying one or more settings of the medical equipment (160).

[0091] In another embodiment, presenting an educational content item during the procedure phase of a medical case increases the number of interactions required to modify one or more settings of the medical equipment (160) used during the medical procedure. For example, presenting an educational content item through the medical equipment (160) causes the medical device (160) to request additional confirmation input from the medical worker after receiving input from the medical worker that changes a specific setting of the medical equipment (160) to a value that deviates from the corresponding value in the educational content item, or specifies a specific value for a specific setting of the medical equipment (160) outside the range corresponding to the educational content item. As an example, presenting an educational content item to the medical worker transmits a command to the medical equipment (160) used during the medical procedure, which causes the medical equipment (160) to display one or more warnings requesting input from the medical worker when the medical equipment (160) receives input from the medical worker that changes a setting value of the medical equipment (160) to a value outside the range included in the educational content item. This increases the difficulty for a medical professional who configures medical equipment (160) in a manner that does not match the selected educational content item, thereby increasing the likelihood that the setting value of the medical equipment (160) will match the selected educational content item.

[0092] In various embodiments, the practitioner training module (235) also creates one or more interfaces in which a medical practitioner requests a medical practitioner performing an evaluation to evaluate the medical practitioner's performance regarding a medical procedure. In various embodiments, the medical practitioner performing the evaluation has a connection with the medical practitioner through a connection graph repository (255). For example, the medical practitioner performing the evaluation collaborated with the medical practitioner in performing the medical procedure. As another example, the medical practitioner performing the evaluation supervised the medical practitioner performing the medical procedure. Alternatively, the medical practitioner performing the evaluation has a connection with the medical procedure in the connection graph repository (255).

[0093] Through an evaluation interface generated by the practitioner training module (235), the medical practitioner selects a medical procedure associated with the medical practitioner for evaluation. For example, the practitioner training module (235) presents an evaluation interface to the medical practitioner in response to receiving a request for evaluation from the medical practitioner. In various embodiments, the practitioner training module (235) receives an identifier of a specific medical procedure from the medical practitioner. Alternatively, the practitioner training module (235) receives a search query through the evaluation interface and presents search results containing one or more medical procedures associated with the medical practitioner and having attributes that match at least partially with the received search query. Subsequently, the medical practitioner selects a medical procedure presented from the search results. An exemplary evaluation interface is described in more detail below together with FIG. 11.

[0094] For a selected medical procedure, the practitioner training module (235) receives a selection of a framework for evaluating the performance of the selected medical procedure by the medical practitioner. In some embodiments, the practitioner training module (235) presents a framework interface that identifies a set of frameworks for evaluating the performance of the medical procedure and receives a selection of a framework from the set from the medical practitioner through the framework interface. One exemplary framework is described in more detail below together with FIG. 12. Different frameworks may correspond to different actions performed during the medical procedure or different techniques performed during the medical procedure. A framework may include guidelines or standards for performing different actions or techniques, or criteria for evaluating one or more actions or techniques. Each framework includes one or more categories corresponding to different competencies or techniques for performing the medical procedure. One or more frameworks may be specific to the medical facility where the medical procedure was performed, while one or more other frameworks may be based on guidelines or criteria designated by a standardization body or other entities associated with multiple medical facilities. Exemplary techniques or actions evaluated by the framework include the use of one or more medical devices (160) (e.g., robotic medical devices), techniques specific to a type of surgery, techniques specific to one or more medical procedures, or other actions affecting the performance of medical procedures.

[0095] When selecting a framework, the healthcare professional may also specify one or more evaluation objectives. An evaluation objective specifies one or more specific skills to be evaluated, identifies specific information to be provided in the evaluation, specifies one or more types of educational content items to be received based on the evaluation, or specifies other information to be received based on the evaluation. In various embodiments, multiple evaluation objectives may be specified. Specifying one or more evaluation objectives enables the healthcare professional to request specific feedback from the evaluation of a medical procedure.

[0096] Additionally, the practitioner training module (235) generates an evaluator selection interface for the practitioner for a selected medical procedure, an example of which is described in more detail below with reference to FIG. 13. The practitioner interacts with the evaluator selection interface to identify the evaluator for the medical procedure. The practitioner training module (235) receives the identifier of the evaluator from the practitioner, for example, through the evaluator selection interface. In various embodiments, the practitioner training module (235) retrieves the identifier of the practitioner connected to the practitioner or connected to the medical procedure through the connection graph repository (255) and presents the retrieved identifier to the practitioner through the evaluator selection interface. The practitioner selects the identifier of the practitioner or the practitioner connected to the medical procedure to select the evaluator through the evaluator selection interface. Alternatively or additionally, the practitioner training module (235) receives one or more search terms through the evaluator selection interface and generates search results including one or more practitioners connected to the practitioner or connected to the medical procedure and having at least partially matching the search query. The medical professional selects an identifier of the medical professional from the search results to select the medical professional to evaluate.

[0097] To evaluate the performance of a medical procedure by a medical worker, the worker training module (235) retrieves stored data describing the performance of a medical procedure by the medical worker and receives a selection of segments of data describing the performance of the medical procedure from the medical worker. For example, the worker training module (235) retrieves video data associated with the medical procedure and associated metadata (including telemetry data) that are captured during the medical procedure and stored in the video library (250). In various embodiments, the data describing the performance of the medical procedure includes captured video data of the medical procedure, telemetry data captured by one or more sensors (e.g., sensors in one or more medical devices (160) used during the medical procedure), or both video data and telemetry data. The segment of data describing the performance of the medical procedure selected by the medical worker includes a portion of data describing the medical worker's performance within a specific time interval. For example, the medical worker identifies a start time and an end time, and the segment of data describing the performance of the medical procedure includes data describing the performance of the medical procedure that occurs between the start time and the end time. Accordingly, in various embodiments, the segment includes video data of the medical worker between the start time and the end time, or telemetry data of the medical worker captured between the start time and the end time. In some embodiments, the worker training module (235) maintains one or more specific time intervals, the medical worker selects a start time, and the worker training module (235) determines an end time by incrementing the start time by the specific time interval to select a segment of data describing the performance of a medical procedure. In some embodiments, the worker training module (235) may maintain a single specific time interval.Alternatively, the practitioner training module (235) maintains a number of specific time intervals and receives the selection of specific time intervals by the medical practitioner when selecting a segment of data describing the performance of a medical procedure. In various embodiments, the practitioner training module (235) creates a segment selection interface, as described in more detail below together with FIG. 14, through which the medical practitioner selects one or more segments of data describing the performance of a medical procedure.

[0098] Alternatively, the practitioner training module (235) may apply one or more models to the data describing the performance of a medical procedure to automatically generate segments of the data describing the performance of the medical procedure and store the generated segments associated with the medical practitioner. Each segment contains a portion of the data describing the performance of the medical procedure between a start time and an end time. In various embodiments, the medical practitioner selects a specific segment from the stored segments.

[0099] In some embodiments, a medical professional may select multiple segments of data describing the performance of a medical procedure. Furthermore, in some embodiments, the medical professional may select different evaluation frameworks to evaluate different segments of data describing the performance of a medical procedure, thereby enabling different competencies or skills in different segments of data describing the performance of a medical procedure to be evaluated against different standards or criteria. In some embodiments, after selecting segments of data describing the performance of a medical procedure, the medical professional selects a framework for evaluating the segments.

[0100] In some embodiments, the healthcare worker provides self-assessment data for one or more categories within a selected framework. For example, the self-assessment data includes self-assigned values ​​for each category included in the selected framework that the worker training module (235) receives from the healthcare worker. In various embodiments, the self-assigned values ​​for the categories are selected from a set of values. Providing self-assessment data for categories within the selected framework enables the healthcare worker to compare the healthcare worker's self-assessment of performance in different categories of the framework with the assessment values ​​for the categories determined by the selected assessment healthcare worker. This comparison enables the healthcare worker to more accurately assess the healthcare worker's proficiency in various skills or competencies against the selected criteria.

[0101] The practitioner training module (235) generates an evaluation message comprising information about a selected medical procedure, information identifying a medical practitioner, a selected framework, and one or more selected segments of data describing the performance of the medical procedure (and, in some embodiments, self-evaluation data). In various embodiments, the practitioner training module (235) transmits the evaluation message to an evaluating medical practitioner who reviews the data describing the performance of the medical procedure in one or more selected segments and compares the data describing the performance of the medical procedure against criteria in the selected framework to generate an evaluation result for the medical practitioner. The evaluation result includes an evaluation value assigned to each category of the framework selected by the evaluating medical practitioner based on the review of the data describing the performance of the medical procedure in one or more segments.

[0102] Additionally, the evaluation results may include comments from the evaluating healthcare professional associated with one or more parts of the data describing the performance of a medical procedure within a selected segment. The evaluating healthcare professional may associate different comments with different parts of the data describing the performance of a medical procedure within the selected segment. For example, a comment may be associated with a specific time within the time interval constituting the segment of data describing the performance of the medical procedure. As one example, a comment may be associated with a specific time within the segment of data describing the performance of the medical procedure. In another example, a comment may be associated with a specific time within the segment of telemetry data describing the performance of the medical procedure. Including one or more comments associated with specific times within the selected segment enables the evaluating healthcare professional to provide feedback to the healthcare professional regarding specific times during the selected segment of data describing the performance of the medical procedure for further review.

[0103] In various embodiments, the practitioner training module (235) generates an evaluation result interface in response to receiving evaluation results from an evaluating medical practitioner. One exemplary evaluation result interface is described in more detail below together with FIG. 15. The evaluation result interface identifies the evaluating medical practitioner, the medical practitioner, and the type of medical procedure performed by the medical practitioner. Additionally, the evaluation result interface includes evaluation values ​​for different categories within a selected framework. In various embodiments, the evaluation result interface may include data describing the performance of a medical procedure corresponding to a selected section and may include comments associated with one or more specific times within the section of data describing the performance of the medical procedure received from the evaluating medical practitioner. For example, the evaluation result interface may include video data corresponding to a selected segment and one or more comments associated with one or more specific times in the video data corresponding to the selected portion. In some embodiments, the practitioner training module (235) displays a specific portion of data describing the performance of a medical procedure during a selected segment in response to receiving a selection of comments from the medical practitioner. In some embodiments, the comment may be displayed near a specific corresponding part of the data describing the performance of a medical procedure during a selected segment, thereby simplifying the review of the relevant part of the data describing the performance of the medical procedure while reviewing the comment.

[0104] In various embodiments, the practitioner training module (235) selects one or more training content items to be presented to medical practitioners based on one or more evaluation values ​​for one or more categories of the selected framework in the evaluation results. For example, in response to the practitioner training module (235) determining that the evaluation value for a category of the selected framework is less than a threshold, the practitioner training module (235) selects a training content item having a baseline criterion that is associated with a type of medical procedure and specifies a value for a category of the selected framework that is equal to or greater than the threshold. As another example, the practitioner training module (235) selects a training content item having a baseline criterion that is associated with a type of medical procedure in the evaluation results and specifies a value for a category of the selected framework that is at least a threshold amount greater than the evaluation value of the category. The practitioner training module (235) may select a training content item associated with each category of the selected framework that has an evaluation value smaller than the value for the corresponding category of the selected framework specified as the baseline criterion in the training content item associated with the type of medical procedure. Accordingly, the evaluation results from the medical worker being evaluated can be utilized by the worker education module (235), as described in more detail above, to select one or more educational content items to present to the medical worker.

[0105] Additionally or alternatively, the practitioner training module (235) may train an evaluation model to generate predictive evaluation values ​​for one or more categories based on data describing the performance of medical procedures using evaluation results obtained for various medical procedures having a specific type. In various embodiments, the practitioner training module (235) generates training examples from evaluation results received for medical procedures. Each training example includes a training data segment describing the performance of a segment of a training medical procedure having a type and a training framework, and each training example also includes one or more labels identifying the categories of the training framework and evaluation scores for the training categories of the training framework. In some embodiments, the training example includes multiple labels each containing a pair of identifiers for the training framework categories and corresponding evaluation scores. Alternatively, each training example includes a single label containing an identifier for the framework categories and evaluation scores for the categories. In various embodiments, the practitioner training module (235) obtains training examples based on evaluation values ​​provided by one or more evaluating medical practitioners when evaluating a segment of a medical procedure for a medical practitioner, as described in more detail above.

[0106] The practitioner training module (235) trains the evaluation model by applying it to a set of training examples. Applying the evaluation model to the training examples generates predicted evaluation values ​​for one or more categories of the training framework included in the training examples based on training data segments describing the performance of the training medical procedure in the training examples. For each training example to which the practitioner training module (235) applies the evaluation model, the practitioner training module (235) generates a score for the evaluation model including an error term based on the label applied to the training example and the predicted evaluation value (or based on one or more error terms based on the label corresponding to the different category of the training framework included in the training example and the predicted evaluation value for the corresponding category of the training framework included in the training example). The error term and the corresponding score increase as the difference between the label applied to the training example and the predicted evaluation value for the training example increases, and decrease as the difference between the label applied to the training example and the predicted evaluation value for the training example decreases. In various embodiments, the worker training module (235) generates a score for an evaluation model applied to a training example using a loss function based on the difference between the label applied to the training example and the predicted evaluation value for the training example. Exemplary loss functions include a mean squared error function, a mean absolute error function, a hinge loss function, and a cross-entropy loss function.

[0107] The practitioner training module (235) updates the set of parameters constituting the evaluation model by backpropagating error terms and stops backpropagating in response to the score or loss function meeting one or more criteria. For example, the practitioner training module (235) updates the parameters constituting the evaluation model by backpropagating the score to the evaluation model through the layers of the evaluation model until the score has a value less than a threshold. For example, the practitioner training module (235) updates the set of parameters constituting the evaluation model using gradient descent. The practitioner training module (235) stores data describing the performance of segments of medical procedures performed by medical practitioners and trained evaluation models to be applied to the framework to generate predicted evaluation scores for one or more categories within the framework. In some embodiments, the practitioner training module (235) trains and maintains different evaluation models for each category within the framework, and the category-specific evaluation models generate predicted evaluation values ​​for each different category within the framework.

[0108] In some embodiments, the practitioner training module (235) selects one or more training content items to present to the medical practitioner based on predicted evaluation values ​​for categories of the framework. For example, in response to receiving an evaluation request and a selection of frameworks from the medical practitioner, the practitioner training module (235) applies one or more evaluation models trained as described in more detail above to the segments of data describing the performance of medical procedures by the medical practitioner and to the selected framework to generate predicted evaluation values ​​for one or more categories of the selected framework. In some embodiments, the practitioner training module (235) applies different evaluation models to the segments of data describing the performance of medical procedures by the medical practitioner and to the selected framework to generate predicted evaluation values ​​for different categories of the selected framework. Similarly, the practitioner training module (235) may apply different evaluation models corresponding to different frameworks to different segments of data describing the performance of medical procedures. This allows different frameworks to be used to evaluate different segments of data describing the performance of medical procedures. Based on the difference between the predicted evaluation value for a category and the value for a category included as a baseline standard in one or more educational content items, the practitioner training module (235) may select one or more educational content items to present to the medical practitioner, as described in more detail above. Thus, the practitioner training module (235) may generate a predicted evaluation score for the medical practitioner based on segments of data describing the performance of the framework and medical procedure using one or more trained evaluation models, and then use one or more additional machine learning models to select the educational content items to present to the medical practitioner.

[0109] When selecting reference content items for a medical professional based on evaluation values ​​for categories of the framework or based on predicted evaluation values ​​for categories of the framework, the professional training module (235) may select training content items based on evaluation values ​​or predicted evaluation values ​​determined for individual medical procedures performed by the medical professional, as described in more detail above. Alternatively or additionally, the professional training module (235) determines an aggregate value for categories of the framework based on evaluation values ​​or predicted evaluation values ​​determined for multiple medical procedures performed by the medical professional that have a common type. For example, the aggregate value for categories of the framework is the average of evaluation values ​​or predicted evaluation values ​​determined for the medical professional for the performance of multiple medical procedures having a common type. As another example, the aggregate value for categories of the framework is the median of evaluation values ​​or predicted evaluation values ​​determined for the medical professional for the performance of multiple medical procedures having a common type. The worker training module (235) selects one or more training content items for a medical worker by comparing the aggregate value for a category, as described in more detail above, with a baseline criterion that specifies the value for the category in one or more training content items. Alternatively or additionally, the worker training module (235) selects one or more training content items for a medical worker based on changes in the evaluation value or predicted evaluation value for a category of the framework for the medical worker over time. For example, in response to the evaluation value or predicted evaluation value for a medical worker in a category of the framework decreasing by at least a threshold amount over a specific duration when performing a medical procedure of a specific type, the worker training module (235) selects training content items associated with a specific type and category of medical procedure to be presented to the medical worker.

[0110] The presentation module (240) facilitates the creation of presentations for education, research, training, or other purposes by utilizing stored information associated with completed medical procedures. The presentation may take the form of a slide deck, poster, video, animation, or other multimedia content. The presentation may incorporate various multimedia (e.g., video, images, 3D models, and associated metadata), patient record data, medical equipment telemetry data, information from content feeds, and other information generated or stored by the analysis or collaborative medical platform (140).

[0111] In one embodiment, the presentation module (240) may maintain one or more presentation templates for creating presentations. The templates may include pre-formatted content with various information fields that can be automatically populated from a series of records. For example, a practitioner who wishes to prepare a presentation related to a set of recently performed procedures may specify the set of procedures to include in the presentation, and the presentation module (240) may automatically populate the presentation based on data stored in association with the procedures, with each page associated with one or more types of data regarding the completed medical procedures. In some embodiments, the presentation module (240) may apply one or more trained machine learning models to automatically generate and / or recommend presentation content that may be of interest to the practitioner. In additional embodiments, the presentation module (240) may intelligently and automatically de-identify patient data included in the presentation.

[0112] The presentation module (240) may also include various editing tools for creating, viewing, and editing the presentation. For example, the editing tools may enable editing of text, video, images, animations, 3D models, or other content to be included in the presentation.

[0113] In one embodiment, the presentation may be presented directly through the presentation module (240) without the data associated with the presentation being exported outside the collaborative medical platform (140). For example, the presentation module (240) may enable live streaming of the presentation to a series of invited attendees during a telepresence session. The invited attendees may be limited to users (155) of the collaborative medical platform (140) or may include external attendees accessible via an external link. Sharing the presentation in this manner allows the practitioner to maintain data privacy and compliance and avoid potential issues that may arise when exporting medical data externally.

[0114] The application integration module (245) manages the integration of the application with the collaborative medical platform (140). The application can be utilized to add additional optional features to the collaborative medical platform (140). For example, the application can enable integration with a specific EHR system, scheduling system, or other existing medical system. The application can also allow the user to optionally add specific features in addition to the core features of the collaborative medical platform (140). The application integration module (245) can allow third parties to create applications that interface with the collaborative medical platform (140) and to add such applications.

[0115] The application integration module (245) may maintain a catalog of applications that can interface with the collaborative medical platform (140) and provide an interface that allows the user (155) to selectively add applications for integration. In various embodiments, applications identified by the application integration module (245) are approved or authorized by the administrator of the collaborative medical platform (140) for installation, thereby enabling the regulation of applications that can be executed on the collaborative medical platform (140).

[0116] Additionally, the application integration module (245) may include one or more application programming interfaces (APIs) for applications installed through the application integration module (245). The APIs for the applications provide the ability to exchange data between the applications and one or more components of the collaborative medical platform (140), thereby simplifying data exchange between the applications and other parts of the collaborative medical platform (140).

[0117] The video library (250) stores videos of various medical procedures, training presentations, simulations, or other medical videos, as well as metadata associated with the videos. Examples of metadata associated with medical procedure videos may include telemetry data from one or more medical devices received with the video, comments or annotations received from one or more medical personnel via a surgical interface during the medical procedure included in the video, segmentation data dividing the video into temporal segments associated with different procedure steps, profile information associated with the patient in the video (e.g., age, body mass index, gender, etc.), or other information supplementing the video. Various reference content items containing video data may be stored in the video library (250) for retrieval by the personnel training module (235) in various embodiments.

[0118] A video library (250) may store videos in an indexed database that indexes videos based on various metadata. Then, the video library (250) may be navigated or searched through a video library interface to identify related videos. The metadata associated with the videos may include permissions stored in a connection graph store (255) that controls which users (155) can access different videos. For example, videos within the video library (250) may be accessible only to users (155) who have viewing permissions for the videos, or the videos may be explicitly shared.

[0119] The connection graph store (255) includes a database that stores information describing connections between entities or other objects (e.g., video or other multimedia) managed by the collaborative medical platform (140). For example, as described above, the connection graph store (255) stores connections between users (155), connections between users (155) and procedures, connections between users (155) and multimedia content or other objects, or other connections between data entities of the collaborative medical platform (140).

[0120] The user profile repository (260) stores profile data for a user (155) of the collaborative medical platform (140). A user profile for a medical professional includes descriptive information such as the medical professional's name, contact information for the medical professional, credentials or licenses for the medical professional, biographical information for the medical professional, types of medical procedures that can be performed by the medical professional, medical facilities associated with the medical professional, operating room preferences (e.g., patient positioning, equipment settings, preferred devices, typical procedure step sequence, etc.), equipment configuration preferences (e.g., ergonomic settings for a robotic console), or other information describing the medical professional. The characteristics of the user profile may be inferred using machine learning techniques. For example, the medical professional's preferred devices or step sequence may be inferred from the application of a machine learning model trained to infer such preferences based on observed historical data. Additionally, a user profile for a medical professional includes information describing medical procedures as well as medical procedures that have been or will be performed by the medical professional. For example, the user profile may identify different types of medical procedures to be performed or performed by a medical professional and may include characteristics for each medical procedure (e.g., the length of time taken to complete the medical procedure, the number of times the medical professional has performed a medical procedure of a type matching the medical procedure, etc.). Furthermore, as described in more detail above, one or more indicators determined for the medical professional by the analysis module (230) may be included in the user profile for the medical professional.

[0121] The patient data repository (265) includes a patient profile for each patient associated with a medical case. The patient profile includes characteristics of the corresponding patient that may be obtained from electronic medical records for the patient or provided through input from a medical professional. The patient characteristics include demographic information about the patient, the patient's medical status, medical procedures previously performed by the patient, the patient's allergies, contact information about the patient, current or previous prescriptions for the patient, or other medically relevant information about the patient. A patient identifier is associated with the patient profile to uniquely identify the patient profile.

[0122] All data stored in the collaborative medical platform (140) (or made available through the collaborative medical platform (140)) may be stored, presented, and in some cases restricted in a manner that ensures compliance with various data privacy and protection regulations.

[0123] FIGS. 3a and 3b illustrate an exemplary worker dashboard (300). FIG. 3a illustrates the top portion of the dashboard (300), and FIG. 3b illustrates the bottom portion of the dashboard (300) (which may be continuously scrollable). The worker dashboard (300) may function as a home landing page for a medical worker when the medical worker logs into the collaborative medical platform (140). The worker dashboard (300) may include various content sections, at least some of which may be specifically targeted at the worker. A search bar (305) enables the input of a text-based search query to search for content available on the collaborative medical platform (140) (e.g., case pages, other user pages, videos, presentations, etc.). In response to the input of a search query, a list of results may be displayed along with links to content that matches the search query. The video promotion section (310) displays videos recently added by a worker along with user interface tools to allow the worker to promote videos by sharing them with other users, create highlight reels, or view various statistical information about the videos. The performance section (315) presents performance related to the use of the collaborative medical platform (140). In this example, the performance section (315) highlights that the user has recently reached 100 videos and provides a link to view the user's videos and access the video library. Other examples of performance in the performance section (315) may relate to the number of managed cases, the time spent using the platform (140), the number of accesses, the frequency of interactions, or other usage performance. The video library (320) includes video thumbnails, video tags, or other links that allow the medical worker to browse videos selected as potentially relevant.For example, relevant videos related to past or scheduled procedures associated with a medical professional may be selected based on the medical professional's history of watching videos, based on the medical professional's specialty of practice or other profile information, or based on other factors. The webinar promotion section (325) includes promotional banners for scheduled webinars that may be viewed within the collaborative medical platform (140). Webinars may be identified as being of potential interest to the medical professional based, for example, the topic of the webinar, the host of the webinar, or other factors. The shared case section (330) provides summary information and links to case pages shared with the medical professional. Examples of case pages are described in more detail below. The analysis summary (335) includes exemplary analyses associated with the medical professional's use of the collaborative medical platform (140), procedures performed by the medical professional, or other analytical data derived from information stored in the collaborative medical platform (140). The analytical data may be presented in one or more visual representations, such as graphs or charts. The feedback section (340) provides a link that allows medical professionals to send feedback to the administrator of the collaborative medical platform (140).

[0124] FIGS. 3a and 3b illustrate only one example of a worker dashboard (300). The types of content presented on the worker dashboard (300) may vary for different workers and / or may change dynamically over time for the same medical worker. Some sections may be fixed and always appear when accessing the dashboard (300) (e.g., search bar (305), video library (320), shared cases (330), analytics (335), and feedback sections (340)), while other sections (e.g., video promotions (310), achievements (315), webinar promotions (325)) may be dynamically inserted only in specific contexts. For example, webinar promotions (325) may be presented only when a scheduled webinar is scheduled that is considered to be of sufficient interest. Achievements (315) may similarly be displayed only when a relevant achievement has recently been accomplished. Additionally, the dashboard (300) may be customized by the user to display the desired sections in a configured order. Where present, various sections (305, 310, 315, 320, 325, 330, 335, 340) may also be presented in different order in different contexts.

[0125] FIG. 4 illustrates an alternative embodiment of a worker dashboard (400). In the example illustrated in FIG. 4, the worker dashboard (400) includes an educational content item section (405) containing information identifying an educational content item selected for a medical worker based on a previously performed medical procedure. The worker training module (235) selects the identified educational content item based on a stored baseline reference for the educational content item associated with the type of previously performed medical procedure and captured data describing the performance of the previously performed medical procedure, as described in more detail above in relation to FIG. 2. In various embodiments, the educational content item section (405) identifies one or more reasons why the identified educational content item is of potential interest to the medical worker. In the example of FIG. 4, the educational content item section (405) indicates that the identified educational content item includes proposed parameters or settings for medical equipment (160) (e.g., a robotic arm) used in the previously performed medical procedure. The educational content item section (405) in the example of FIG. 4 includes a link (410) for searching for educational content items to be presented to the medical worker through the medical worker's client device (150) when selected by the medical worker.

[0126] For the purpose of illustration, FIG. 4 illustrates an exemplary worker dashboard (400) in which an educational content item section (405) is displayed in proximity to a search bar (305). For example, the educational content item section (405) is displayed below the search bar (305) at a location on the worker dashboard (400), and thus the educational content item section (405) is displayed prominently on the worker dashboard (400) to increase the likelihood that a medical worker will select a link (410) to an identified educational content item. However, in other embodiments, the worker dashboard (400) displays the educational content item section (405) at a different location relative to other sections. Similarly, FIG. 4 illustrates an example in which a performance section (315) and a video library section (320) are displayed together with the educational content item section (405), but in other embodiments, different or additional sections are displayed by the worker dashboard (400) together with the proposed reference content item section (405).

[0127] In various embodiments, the educational content item section (405) is dynamically inserted into the worker dashboard (400) in certain contexts and is not included in the worker dashboard (400) in other contexts. For example, the worker dashboard (400) displays the educational content item section (405) after a medical worker completes a medical procedure. In one example, the worker dashboard (400) displays the educational content item section (405) after a certain amount of time has passed after the medical worker completes a medical procedure, but does not display the educational content section (405) before a certain amount of time has elapsed after the medical procedure is completed. In various embodiments, the worker dashboard (400) displays the educational content item section (405) for a specific time interval after the medical worker completes a medical procedure.

[0128] FIG. 4 illustrates an example in which an educational content item section (405) identifies a single reference content item, but in other embodiments, the educational content item section (405) displays multiple reference content items selected for a medical professional. For example, the educational content item section (405) is a carousel content item having multiple slides, each slide containing information identifying a different selected educational content item and a link to a different selected educational content item. In response to a medical professional performing a specific interaction with the educational content item section (405), the educational content item section (405) is updated to display a different slide containing information identifying a different selected educational content item. For example, the educational item section (405) displays an alternative slide containing information identifying a different selected educational content item in response to a medical professional performing a swipe gesture along an axis perpendicular to the axis including the search bar (305), the educational content item section (405), the performance section (315), and the video library section (320). This allows a single section of the worker dashboard (400) to enable medical workers to identify multiple educational content items.

[0129] In some embodiments, the collaborative medical platform (140) generates one or more educational interfaces, such as an educational dashboard. The educational interface additionally or alternatively displays one or more proposed educational content items to the medical worker, thereby providing the medical worker with an additional way to access the proposed educational content items. The worker dashboard (400) may include interface elements that cause the display of the educational interface when selected by the medical worker. In some embodiments, the educational interface may display information identifying multiple proposed educational content items, thereby enabling the medical worker to more easily access a wider range of proposed educational content items.

[0130] FIG. 5 illustrates an exemplary embodiment of a case sharing interface (500) for sharing a case with one or more contributors. Adding contributors to a case can create a connection between the contributor and the case, and between the contributor and the case owner. The case sharing interface (500) includes a selection element (505) for receiving identification information to identify a desired contributor. For example, the selection element (505) may receive the email address, name, username, or other identifier of a medical professional or other requested contributor. In some embodiments, when identification information for a desired contributor is selected, the case sharing interface (500) displays all or part of the profile data for the requested collaborator so that the requester can verify whether the matching profile data is the intended collaborator. Then, the case sharing interface (500) allows the requester to confirm or reject the selection of the collaborator and to interact with an authority selection element (520) to set the desired authority level for the requested collaborator. Here, the authority level may restrict the invited collaborator's access to data regarding the case and / or restrict the actions the collaborator is allowed to perform in connection with the case. In one exemplary embodiment, the authority level may be selected from the "collaborator" level (525A) and the "delegate" level (525B).

[0131] In response to receiving input to select a requested collaborator and set a desired level of permission (via the permission selection element (520)), the case sharing interface (500) may send an invitation to the requested contributor (e.g. via email, text message, phone call, portal message, or other communication mechanism) so that the requested collaborator may accept or decline the request. If the request is accepted, the case sharing interface (500) may add an identifier or other information about the new collaborator to the linked healthcare worker list (510) that lists contributors added to the case. For example, the illustrated example shows a linked healthcare worker list (510) including a case owner (515) and three additional contributors added to the case.

[0132] The case sharing interface (500) may also allow the case owner to change the authority level of an existing contributor in the linked medical worker list (510). Additionally, the case sharing interface (500) may include a removal element (530) associated with each contributor in the linked medical worker list (510) to allow the contributor to be removed from the case. The selection of the removal element (530) may remove the stored connection between the worker and the case, thereby preventing the worker from accessing the case any longer.

[0133] FIG. 6 is an exemplary embodiment of a case dashboard (600) for a medical professional. The case dashboard (600) enables access to cases owned by the medical professional and cases shared with the medical professional by other users (155), as indicated in the case summary (610). In this example, the case dashboard (600) is composed of a series of case cards (605) each graphically illustrating a summary of the case. Selecting a case card (605) leads to a case page (400) for the case. In an alternative embodiment, the dashboard (600) may be presented in a list view or other view, without necessarily presenting the case cards (605) in the visual form shown in FIG. 6.

[0134] FIG. 7 illustrates an example of a telepresence interface (700) associated with a telepresence session that may occur during an actual procedure or a simulated procedure. Alternatively, the telepresence session may be utilized for live planning purposes without necessarily performing or simulating the procedure. In this example, the telepresence interface (700) displays a three-dimensional model of a target anatomical structure (705) associated with the procedure. The model may include annotated comments that may be acquired during the telepresence session or added during the pre-procedure stage. Alternatively, the telepresence interface (700) may include a real-time video or image view associated with the procedure in progress. In one embodiment, each contributor may switch between different associated views, such as real-time video or image, a three-dimensional model, a pre-procedure image, or other related multimedia.

[0135] The telepresence interface (700) may also include a telepresence content feed (715) for exchanging real-time messages between contributors. For example, the telepresence content feed (715) allows a user to post messages and / or view messages from other participants. Messages may include text, media content (e.g., images, videos, animations, etc.), or links to various media content or other resources (e.g., research papers). The telepresence interface (700) may also allow a participant to provide annotations on a target anatomical structure (presented in the form of an image, video, or model). For example, a participant may pin annotations to a specific location on the depicted anatomical structure, as indicated by an identifier (710).

[0136] Additionally, the telepresence interface (700) may display statistics (720) or other analyses that may be relevant to the procedure. The statistics (720) may include estimated or modeled values ​​or indicators related to anatomical structures based on various detected data from the medical equipment (160). The telepresence interface (700) may dynamically update the statistics (720) over time during the procedure.

[0137] FIG. 8 is another example of a telepresence interface (800) associated with a telepresence session. In this example, the telepresence interface (800) displays a live video of a procedure being performed, along with an annotation tool set (810) that allows a remote contributor to add annotations (805) overlaid on the video. The telepresence interface (800) also includes an alternative set of views (815) that the contributor can switch to during the telepresence session. These alternative views (815) may include one or more different camera views (e.g., a view of the medical environment), one or more three-dimensional models (e.g., as illustrated in FIG. 7), a view of pre-procedure images, or other multimedia associated with the case. In various embodiments, a telepresence interface such as that illustrated in FIG. 7 or FIG. 8 is stored for a medical case and, as described in more detail above in relation to FIG. 2, may be presented later to another medical worker if the medical worker associated with the case approves the creation of a reference case based on the medical case.

[0138] In the example of FIG. 8, the telepresence interface (800) also displays an educational content item (820) to a medical worker, such as a medical worker performing a medical procedure. The educational content item (820) is dynamically selected by the telepresence interface (800) in various embodiments based on video data or telemetry data captured during the medical procedure. The telepresence interface (800) describes the educational content item (820) or includes information extracted from the educational content item (820), enabling the medical worker to capture content from the educational content item (820) through the telepresence interface (800). In various embodiments, the telepresence interface (800) limits the presentation of the educational content item (820) to a specific time interval. For example, the telepresence interface (800) displays the educational content item (820) in response to the collaborative medical platform (140) determining that video data or telemetry data captured during the performance of a medical procedure deviates by at least a threshold amount from the baseline reference associated with the educational content item (820). When the captured video data or telemetry data does not deviate by at least a threshold amount from the corresponding baseline reference for the educational content item (820), the telepresence interface (800) does not present the educational content item (820). For example, a client device (150) displaying the telepresence interface (800) receives a presentation command from the collaborative medical platform (140) to present the educational content item (820) and subsequently receives an alternative command from the collaborative medical platform (140) to stop presenting the educational content item (820).An alternative command may be received in response to the collaborative medical platform (140) determining that the video data or telemetry data received during the performance of a medical procedure meets the baseline criteria associated with the educational content item (820), or in response to the determination that the video data or telemetry data no longer identifies a pattern corresponding to the baseline criteria associated with the educational content item (820).

[0139] To simplify the integration of information from the educational content item (820) into a medical procedure, the telepresence interface (800) presents a modification command (825) associated with the educational content item (820). The modification command (825) includes an identifier of the medical equipment (160) and one or more setting values ​​for the medical equipment (160). In response to a medical professional selecting the modification command (825) through the telepresence interface (800), the collaborative medical platform (140) receives a request to identify the educational content item (820) and the medical equipment (160). In response to receiving the request, the collaborative medical platform (140) transmits the command to the identified medical equipment (160) to modify one or more setting values ​​with values ​​retrieved from the educational content item (820) and included in the command transmitted to the medical equipment (160). In various embodiments, the collaborative medical platform (140) determines the identifier of the medical equipment (160) based on the identifier included in the telemetry data received by the collaborative medical platform (140) or the identification information included in the received video data of the medical procedure. This simplifies the modification of one or more settings of the medical equipment based on the educational content items (820) through interaction with the telepresence interface (800) rather than by manually entering the setting values ​​identified by the educational content items into the medical equipment (160).

[0140] FIG. 9 is an exemplary embodiment of an analysis dashboard (900) for a medical worker. In this example, the analysis dashboard (900) displays a summary of cases managed by a medical worker, including, for example, the total number of cases, the number of cases in the current month, the number of cases in the current week, and the distribution of the types of cases performed by the worker. In the example of FIG. 9, the analysis dashboard (900) also displays an educational content item section (905) to the medical worker. The educational content item section (905) includes information identifying an educational content item selected by the collaborative medical platform (140) for the medical worker based on data describing the performance of medical procedures by the medical worker, as described in more detail above in relation to FIG. 2. In various embodiments, the educational content item section (905) includes a link to retrieve an educational content item from the collaborative medical platform (140) or a third-party server (170) for presentation when accessed. In some embodiments, the educational content item section (905) may identify selected educational content items based on the most recently completed medical procedure by the medical professional. Alternatively, the analytics dashboard (900) includes a plurality of educational content item sections (905), each of which includes an educational content item selected for a medical procedure previously performed by the medical professional, thereby simplifying access to different educational content items related to various medical procedures performed by the medical professional.

[0141] FIG. 10 is an exemplary embodiment of a case video interface dashboard (1000) for viewing case videos. The case video may be captured during a telepresence session or similarly captured during a procedure without a telepresence session that is live-streamed. The case video interface (1000) includes a video interface that displays one or more views of a video associated with a medical procedure. The video interface (1000) may include a number of captured views that may originate from a camera within the medical environment, a camera inserted into an anatomical structure (e.g., an endoscope camera), or other cameras. The captured views may also include a 3D model, pre-procedure images, procedure planning documents, or other visual information. The video may be segmented (manually or automatically using video processing and content recognition technologies) to divide the video into segments associated with different stages of the procedure. The video may include annotations provided by a medical professional during a telepresence session or in a post-procedure review. A content feed (1010) may be presented in conjunction with a video that allows a user (155) to post comments, links, media, or other content in connection with the presentation. Reference content items, such as reference cases, may display videos of medical procedures and other information (e.g., content feed (1010)) to medical professionals using the video interface (1000) described in relation to FIG. 10.

[0142] FIG. 11 is an exemplary evaluation interface (1100) in which a medical professional requests an evaluation of at least part of a medical procedure performed by a medical professional. In various embodiments, a collaborative medical platform (140) displays the evaluation interface (1100) in response to receiving a request from the medical professional’s client device (150) for an evaluation of a medical procedure performed by the medical professional. The evaluation interface (1100) illustrated in FIG. 11 includes a search element (1105) configured to receive one or more search terms from the medical professional. Based on the received search terms, the collaborative medical platform (140) identifies one or more medical procedures having one or more characteristics associated with the medical professional and satisfying at least partially the received search terms. In response to the received search terms, the evaluation interface (1100) displays a medical procedure identifier (1110) of the identified one or more medical procedures. For example, the collaborative medical platform (140) searches for medical procedures connected to a medical professional in the connection graph repository (255) and identifies one or more searched medical procedures having characteristics that at least partially match a search term received through a search element (1105) in the evaluation interface (1100). The medical professional selects a medical procedure identifier (1110) corresponding to the medical procedure to be evaluated.

[0143] FIG. 12 is an exemplary framework interface (1200) for a medical professional to select a framework for evaluating at least a portion of a medical procedure performed by the medical professional. In various embodiments, a collaborative medical platform (140) presents the framework interface (1200) to the medical professional after receiving the selection of a medical procedure from the medical professional. The framework interface (1200) presents a set of frameworks (1205) for evaluating the medical procedure. Different frameworks in the set (1205) may correspond to different types of actions performed during the medical procedure or different techniques performed during the medical procedure. A framework may include guidelines or standards for performing different types of actions or techniques, or may include criteria for evaluating one or more actions or techniques. Each framework includes categories corresponding to different competencies or techniques applicable to performing the medical procedure. Different frameworks may specify different criteria for evaluating categories, or different frameworks may include different categories. One or more frameworks may be specific to the medical facility where the medical procedure was performed, while one or more other frameworks may be based on guidelines or standards designated by a standardization body or other entities associated with multiple medical facilities. A medical professional selects a framework from a set (1205) through interaction with the framework interface (1200). For example, a medical professional performs a specific interaction with information identifying a framework from the set (1205) to select a framework for evaluating the medical procedure.

[0144] In various embodiments, the framework interface (1200) also includes an evaluation goal element (1210). An evaluation goal identifies one or more specific skills to be evaluated by an evaluating healthcare professional, identifies specific information to be provided in the evaluation, identifies one or more types of educational content items to be received based on the evaluation, or identifies other information to be received based on the evaluation. In various embodiments, multiple evaluation goals may be specified. The evaluation goal element (1210) receives input from a healthcare professional describing or identifying an evaluation goal. For example, the evaluation goal element (1210) includes a text box configured to receive text input describing an evaluation goal from a healthcare professional. As another example, the evaluation goal element (1210) includes a set of evaluation goals, and a healthcare professional interacts with the evaluation goal element (1210) to select one or more evaluation goals from the set of evaluation goals.

[0145] FIG. 13 is an exemplary evaluator selection interface (1300) for a medical professional to select an evaluator for a medical procedure. In some embodiments, the collaborative medical platform (140) presents the evaluator selection interface (1300) to the medical professional after presenting the evaluation interface (1100) and the framework interface (1200). The evaluator selection interface (1300) includes an evaluator search element (1305) configured to receive a search query from the medical professional. Based on the search query, the evaluator selection interface (1300) displays one or more medical professional identifiers (1310). Each medical professional identifier (1310) corresponds to a medical professional having one or more characteristics that satisfy at least partially the search query. In some embodiments, the medical professional identifier (1310) corresponds to a medical professional having one or more characteristics that have a connection with the medical professional or medical procedure in the connection graph store (225) and match at least partially the search query. The medical worker corresponding to the medical worker identifier (1310) selected by the medical worker through the evaluation interface (1300) is the medical worker who evaluates the medical procedure performed by the medical worker.

[0146] FIG. 14 is an exemplary segment selection interface (1400) for a medical professional to select a segment of data describing the performance of a medical procedure for evaluation by an evaluating medical professional. In some embodiments, the segment selection interface (1400) is presented after the collaborative medical platform receives the evaluation medical professional's selection. However, in other embodiments, the segment selection interface (1400) may be displayed after the evaluation interface (1100) and before the framework interface (1200), or at a different time in relation to the selection of the framework and the selection of the evaluation medical professional. In the example of FIG. 14, the segment selection interface displays video data (1405) of the medical procedure. In some embodiments, the segment selection interface (1400) displays telemetry data captured during the performance of the medical procedure, or displays a combination of video data (1405) of the medical procedure and telemetry data captured during the performance of the medical procedure. In various embodiments, the segment selection interface (1400) includes a media player that plays the video data (1405) of the medical procedure to the medical professional. The media player includes control means for changing the playback speed of the video data (1405) and navigating the video data (1405) to simplify navigation through the video data (1405) by a medical professional.

[0147] Additionally, the segment selection interface (1400) includes a start time (1410) and an end time (1415) for the segment. In some embodiments, a medical professional manually enters the start time (1410) into a corresponding interface element of the segment selection interface (1400) and manually enters the end time (1415) into a corresponding interface element of the segment selection interface (1400) to specify a time range of data (e.g., video data (1405), telemetry data, or a combination of video data and telemetry data) describing the performance of a medical procedure including the segment. Alternatively or additionally, the medical professional performs a specific interaction with the video data (1405), and the time at which the medical professional performs the specific interaction includes the start time (1410) of the segment. Similarly, the medical professional performs an additional specific interaction with the video data at a subsequent time, and the subsequent time includes the end time (1415) of the segment. The segment selection interface (1400) displays the start time (1410) and end time (1415) of the segment, allowing a medical professional to review or modify the time intervals that make up the segment.

[0148] In various embodiments, the collaborative medical platform (140) determines the end time (1415) of a segment based on the start time (1410). For example, the segment has a specific duration, and thus the collaborative medical platform (140) determines the end time (1415) by increasing the start time (1410) by a specific time interval. In some embodiments, a specific time interval is used for each segment. Alternatively, different specific time intervals may be selected for different segments, and the medical professional selects a specific time interval from a set presented by the segment selection interface (1400) for the segment. Maintaining a set of specific time intervals allows the medical professional to change the duration of different segments.

[0149] In some embodiments, such as the example illustrated in FIG. 14, the segment selection interface (1400) includes a video data timeline (1420) that can indicate a specific time in the video data (1405). For example, the video data timeline (1420) includes markings corresponding to every 15-minute interval in the video data (1405). In response to a healthcare worker selecting a portion of the video data timeline (1420), the collaborative healthcare platform (140) updates the corresponding portion of the video data (1405) presented by the segment selection interface (1400) to start at a time corresponding to the selected portion of the video data timeline (1420). The video data timeline (1420) displays a segment identifier (1425) that visually indicates the start time (1410) and end time (1415) of the selected segment for the entire time length of the video data (1405), thereby enabling the healthcare worker to easily identify the location of the selected segment within the video data (1405).

[0150] A segment designated by a start time (1410) and an end time (1415) includes video data that occurred between the start time (1410) and the end time (1415). Furthermore, in various embodiments, the segment includes telemetry data captured during the performance of a medical procedure between the start time (1410) and the end time (1415), along with video data (1405) captured between the start time (1410) and the end time (1415). A collaborative medical platform (140) may perform one or more preprocessing steps to synchronize the video data (1405) with the telemetry data captured between the start time (1410) and the end time (1410). In some embodiments, a segment selection interface (1400) includes an interface element that allows a medical professional to select whether the segment includes both video data and telemetry data, only video data, or only telemetry data.

[0151] FIG. 15 is an exemplary evaluation result interface (1500) that presents an evaluation result for a medical professional performing a medical procedure. The evaluation result interface (1500) includes a medical professional identifier (1505) of the medical professional who requested the evaluation. For example, the medical professional identifier (1505) may be the medical professional's name, the medical professional's image, the medical professional's username, or any other information that uniquely identifies the medical professional. In various embodiments, such as the example illustrated in FIG. 15, the evaluation result interface (1500) also presents medical professional information (1510) for the medical professional who requested the evaluation. In some embodiments, the medical professional information (1510) is presented in response to an interaction with the medical professional identifier (1505). Exemplary medical worker information (1510) includes the role of the medical worker in medical procedures, the years of experience of the medical worker, the number of medical procedures performed by the physician, the number of medical procedures having a common type with the medical procedure selected for evaluation performed by the medical worker, one or more previous evaluation results for the medical worker, or other information about the medical worker.

[0152] The evaluation result interface (1500) also presents an evaluation medical worker identifier (1515) for the evaluation medical worker. For example, the evaluation medical worker identifier (1515) may be the name of the evaluation medical worker, an image of the evaluation medical worker, or other information that uniquely identifies the evaluation medical worker. Additionally, the evaluation result interface (1500) identifies the type of medical procedure (1520) selected for evaluation. In various embodiments, performing one or more interactions with the type of medical procedure (1520) selected for evaluation causes procedure details (1525) to be presented. The procedure details (1525) include contextual information about the medical procedure selected for evaluation. Exemplary procedure details (1525) include the gender of the patient on whom the medical procedure was performed, the age of the patient on whom the medical procedure was performed, one or more physical characteristics of the patient on whom the medical procedure was performed, one or more other medical workers associated with the medical procedure, or other information about the medical procedure.

[0153] To present evaluation results to a medical professional, the evaluation result interface (1500) includes a framework identifier (1530) that identifies a framework selected by the medical professional for evaluation. The evaluation result interface (1500) also identifies one or more categories within the framework, along with a corresponding evaluation value for each category. In the example of FIG. 15, the evaluation result interface (1500) displays information identifying a category (1535) (e.g., the name of the category (1535)) and displays an evaluation value (1540) for the category (1535) near the information identifying the category (1535). The combination of the categories of the framework and the corresponding evaluation values ​​constitutes an evaluation result for the medical professional regarding a medical procedure. In some embodiments, the evaluation result also includes an evaluation summary (1545) containing text data or other information of the evaluating medical professional explaining the reasoning or basis for one or more evaluation values ​​(1540).

[0154] The evaluation result interface (1500) also displays video data (1550) of a medical procedure selected for evaluation. In various embodiments, the evaluation result interface (1500) includes a media player having one or more control means to allow a medical professional to view or navigate the video data (1550) of the medical procedure. As described in more detail above with reference to FIG. 2, the evaluating medical professional may provide one or more comments when evaluating the medical procedure. The comments may be associated with a specific time in the video data (1550) (or telemetry data captured during the medical procedure). In various embodiments, the evaluation interface (1500) displays a comment (1555) containing text data and displays a specific time (1560) associated with the comment (1555) near the comment (1555). In various embodiments, the specific time (1560) identifies the start time of a portion of the data (e.g., video data (1550)) describing the performance of the medical procedure corresponding to the comment (1555). A medical professional may select a specific time (1560) associated with a comment (1550) through the evaluation result interface (1500) (or select the comment (1550) through the evaluation result interface (1500)) to initiate the presentation of video data (1550) starting at the specific time (1560) by the evaluation result interface (1500), thereby simplifying the review of a specific portion of the video data (1550) (or other data describing the performance of a medical procedure) corresponding to an individual comment. In some embodiments, the evaluation result interface (1500) presents telemetry data or other data describing the performance of a medical procedure together with (or instead of) the video data (1550).

[0155] FIG. 16 is an exemplary embodiment of a process in which a medical professional obtains an evaluation of a medical procedure performed by an evaluating medical professional through a collaborative medical platform (140). The collaborative medical platform (140) receives a request from a medical professional for an evaluation of a performance during a medical procedure (1602) and receives a selection of a medical procedure for evaluation from the medical professional (1604). For example, in response to receiving the request (1602), the collaborative medical platform (140) identifies a medical procedure associated with the medical professional and receives a selection of the identified medical procedure from the medical professional (1604). As another example, the request for an evaluation of a performance during a medical procedure may include an identifier of the medical procedure to be evaluated.

[0156] Additionally, the collaborative medical platform (140) receives a selection of a framework for evaluating a selected medical procedure from a medical practitioner (1606). Each framework includes one or more categories corresponding to different competencies or skills applicable to performing the medical procedure. The framework also includes guidelines or standards for performing different types of actions or skills, or criteria for evaluating one or more actions or skills. In various embodiments, the medical practitioner selects a framework from a set of frameworks identified by the collaborative medical platform (140).

[0157] The collaborative medical platform (140) receives a selection of a segment of data describing the performance of a selected medical procedure from a medical professional (1608). The data describing the performance of the selected medical procedure includes video data, telemetry data, or a combination thereof captured during the performance of the selected medical procedure. In various embodiments, the collaborative medical platform (140) retrieves stored data describing the performance of the selected medical procedure, and the medical professional identifies a segment by selecting a start time and an end time of the stored data describing the performance of the selected medical procedure (1608). The segment includes a portion of the data describing the performance of the selected medical procedure between the start time and the end time. In some embodiments, the medical professional specifies a start time, and the collaborative medical platform (140) determines the end time of the segment by incrementing the start time by a specific time interval. In various embodiments, the specific time interval may be predefined by the collaborative medical platform (140) or specified by the medical professional. A medical professional selects multiple segments of data describing the performance of a medical procedure (1608) and selects multiple frameworks (1606), so that the use of different frameworks can evaluate different segments.

[0158] Based on a selected medical procedure, a selected framework, a selected medical professional for evaluation, and a selected segment of data describing the performance of the selected medical procedure, the collaborative medical platform generates an evaluation message and transmits it to the medical professional for evaluation (1612). In response to receiving the evaluation message, the medical professional for evaluation reviews the selected segment of data describing the performance of the selected medical procedure with consideration of the selected framework and generates an evaluation result including evaluation values ​​for each category within the selected framework. As described in more detail above with Fig. 2, the evaluation result may also include comments associated with a specific time within the selected segment of data describing the performance of the medical procedure. The collaborative medical platform (140) presents the evaluation result to the medical professional through an evaluation result interface or other communication channel described in more detail above with Fig. 15 (1614). In some embodiments, the collaborative medical platform (140) selects one or more educational content items for the medical professional based on the evaluation result, as described in more detail above with Fig. 2. For example, the collaborative medical platform (140) selects an educational content item having a baseline criterion that specifies a value of a selected framework category that is at least a threshold greater than the evaluation value for a selected framework category associated with a type of selected medical worker.

[0159] In an alternative embodiment, the collaborative medical platform (140) applies one or more trained evaluation models to a selected segment of data describing the performance of a selected medical procedure and to a selected framework. As described in more detail above with reference to FIG. 2, the evaluation model generates a predicted evaluation value for a category of the selected framework based on the selected segment of data describing the performance of the selected medical procedure. In some embodiments, the predicted evaluation value may be presented along with its corresponding category. Furthermore, in various embodiments, the collaborative medical platform may select one or more educational content items for medical practitioners based on one or more predicted evaluation values. Thus, in some embodiments, the collaborative medical platform (140) automatically generates evaluation results using one or more trained models instead of receiving evaluation results from a selected medical practitioner.

[0160] The described embodiments incorporate a number of technical improvements that enhance the capabilities of computer systems, machine learning techniques, data management systems (particularly those related to medical data management), computer-based user interfaces, robotic and / or other medical device systems, and other technologies and technical fields. For example, the described embodiments provide technical improvements in data availability and data privacy by enabling the automated processing of sensitive and / or restricted data, such as operating room videos, patient health records, or other sensitive health data.

[0161] The described embodiments also include improvements in machine learning methods in that they improve predictive capabilities compared to traditional machine learning techniques by combining information from heterogeneous data sources, including medical equipment telemetry data, video data, and mobile device data. Furthermore, the described embodiments provide better patient outcomes and consequently provide technical improvements in disease treatment by enabling the generation of various notifications, recommendations, or other content customized for specific healthcare professionals, thereby improving the care provided by healthcare professionals.

[0162] In addition, the described embodiment provides technical improvements in the field of robot-assisted surgery by enabling the automated configuration of surgical robots based on accumulated and aggregated healthcare data associated with patients, facilities, and personnel. This provides improved performance of surgical robots, improved human-robot interaction, and improved patient care outcomes.

[0163] The foregoing description of the embodiments is provided for illustrative purposes only and is not intended to encompass all embodiments or limit them to the exact form disclosed. Those skilled in the art will understand that many modifications and changes are possible in light of the foregoing disclosure.

[0164] Some parts of this description describe embodiments in terms of algorithms and symbolic representations of operations on information. These operations are described functionally, computationally, or logically, but are understood to be implemented by computer programs or equivalent electrical circuits, microcode, etc. Furthermore, it has been found convenient from time to time to refer to such arrangements of operations as modules without loss of generality. The described operations and their associated modules may be implemented in software, firmware, hardware, or a combination thereof.

[0165] Any step, operation, or process described herein may be performed or implemented by one or more hardware or software modules, either alone or in combination with other devices. Embodiments may also relate to devices for performing the operations of the present invention. Such devices may include general-purpose computing devices that are specifically configured for the required purpose and / or are optionally activated or reconfigured by a computer program stored in a computer. Such computer programs may be stored on a non-transient computer-readable storage medium of a tangible form or on any type of medium suitable for storing electronic instructions and may be coupled to a computer system bus. Additionally, any computing system mentioned herein may include a single processor or may include an architecture employing a multi-processor design for increased computing power.

[0166] Unless explicitly stated otherwise, the use of “or” herein refers to an inclusive “or” rather than an exclusive “or.” For example, the condition “A or B” is satisfied by any one of the following: when A is true (or exists) and B is false (or absent); when A is false (or absent) and B is true (or exists); and when both A and B are true (or exist). Similarly, the condition “A, B or C” is satisfied by any combination of A, B, and C being true (or exist). As a non-limiting example, the condition “A, B or C” is satisfied when A and B are true (or exist) and C is false (or absent). Similarly, as another non-limiting example, the condition “A, B or C” is satisfied when A is true (or exists) and B and C are false (or absent).

[0167] Finally, the language used in this specification has been chosen primarily for readability and educational purposes and may not have been chosen to describe or limit the essence of the invention. Therefore, the scope is not intended to be limited by this detailed description, but rather by any claims derived therefrom for an application based herein. Accordingly, the disclosure of the embodiments is intended to exemplify, rather than limit, the scope of the invention as defined in the following claims.

Claims

Claim 1 A method for evaluating a medical professional performing a medical procedure through an online collaborative medical platform, comprising: receiving a request for evaluation from the medical professional on the collaborative medical platform; receiving a selection of the medical procedure for evaluation from the medical professional on the collaborative medical platform; receiving a selection of a framework for evaluating the medical professional based on the medical procedure on the collaborative medical platform, wherein the framework includes categories corresponding to different competencies or skills applicable to performing the medical procedure; receiving a selection of the medical professional to be evaluated from the medical professional on the collaborative medical platform; receiving a selection of a segment of data describing the performance of the medical procedure by the medical professional, wherein the selection includes a portion of the data describing the performance of the medical procedure between a start time and an end time; obtaining evaluation results from the medical professional to be evaluated, wherein the evaluation results include evaluation values ​​for each category of the framework from the medical professional to be evaluated; and presenting the evaluation results to the medical professional. Claim 2 A method according to claim 1, further comprising the step of selecting an educational content item associated with at least one baseline criterion that is not satisfied by an evaluation value for a category of the framework, wherein the educational content item is associated with a type of medical procedure; and the step of presenting the selected educational content item to the medical practitioner. Claim 3 A method according to paragraph 2, wherein the step of selecting the educational content item associated with at least one baseline criterion that is not satisfied by the evaluation value for the category of the framework, the step of selecting the educational content item associated with the type of medical procedure comprises the step of selecting the educational content item having a baseline criterion that specifies a value for the category of the framework that is at least greater than the evaluation value for the category of the framework by a threshold amount. Claim 4 A method according to paragraph 2, comprising the step of selecting an educational content item associated with at least one baseline criterion not satisfied by the evaluation value for the category of the framework, wherein the educational content item is associated with the type of medical procedure; the step of determining an aggregate value for the category of the framework based on the evaluation result for the category of the framework for the medical practitioner and one or more previously determined evaluation results for the category of the framework; and the step of selecting an educational content item associated with at least one baseline criterion not satisfied by the aggregate value for the category of the framework, wherein the educational content item is associated with the type of medical procedure. Claim 5 A method according to claim 1, further comprising the step of applying an evaluation model to a segment of data describing the performance of the medical procedure and to a framework, wherein the evaluation model generates a predicted evaluation value for a category of the framework based on the segment of data describing the performance of the medical procedure; the step of scoring the evaluation model using a loss function based on the difference between the predicted evaluation value for a category of the framework and the evaluation value for a category of the framework; and the step of updating one or more parameters of the evaluation model by backpropagation based on the scoring step. Claim 6 A method according to claim 1, wherein the data describing the performance of the medical procedure comprises video data captured during the performance of the medical procedure and telemetry data captured by one or more sensors during the performance of the medical procedure. Claim 7 A method according to claim 1, wherein the evaluation results further include one or more comments from the evaluating medical practitioner, and the comments are associated with a specific time in the segment of the data describing the performance of the medical procedure. Claim 8 A method according to claim 1, wherein the step of receiving a selection of a segment of data describing the performance of the medical procedure by the medical practitioner comprises: receiving a selection of the start time within the segment of data describing the performance of the medical procedure by the medical practitioner from the medical practitioner; and determining the end time by the collaborative medical platform by incrementing the start time by a specific time interval. Claim 9 A non-transient computer-readable storage medium for storing instructions for evaluating a medical professional performing a medical procedure through an online collaborative medical platform, wherein, when executed by one or more processors, the instructions cause the one or more processors to receive a request for evaluation from the medical professional on the collaborative medical platform; receive a selection of the medical procedure for evaluation from the medical professional on the collaborative medical platform; receive a selection of a framework for evaluating the medical professional based on the medical procedure on the collaborative medical platform, wherein the framework includes categories corresponding to different competencies or skills applicable to performing the medical procedure; receive a selection of the medical professional to be evaluated on the collaborative medical platform from the medical professional; receive a selection of a segment of data describing the performance of the medical procedure by the medical professional, wherein the selection includes a portion of the data describing the performance of the medical procedure between a start time and an end time; and obtain evaluation results from the medical professional to be evaluated, wherein the evaluation results include evaluation values ​​for each category of the framework from the medical professional to be evaluated A non-transient computer-readable storage medium that enables the performance of steps including: presenting the evaluation results to the medical professional. Claim 10 A non-transient computer-readable storage medium that further stores instructions, wherein, when executed by one or more processors, the one or more processors perform steps including: selecting an educational content item associated with at least one baseline criterion that is not satisfied by an evaluation value for a category of the framework, wherein the educational content item is associated with a type of medical procedure; and presenting the selected educational content item to the medical practitioner. Claim 11 A non-transient computer-readable storage medium according to claim 10, wherein selecting the educational content item associated with at least one baseline criterion not satisfied by the evaluation value for the category of the framework, wherein the educational content item is associated with the type of medical procedure, and selecting the educational content item comprises selecting the educational content item having a baseline criterion that specifies a value for the category of the framework that is at least greater than the evaluation value for the category of the framework by a threshold amount. Claim 12 A non-transient computer-readable storage medium comprising, in paragraph 10, selecting the educational content item associated with at least one baseline criterion not satisfied by the evaluation value for the category of the framework, wherein the educational content item is associated with the type of medical procedure; determining an aggregate value for the category of the framework based on the evaluation result for the category of the framework for the medical practitioner and one or more previously determined evaluation results for the category of the framework; and selecting the educational content item associated with at least one baseline criterion not satisfied by the aggregate value for the category of the framework, wherein the educational content item is associated with the type of medical procedure. Claim 13 A non-transient computer-readable storage medium further storing instructions that, when executed by one or more processors, cause the one or more processors to perform steps including: applying an evaluation model to a segment of data describing the performance of the medical procedure and to a framework, wherein the evaluation model generates a predicted evaluation value for a category of the framework based on the segment of data describing the performance of the medical procedure; scoring the evaluation model using a loss function based on the difference between the predicted evaluation value for a category of the framework and the evaluation value for a category of the framework; and updating one or more parameters of the evaluation model by backpropagation based on the scoring. Claim 14 In claim 9, the data describing the performance of the medical procedure comprises video data captured during the performance of the medical procedure and telemetry data captured by one or more sensors during the performance of the medical procedure, in a non-transient computer-readable storage medium. Claim 15 In claim 9, the evaluation results further include one or more comments from the evaluating medical practitioner, and the comment is associated with a specific time in the segment of the data describing the performance of the medical procedure, in a non-transient computer-readable storage medium. Claim 16 A non-transient computer-readable storage medium according to claim 9, wherein receiving a selection of a segment of data describing the performance of the medical procedure by the medical practitioner comprises receiving from the medical practitioner a selection of the start time within the segment of data describing the performance of the medical procedure by the medical practitioner; and determining the end time by the collaborative medical platform by incrementing the start time by a specific time interval. Claim 17 A method for evaluating a medical professional performing a medical procedure through an online collaborative medical platform, comprising: receiving a request for evaluation from the medical professional on the collaborative medical platform; receiving a selection of the medical procedure for evaluation from the medical professional on the collaborative medical platform; receiving a selection of a framework for evaluating the medical professional based on the medical procedure on the collaborative medical platform, wherein the framework includes categories corresponding to different competencies or skills applicable to performing the medical procedure; receiving a selection of a segment of data describing the performance of the medical procedure by the medical professional, wherein the selection includes a portion of the data describing the performance of the medical procedure between a start time and an end time; and generating evaluation results by the collaborative medical platform including predicted evaluation values ​​for categories of the framework by applying an evaluation model to a combination of the framework and the segment of data describing the performance of the medical procedure, wherein the evaluation model A step of acquiring a plurality of training examples, wherein each training example includes a training segment of data describing the performance of a training medical procedure and a training framework, and each training example has a label indicating an evaluation value for a category based on said training framework and said training segment; A step of applying the evaluation model to each training example to generate a predicted evaluation value for the category based on the training segment and the training framework; A step of scoring the evaluation model using a loss function and the labels of the training examples; and A method comprising: a step of generating by the collaborative medical platform, which is trained by a step of updating one or more parameters of the evaluation model by backpropagation based on a step of scoring until one or more criteria are satisfied; and a step of presenting the evaluation results to the medical professional. Claim 18 A method according to claim 17, further comprising the step of selecting an educational content item associated with at least one baseline criterion that is not satisfied by an evaluation value for a category of the framework, wherein the educational content item is associated with a type of medical procedure; and the step of presenting the selected educational content item to the medical practitioner. Claim 19 In claim 17, the method wherein the data describing the performance of the medical procedure comprises video data captured during the performance of the medical procedure and telemetry data captured by one or more sensors during the performance of the medical procedure. Claim 20 In paragraph 17, the evaluation results further include one or more comments from the medical practitioner evaluating the medical procedure, and the comment is associated with a specific time in the segment of the data describing the performance of the medical procedure.

Images