Assessing performance of a medical procedure by a medical practitioner via an online collaborative medical platform

Abstract

Collaborative healthcare platforms facilitate remote collaboration related to medical procedures during the preoperative, intraoperative, and postoperative phases of a medical case. The platform receives assessment requests from practitioners, along with selections of a framework for the assessment and data snippets describing the execution of the medical procedure. The platform can then send the assessment request, including the selected framework and snippets, to the assessing practitioner who has chosen the assessment results. Alternatively, the platform applies an assessment model to the selected framework and snippets to automatically generate assessment results. These results can be used to select educational content items for the practitioner.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefits of U.S. Provisional Patent Application No. 63 / 605,879, filed December 4, 2023; U.S. Provisional Patent Application No. 63 / 641,754, filed May 2, 2024; U.S. Provisional Patent Application No. 63 / 661,015, filed June 17, 2024; U.S. Provisional Patent Application No. 63 / 661,858, filed June 19, 2024; U.S. Provisional Patent Application No. 63 / 717,950, filed November 8, 2024; U.S. Provisional Patent Application No. 63 / 718,000, filed November 8, 2024; and U.S. Provisional Patent Application No. 63 / 719,015, filed November 11, 2024, the contents of which are each incorporated herein by reference. Background Technology Technical Field

[0003] The described implementation relates to a system and method for evaluating the performance of practicing physicians in performing medical procedures based on data describing medical procedures captured by a collaborative healthcare platform.

[0004] Description of related technologies

[0005] The different techniques employed by a physician during a surgical procedure affect patient outcomes. To maintain or improve patient outcomes, data describing the surgical procedure are reviewed after its completion to identify techniques used during the procedure that can be improved or refined by the physician. In many routine settings, the assessing physician is prompted to evaluate the physician's performance of the surgical procedure against one or more frameworks that specify guidelines for the techniques or skills used during the procedure. For example, the assessing physician compares the physician's performance of the surgical procedure to the standards specified in the framework to evaluate the physician's performance. This assessment can be used to select educational content for the physician to improve one or more skills or techniques. Similarly, the assessment can be used to evaluate changes in one or more aspects of the physician's performance of the surgical procedure over time. However, the assessing physician evaluates the physician's performance after the surgical procedure has been performed. Therefore, the assessing physician must rely on their recollection of the surgical procedure and the physician's performance of the surgical procedure to evaluate the physician. This reliance on assessing a physician’s memory of medical procedures increases the difficulty of evaluating a physician’s ability to identify specific areas for improvement. Attached Figure Description

[0006] Figure (FIG.) 1 is an example implementation of a computing environment for electronically assisted medical surgery.

[0007] Figure 2 This is a block diagram of an example architecture for a collaborative healthcare platform.

[0008] Figure 3A This shows a first view of a sample physician dashboard associated with a collaborative healthcare platform.

[0009] Figure 3B A second view of a sample physician dashboard associated with a collaborative healthcare platform is shown.

[0010] Figure 4 This example physician dashboard displays educational content items to physicians associated with a collaborative healthcare platform.

[0011] Figure 5 This is an example implementation of a case-sharing interface associated with sharing medical records in a collaborative healthcare platform.

[0012] Figure 6 This is an example implementation of a case dashboard associated with a set of cases in a collaborative healthcare platform.

[0013] Figure 7 This is a sample remote presentation interface associated with a collaborative healthcare platform.

[0014] Figure 8 This is another example of a remote presentation interface associated with a collaborative healthcare platform.

[0015] Figure 9 This is an example analytics dashboard associated with a collaborative healthcare platform.

[0016] Figure 10 This is a sample video interface associated with a collaborative healthcare platform.

[0017] Figure 11 This is a sample evaluation interface associated with a collaborative healthcare platform.

[0018] Figure 12 This is a sample framework interface associated with a collaborative healthcare platform.

[0019] Figure 13 This is a sample evaluator selection interface associated with a collaborative healthcare platform.

[0020] Figure 14 This is a sample segment selection interface associated with the collaborative healthcare platform.

[0021] Figure 15 This is a sample evaluation results interface associated with a collaborative healthcare platform.

[0022] Figure 16 This is a flowchart of an example implementation plan for a method by which licensed physicians obtain assessments of their performance of medical procedures through a collaborative medical platform. Detailed Implementation

[0023] The accompanying drawings and the following description illustrate certain embodiments only by way of example. Those skilled in the art will readily recognize from the following description alternative embodiments that can be employed without departing from the principles described herein. Reference will now be made to several embodiments, examples of which are shown in the accompanying drawings. Similar or analogous reference numerals may be used in the drawings, and may indicate similar or analogous functions, whenever practicable.

[0024] Collaborative healthcare platforms facilitate data exchange among remote practitioners related to medical cases during the preoperative, intraoperative, and postoperative phases. These platforms store or enable access to patient records, imaging data, video data, telemetry data from medical equipment, biometric sensor data from patients, and other medical information that may be obtained before, during, or after a medical procedure. Based on data describing the execution of a medical procedure (which may include video or telemetry data obtained during the procedure), the platform can facilitate the evaluation of a practitioner's performance. This includes, for example, the practitioner selecting a medical procedure, a framework for evaluating the practitioner, the evaluation of the practitioner, and data snippets describing the execution of the medical procedure. The platform provides the evaluating practitioner with the data snippets and framework describing the execution of the medical procedure, who then creates an evaluation result, which the platform presents to the practitioner. Alternatively, the platform applies one or more evaluation models to a combination of data snippets and frameworks describing the execution of the medical procedure, whereby the evaluation models generate a predictive portion of the evaluation result based on the data snippets and frameworks describing the execution of the medical procedure. In addition, collaborative healthcare platforms can select educational content for licensed physicians based on their assessment results.

[0025] Figure 1An example implementation of a computing environment 100 for a collaborative healthcare platform 140 is illustrated. The collaborative healthcare platform 140 may include one or more servers coupled via a network 130 to client devices 150, medical equipment 160, and various third-party servers 170 associated with users 155 of the collaborative healthcare platform 140. The collaborative healthcare platform 140 facilitates the collaborative exchange of data among practitioners, patients, administrators, or other users 155 via client devices 150 to support the preoperative, intraoperative, and postoperative phases of medical cases. The collaborative healthcare platform 140 may also facilitate access to telemetry data from medical equipment 160, including, for example, real-time video, images, biometric sensing data, equipment control and / or status signals, which can be used in conjunction with performing medical procedures and managing patient cases. Furthermore, the collaborative healthcare platform 140 may facilitate access to various third-party servers 170 providing external services such as, for example, electronic medical record (EHR) services, telemedicine services, operating room scheduling, data analytics services, etc.

[0026] To further support the preoperative phase of medical cases, the collaborative healthcare platform 140 can select one or more reference content items to present to practicing physicians before performing medical procedures. In various implementations, the collaborative healthcare platform 140 maintains a store or library of reference content items from which it selects reference content items for use by the practicing physician. Alternatively or additionally, one or more third-party servers 170 maintain reference content items, and the collaborative healthcare platform 140 selects one or more reference content items from the third-party servers 170. Reference content items for the practicing physician can be retrieved from a combination of one or more third-party servers 170 and the collaborative healthcare platform 140. The collaborative healthcare platform 140 utilizes data from the practicing physician's user profile to select one or more reference content items for the practicing physician.

[0027] To support the intraoperative phase of medical cases, the collaborative medical platform 140 can facilitate the presentation of various information to support the surgery, such as preoperative images, models, patient data, equipment information, or other data. The collaborative medical platform 140 can also facilitate telepresence sessions, enabling one or more remote contributors to access video, images, 3D models, equipment telemetry data, or other data streams captured during the ongoing medical procedure. The collaborative medical platform 140 can also enable telemedicines to provide annotations or other comments related to real-time video, images, or 3D models associated with the surgery. The collaborative medical platform 140 tracks and stores all data from the surgery associated with the case identifier, including video, medical equipment telemetry, and collaborative comments, enabling subsequent access.

[0028] During the intraoperative phase, the collaborative medical platform 140 can present educational content about the medical procedure being performed to one or more physicians. The educational content describes the execution of the medical procedure, such as information about the techniques to be used, the movement of medical devices or equipment, the setup of medical equipment, or other relevant information. The collaborative medical platform 140 compares telemetry or video data of the medical procedure during the intraoperative phase with a baseline standard associated with the educational content and selects the educational content associated with the baseline standard that deviates from the telemetry or video data. The educational content may include instructions that, when executed by a medical device 160, modify one or more settings of that medical device based on the educational content, thereby simplifying adjustments to the operation of that medical device 160.

[0029] To support the post-operative phase of medical procedures, the collaborative healthcare platform 140 enables physicians connected to cases to collaboratively monitor data related to patient recovery. For example, the collaborative healthcare platform 140 can provide an interface for viewing health records associated with patient recovery and facilitate collaborative exchange among physicians through case-specific content feeds. The collaborative healthcare platform 140 can also perform various analyses related to the performed medical procedures based on data aggregation. These analyses can be used to support patient recovery, improve future surgeries, and track physician performance.

[0030] Educational content related to medical procedures can be selected and presented by the collaborative healthcare platform 140 to physicians performing the procedures during the postoperative phase. For example, indicators or analyses determined by the collaborative healthcare platform 140 for the medical procedure are compared with baseline standards for various educational contents. In various implementations, the collaborative healthcare platform 140 selects educational content associated with baseline standards that deviate from the indicators and presents the selected educational content to the physicians. For example, the collaborative healthcare platform 140 includes information identifying the selected educational content in generating one or more interfaces for presentation to physicians, thereby simplifying access to guidance information related to the medical procedure.

[0031] The collaborative healthcare platform 140 can intelligently utilize data collected during different stages of the same or other cases, including the preoperative, intraoperative, and / or postoperative phases. For example, annotations of images or 3D models, physician comments from content feeds, or other information obtained during the preoperative phase can be made available during the intraoperative phase to assist the performing physician in completing the surgery. Analytical data associated with postoperative data can be used to generate recommendations for future surgeries, such as educational content, to improve efficiency and / or outcomes.

[0032] The collaborative healthcare platform 140 can also facilitate functions such as managing clinical trials, promoting education and training and performance tracking, facilitating the broadcasting of medical-related demonstrations, and facilitating surgical scheduling. Advantageously, the collaborative healthcare platform 140 stores a complete record of medical cases (including video and telemetry from surgeries) in a centralized and standardized platform, which naturally allows for collaboration in an online environment where physicians can interact from different remote locations. The collaborative healthcare platform 140 can maintain data in a manner that complies with data privacy and the compliance obligations of practicing physicians and organizations.

[0033] The collaborative healthcare platform 140 can also employ various machine learning techniques to infer recommendations, insights, or other artificially generated contributions based on data collected within the platform. For example, the platform can generate recommendations for educational content relevant to a physician's review based on data captured during a medical procedure. For instance, the platform might select educational content for a physician based on telemetry data captured during a medical procedure. As another example, the platform might select educational content based on video data captured during a medical procedure. The educational content selected by the platform can be video, audio, text, or other data describing the execution of a medical procedure. Additionally or alternatively, educational content configuration instructions or configuration data may be provided for one or more medical devices 160.

[0034] The collaborative healthcare platform 140 can generate and present additional recommendations to licensed physicians based on stored information. For example, the platform can generate recommendations based on the type of surgery scheduled for the physician; in various implementations, these recommendations include case records associated with one or more historical cases captured in the platform, relating to previous performance of that type of surgery on patients in similar situations. If appropriate permissions are granted, physicians can review the entire case record through the platform, including preoperative information, videos, or other information from the surgery itself, as well as postoperative outcome information. In another example, the platform can intelligently generate recommendations to invite a specific licensed physician with relevant expertise, experience, and / or availability to collaborate on a case. Invitations can then be generated to the physician to enable access to and collaboration on the case during at least one of the preoperative, intraoperative, and postoperative phases. Furthermore, the platform can intelligently identify and present patient risk factors related to surgical execution, planning, and postoperative care. The collaborative healthcare platform 140 can also intelligently recommend educational content for training practicing physicians based on their individual tracking performance and various comparative analyses.

[0035] The collaborative healthcare platform 140 can be implemented using on-site computing or storage systems, cloud computing or storage systems, or combinations thereof, and can be implemented using local servers or cloud-based servers, which may include physical machines or virtual machines, or combinations thereof. Cloud-based servers may include private cloud systems, public cloud systems, hybrid public / private cloud systems, or combinations thereof. Therefore, the collaborative healthcare platform 140 can be local, remote, and / or distributed relative to the medical environment in which surgery is performed and relative to the client devices 150 providing user access. Furthermore, different parts of the collaborative healthcare platform 140 can execute on different remote servers, and the various system components of the collaborative healthcare platform 140 can be communicatively coupled via a network 130.

[0036] Client device 150 may include, for example, a mobile phone, tablet, laptop computer, or desktop computer, other computing device, or an application running thereon for accessing collaborative healthcare platform 140 via network 130. Client device 150 may provide access to various user interfaces, which may include a web-based interface accessed via a browser or an application interface accessed via an application, for viewing and / or editing information associated with collaborative healthcare platform 140. Client device 150 may include conventional computer hardware such as a display, input devices (e.g., a touchscreen), memory, a processor, and a non-transitory computer-readable storage medium storing instructions for execution by the processor to perform the functions described herein. Referring below to Figures 3 to 4... Figure 15 A more detailed description of an example of a user interface.

[0037] Third-party server 170 can facilitate various services utilized by collaborative healthcare platform 140. For example, third-party server 170 may include various EHR systems for managing patient records, robot control platforms for controlling surgical robots or other medical equipment, telepresence servers for facilitating telepresence services, patient scheduling systems, hospital information systems (HIS), or other servers. As another example, one or more third-party servers 170 may include educational content on various medical procedures, such as articles on various medical procedures, audio data related to medical procedures, video data related to medical procedures, setup or configuration details of medical equipment 160 used in medical procedures, or other descriptive information about medical procedures. Third-party server 170 may be implemented using various on-site computing or storage systems, cloud computing or storage systems (such as private cloud systems, public cloud systems, hybrid public / private cloud systems), or combinations thereof.

[0038] Medical device 160 may include various sensors, such as cameras or other imaging devices, biometric monitors, or other sensing devices that collect data associated with the medical procedure being performed. Sensor data may include physiological or biological signals (such as pulse rate, blood pressure, body temperature, etc.), video, electrical signals indicating the status of medical instructions, or other information. Cameras or image sensors may include still image cameras, video cameras, 3D imaging devices, or combinations thereof. Cameras may include fixed cameras in medical environments (e.g., operating rooms) or may include cameras integrated into medical devices, such as endoscopic cameras. Imaging systems may include computed tomography (CT) imaging systems, medical resonance imaging (MRI) systems, X-ray systems, or other imaging devices. Medical device may also include robotic devices that facilitate robot-assisted medical procedures. Robotic devices may include, for example, robotic arms or other computer-controlled mechanical devices that perform or assist medical procedures. Robotic devices may be pre-programmed to perform a specific set of steps or tasks and / or may be manually controlled by an operator. Telemetry data associated with the robotic device may include force data, position data or other sensor data, control signals, fault conditions, or other data related to the operation of the robotic device during surgery. Medical device data may be streamed to the collaborative medical platform 140 in real time, or it may be stored on a third-party server 170 and uploaded to the collaborative medical platform 140 later.

[0039] Network 130 includes communication paths for communication between collaborative medical platform 140, medical equipment 160, client device 150, and third-party server 170. Network 130 may include one or more local area networks (LANs) and / or one or more wide area networks (WANs), including the Internet. Network 130 may also include one or more direct wired or wireless connections (e.g., Ethernet, WiFi, cellular protocols, WiFi Direct, Bluetooth, Universal Serial Bus (USB), or other communication links).

[0040] Figure 2 This is a block diagram illustrating an example architecture of an implementation of the collaborative healthcare platform 140. Figure 2 In one implementation, the collaborative healthcare platform 140 includes a data ingestion module 205, an entity management module 210, an interface management module 215, a medical intelligence module 220, a telepresence module 225, an analysis module 230, a physician education module 235, a demonstration module 240, an application integration module 245, a video library 250, a connection graph repository 255, a user profile repository 260, and a patient data repository 265. In other implementations, the collaborative healthcare platform 140 includes... Figure 2 The function blocks shown are different function blocks or additional function blocks. Furthermore, in some embodiments, a single function block provides... Figure 2The functions of the multiple function boxes shown.

[0041] While in one implementation, the illustrated functional blocks may be executed entirely within the collaborative healthcare platform 140, alternative implementations may include various modules or discrete functionalities of modules executed by one or more third-party servers 170. Here, the collaborative healthcare platform 140 may interact with the third-party server 170 via an application programming interface (API) to enable the collaborative healthcare platform 140 to request and utilize services provided by the third-party server 170 to facilitate any of the functionalities described herein. For example, in one implementation, electronic health records may be provided by the third-party server 170. Here, the collaborative healthcare platform 140 may query relevant data from the third-party server 170, but does not necessarily store complete patient records locally. Furthermore, the third-party server 170 may facilitate services such as telepresence sessions, presentation creation, access to video resources, 3D model generation, or other aspects of the functionality of the collaborative healthcare platform 140 described herein.

[0042] Data ingestion module 205 ingests various medical data used by collaborative healthcare platform 140. Data ingestion module 205 may be electrically coupled to one or more external servers, databases, or other data sources that provide medical data. Medical data may include, for example, patient profile data (e.g., demographic information, health history, etc.), medical professionals (e.g., expertise, experience, etc.) or facilities, information about medical conditions, surgeries and medications, information about robotic systems, imaging systems, interventional tools or other medical devices, information about postoperative outcomes, or other medical information discussed herein.

[0043] Data ingestion module 205 can aggregate data from various input data sources. For example, data ingestion module 205 can obtain medical data from a conventional electronic health record (EHR) system. Here, data ingestion module 205 can perform various preprocessing steps to normalize the data relative to a standardized format used by the collaborative healthcare platform 140. For example, medical records can be organized in a database structure that includes values ​​(strings, numeric values, binary values, or other data types) assigned to each of a set of predefined information fields.

[0044] The data acquisition module 205 can also interface with one or more imaging systems to acquire preoperative, intraoperative, or postoperative images, videos, or 3D models associated with the patient. For example, the data acquisition module 205 can acquire and store X-ray images, magnetic resonance imaging (MRI) images, computed tomography (CT) scan images, visible light images, near-infrared fluorescence (NIRF) images, or other medical images, videos, or 3D models derived from them. Image data may also include image or video data from one or more cameras (such as one or more overhead cameras and / or one or more endoscopic cameras) present in the medical environment where the medical procedure is being performed. Imaging data may include associated metadata, such as telemetry data from one or more medical devices used to perform the medical procedure, annotations or comments associated with videos received from one or more physicians associated with the medical procedure, segmentation data associated with dividing the video into segments related to different steps of the procedure, or other information related to the image or video data.

[0045] To simplify the subsequent retrieval and review of medical surgery videos and associated metadata, the data ingestion module 205 can perform various preprocessing and indexing operations on the content and associated metadata. For example, the data ingestion module 205 can index the medical surgery video with associated metadata to associate different metadata with different parts of the video, synchronize videos associated with the same medical surgery, or perform various encoding or reformatting operations on the video data. The video can also be automatically segmented and indexed into video clips corresponding to different steps of the surgery.

[0046] The data acquisition module 205 can also be integrated with various robotic platforms or other medical devices to obtain telemetry data associated with surgery. For example, the data acquisition module 205 can obtain various sensor data, identification information associated with medical devices, control data associated with controlling robotic platforms or other medical devices, or other data generated from medical devices associated with the performed medical procedure from sensors used during the medical procedure.

[0047] The data ingestion module 205 may also provide an interface accessible via the client device 150 for ingesting data directly input into the collaborative healthcare platform 140. For example, the data ingestion module 205 may present various forms or free-form input elements to enable the input of operationally relevant medical information.

[0048] In one implementation, the data ingestion module 205 can manage data in a manner consistent with various compliance and privacy policies. For example, the data ingestion module 205 can remove or edit portions of the received data to protect patient privacy when the data is used for purposes that do not require patient identification.

[0049] Entity management module 210 manages the presentation of entity pages associated with different entities attached to the collaborative healthcare platform 140 and manages the connections between entities. Entities may include, for example, users 155 (which may be practicing physicians, patients, administrators, etc.), medical records associated with surgery, facilities, medical equipment 160, documents or media content, events (e.g., meetings), presentations, training modules, or other data objects. Entity pages may include web pages accessible via a web browser on client device 150, or pages of desktop or mobile applications installed on client device 150.

[0050] Each entity page for an entity enables the viewing of information associated with the entity and / or interaction with the entity. For example, each user 155 of the collaborative healthcare platform may have a dedicated page providing information about user 155, such as identification information, roles (e.g., surgeon, nurse, practitioner, administrator, patient, etc.), profile information (e.g., resume, certificates, etc.), assigned medical records, surgical history, connections with other users or cases, scheduling information, or other user-specific data. A patient's entity page (regardless of whether the patient is a user 155 of the collaborative healthcare platform 140) may include patient profile information, health history, planned surgeries, risk factors, or medical information associated with the patient. A medical case's entity page may include information about the patient associated with the case, descriptive information about the medical procedure associated with the case (such as the type of medical procedure), the medical environment in which the medical procedure will be performed, other descriptive information about the medical procedure, the status of the procedure (e.g., preoperative, intraoperative, or postoperative), or other information related to the medical case. The page may also include various interactive elements (e.g., content feeds) that enable users to share and interact with data associated with the entity, as will be described further below.

[0051] The entity management module 210 also receives pages and associated data from the collaborative healthcare platform 140, organizing them into a connection graph (stored in the connection graph repository 255). This connection graph captures the relationships between different entities and associated data. Some connections can be configured as default connections, while others can be created based on specific actions from user 155. For example, user 155 can be connected by default to other users 155 within the same organization (with at least viewing permissions). Alternatively, a connection can be generated only when user 155 explicitly invites another user 155 to connect and user 155 accepts the connection request. Similarly, connections between physicians and medical cases can be created by default or in response to invitations to create connections. For example, a default connection can be created between an entry for a planned medical procedure and the physician assigned responsibility for that procedure. Alternatively, all physicians within an organization or related department can be connected by default to a planned procedure. In other scenarios, a user can share a medical case with one or more other physicians to generate a connection request that invites other physicians to collaborate on the medical case. Accepting a connection request allows a connection to be created between the invited physician and the medical case. Supplementary connections can also be generated automatically (e.g., between the owner of a surgery and the invited contributor). Furthermore, connections can be created between user 155 and individual videos, files, presentations, or other data objects. For example, user 155, who created or owns a video, can share the video with one or more other users 155 to grant access to the video.

[0052] Connections between entities can be of different types and can be managed by different permissions. Typically, a page can be accessed only by users with appropriate access rights. Different permission levels can define different access levels for different pages. For example, depending on the user-specific permissions for a particular page, users may be allowed or prevented from accessing, editing, commenting on, annotating, deleting, or performing other modifications to data. In one implementation, a page may have a page owner with the highest access level. Typically, a practicing physician may be the owner of their own profile page and the pages for the surgeries they primarily perform. Pages associated with facilities, medical equipment, or other entities may be variably owned by the assigned practicing physician. Non-owners may have different access levels to the page, depending on the configured permissions. Permissions can be granted by the page owner or by another user with appropriate permissions to assign or relinquish permissions to other users.

[0053] Based on the different connections available to different users 155, the collaborative healthcare platform 140 can provide a personalized experience for each user 155. For example, after logging into the collaborative healthcare platform 140, user 155 may be presented with a personalized interface that includes their connections with other users 155, medical records, videos, demonstrations, or other content hosted by the collaborative healthcare platform 140.

[0054] Interface management module 215 manages the content associated with various interfaces hosted by collaborative healthcare platform 140 and accessible via client device 150. As described above, interface management module 215 can manage pages associated with various entities managed by collaborative healthcare platform 140, including users 155 (which may include physicians, patients, administrators, etc.), medical cases associated with surgeries, facilities, medical equipment 160, documents or media content, events (e.g., meetings), demonstrations, training modules, or other data objects. Access to different pages by a specific user 155 may depend on the user's connections and permissions configured in connection graph repository 255. Furthermore, patient data may be pseudonymous for viewing by certain other users (depending on the type of connection and / or permissions), making the patient data not attributable to a specific individual.

[0055] The medical case page associated with the medical case may include information organized into preoperative, intraoperative, and postoperative phases. In the preoperative phase, the medical case page may include information about the patient, the surgery being performed, and the surgeon performing the surgery. Interface management module 215 may also provide access to various analytical information (e.g., generated by analysis module 230 described below), such as the patient's risk factors, the surgeon's experience / expertise, the outcome of the planned surgical type, or other data. In the intraoperative phase, the medical case page may provide access to a telepresence session, enabling remote collaborators to collaborate remotely on the ongoing surgery. In the postoperative phase, the medical case page may include information about the patient's treatment plan, risk factors, follow-up visits, or other postoperative information.

[0056] Some entity pages within the collaborative healthcare platform 140 may include content feeds to facilitate collaboration among users 155. Content feeds may include a variety of content (e.g., posts), such as text-based comments, images, videos, 3D models, or other multimedia content related to medical cases. Content may be posted directly to pages associated with medical cases, or posts may include links to content stored by the collaborative healthcare platform 140 or on external servers. Posts may be grouped into hierarchical dialogues that track relationships between posts. For example, a post may be created as an original post (which starts a new dialogue) or a reply to an existing post (which becomes part of a dialogue).

[0057] In the example use case, user 155 can invite one or more other users 155 to collaborate on a medical case, thereby gaining access to the case page of the medical case. Content feeds on the page allow collaborating users 155 to post to the case page in association with the medical case. Therefore, content feeds can enable discussions about the surgery to be performed, risks, best practices, or other information that may be useful to the physician performing the surgery. Furthermore, contributing user 155 can post videos or 3D models (or links to content) related to historical surgeries performed on patients with similar conditions. Additionally, contributing user 155 can share links to entity pages associated with potentially relevant past surgeries, allowing the performing physician to view historical content feeds associated with those surgeries. When shared with other users, patient data can optionally be pseudonymous (depending on the type of connection and / or permissions) so that patient data cannot be attributed to a specific individual.

[0058] Furthermore, as discussed in further detail below, content feeding can be utilized in relation to the ongoing surgery during a live telepresence session. Here, content feeding can be presented as a live chat window that allows contributors to comment, share videos, images, or other media during the surgery, provide links to relevant resources, or otherwise contribute content during the surgical procedure.

[0059] In the postoperative phase, contributors can use content feeds to discuss postoperative treatment, patient recovery, risk management, or other information related to patient recovery. Figure 7 An example of content feeding is provided, which will be described in further detail below.

[0060] The medical intelligence module 220 generates medical intelligence data, which can be automatically added to content feeds or otherwise made available in the context of the collaborative healthcare platform 140. For example, the medical intelligence module 220 can automatically contribute posts to content feeds that an AI agent infers to be relevant medical cases. Human-generated posts can mimic posts provided by human contributors and can include text-based comments, multimedia, links, etc. Medical intelligence data can be generated during the preoperative phase, during surgery, or during the postoperative phase.

[0061] In an example implementation, the medical intelligence module 220 may include one or more machine learning models trained to generate content that the model infers to be relevant to a specific medical case or more generally relevant to the user 155. In one implementation, the machine learning model generates an embedding of the medical case based on descriptive information about the medical procedure, characteristics of the patient to be operated on, characteristics of the practicing physician performing the procedure, posts in the content feed, or other information available in the collaborative healthcare platform 140. The medical intelligence module 220 determines a measure of similarity (e.g., cosine similarity, dot product) between the embedding of the medical case and the embeddings of other content available in the collaborative healthcare platform 140 and which may be included in automated posts. The medical intelligence module 220 may then generate posts and / or select content for posts based on the embedding similarity. The medical intelligence module 220 may also employ various large language models (LLMs) to analyze text-based content associated with the medical case and manually generate relevant natural language content for the content feed. Machine learning models may also include one or more neural networks (such as convolutional neural networks (CNNs), artificial neural networks (ANNs), residual neural networks (ResNets), or recurrent neural networks (RNNs)), regression-based models, generative models, or other types of machine learning models capable of achieving the functionality described herein.

[0062] In an example use case, the medical intelligence module 220 can identify one or more historical medical cases similar to the current medical case and automatically generate links to the relevant case pages. The physician can then view data from videos, models, or other records associated with the relevant medical case to help prepare for surgery. In other examples, the medical intelligence module 220 can automatically respond to questions posed by users in content feeds. For example, the medical intelligence module 220 can operate like a chatbot intelligently responding to text-based queries. In another implementation, the medical intelligence module 220 can generate recommendations for inviting specific physicians to collaborate on a medical case based on relevant expertise and experience. The user can then select recommended collaborators to collaborate on the medical case based on these human-generated recommendations.

[0063] The telepresence module 225 facilitates telepresence sessions during surgery. A telepresence session can be joined by one or more collaborators who have been invited to collaborate on a medical case and enable other users 155 to remotely access video, telemetry data from one or more medical devices, or other real-time data captured during the medical procedure. As described above, content feeds can also be displayed in conjunction with the telepresence session, allowing contributors to comment on or share surgery-related multimedia or links.

[0064] The telepresentation module 225 also enables contributors to provide real-time annotations on images, videos, 3D models, or other visual content related to anatomical structures in the context of an ongoing surgery. For example, contributors can tag locations within the visual content in association with the provided annotations. The telepresentation module 225 also allows contributors to add overlays, highlights, or other visual indicators during an ongoing telepresentation session.

[0065] In one implementation, the telepresence module 225 enables a remote contributor to control the medical device 160. For example, the remote contributor can access a control interface that provides control elements for controlling the position or orientation of a camera, controlling a robotic arm, configuring sensing devices, or performing other control functions of the medical device.

[0066] After the surgery is completed, the telepresence module 225 can store recorded video, telemetry data, content feeds, annotations, and other captured data associated with the surgery. This information can later be accessed by the user 155 of the collaborative healthcare platform 140 (with appropriate permissions) and / or can be utilized by the medical intelligence module 220 to further train machine learning models and / or generate inferences.

[0067] Analysis module 230 facilitates the generation of various statistical, metric, or other analyses associated with information stored in collaborative healthcare platform 140. Analyses can typically be created based on a set of filtering parameters that produce a subset of data records for aggregation, and a combinatorial function that specifies how the filtered data should be combined. Filtering parameters can filter medical data based on data fields such as patient data, physician data, facilities, surgical types, and medical equipment used. Combination functions can include, for example, averaging functions, median functions, histogram functions, or other functions. A specific analysis function can result in a single output value or a series of values ​​across one or more dimensions. A series of outputs can be visually represented in tables, charts, graphs, or other visual outputs.

[0068] For example, analysis module 230 can generate a metric describing the average length of time a particular practitioner or group of practitioners completes a medical procedure. The average time for various procedures performed by the same practitioner or group of practitioners can be presented alongside similar metrics for other practitioners for comparative purposes. In another example, analysis module 230 can generate a metric describing the number of times a practitioner has historically performed a particular type of medical procedure. Such counts can be further aggregated to indicate the percentage of each different type of medical procedure performed out of the total number of procedures performed by the practitioner.

[0069] In another implementation, the analysis module 230 can generate analyses based on interactions between practicing physicians in the collaborative healthcare platform 140. For example, statistics can be derived based on the count of posts, comments, or other content contributed to the collaborative healthcare platform 140 by practicing physicians. Such analyses can be expressed based on the count of interactions, the frequency of interactions, or other aggregations. These analyses can also be aggregated separately based on whether the interactions are related to the preoperative, intraoperative, or postoperative stages of surgery.

[0070] In one implementation, the analysis module 230 may generate analyses based on specific filtering and / or combination functions specified by user 155 of the collaborative healthcare platform 140. Additionally, the analysis module 230 may include various preset analyses that can be generated without receiving specific user input. Furthermore, in some implementations, the healthcare intelligence module 220 may automatically generate analyses whose inferences will be relevant to a specific user 155.

[0071] In some implementations, the analysis module 230 can generate analyses based on any aspect of collective case data, including preoperative data, telepresence session data (including recorded video, telemetry data, content feeds during the session, etc.), and postoperative data. Analyses associated with the telepresence session data may include performing various video processing, content recognition, or other advanced image processing techniques to extract useful information from the video. Furthermore, the analysis module 230 can utilize various medical intelligence data generated from the medical intelligence module 220 to generate analyses.

[0072] The physician education module 235 manages and stores training data for practicing physicians associated with medical procedures. In various implementations, the training data includes educational content that includes descriptive information about medical procedures or parts thereof. Example educational content includes training videos related to performing medical procedures, articles about performing medical procedures, articles or videos about using one or more medical devices 160 in medical procedures, articles or videos about using one or more medical instruments in medical procedures, best practices for medical procedures, training manuals for medical procedures, instructional materials for one or more medical instruments used in medical procedures, digital training modules, webinars, audio data about medical procedures (e.g., podcasts about medical procedures), or other information used to train practicing physicians associated with medical procedures.

[0073] In various implementations, the educational content also includes configuration data or configuration instructions for one or more medical devices 160 used in one or more medical procedures. For example, the educational content includes a set of configuration instructions for configuring or calibrating a robotic arm or other medical device 160 used in a medical procedure. The configuration instructions may include one or more settings for the medical device 160. Example settings include: one or more limits on the amount of force applied by the medical device 160, one or more limits on the range of motion of the medical device 160, one or more limits on the amount of energy supplied by the medical device 160, an identifier of the operating mode of the medical device 160, or values ​​for one or more other settings of the medical device 160. As another example, the educational content includes a set of instructions that, when executed by the medical device 160, cause the medical device 160 to perform a series of actions for calibration. Certain educational content can be executed by a medical device 160 to modify the values ​​of one or more settings of the medical device 160 or the operating mode of the medical device 160, thereby allowing one or more settings of the medical device 160 to be automatically modified via educational content items without requiring the physician to manually specify the values ​​of the settings of the medical device 160.

[0074] The physician education module 235 stores educational content as distinct educational content items, each comprising discrete portions of content, such as files. Each educational content item has one or more attributes that provide descriptive information about the educational content item. For example, attributes of an educational content item may identify one or more types of medical procedures associated with the educational content item, thereby allowing the identification of educational content items corresponding to different types of medical procedures. Other example attributes of an educational content item include: one or more practicing physicians associated with the educational content item (e.g., the physician performing the medical procedure associated with the educational content item, the physician creating the educational content item), the location associated with the educational content item (e.g., geographic location, specific medical facility), the time associated with the educational content item (e.g., the time when the educational content item was created), identifiers of one or more pieces of medical equipment 160 associated with the educational content item, identifiers of one or more medical devices used in the medical procedure associated with the educational content item, the format of the educational content item (e.g., audio, video, text), or other information describing the educational content item. In various implementations, educational content items may be stored locally by the collaborative healthcare platform 140 (e.g., in a video library 250 or another storage device) or retrieved from one or more third-party servers 170.

[0075] One or more educational content items may include reference cases, which are medical cases performed by a licensed physician in medical cases selected for use by other licensed physicians. For reference cases, physician education module 235 stores video data, telemetry data from medical device 160, or other data captured by collaborative healthcare platform 140 during the performance of the completed medical procedure. In various embodiments, reference cases include content feeds that include comments or other data obtained by collaborative healthcare platform 140 from contributors during the completed medical procedure. For reference cases, physician education module 235 pseudonyms patient data to prevent reference cases from including patient data that could be attributed to a specific patient. In some embodiments, pseudonymed patent data in reference cases identifies the scope of one or more types of patent data to maintain information about patents performed by another licensed physician while preventing the identification of a specific patient who performed the completed medical procedure.

[0076] Each educational content item is associated with one or more baseline standards. Different baseline standards specify values ​​from indicators used in performing the medical procedure, settings of a piece of medical equipment 160 used in the medical procedure, movement patterns of the piece of medical equipment 160 during the medical procedure, patterns of telemetry data acquired during the medical procedure, movement patterns of the practicing physician during the medical procedure, or other descriptive information about performing the medical procedure. Baseline standards specify standardized values ​​of indicators, standardized techniques or methods used in the medical procedure, or other standardized values ​​or techniques related to the medical procedure. The physician education module 235 maintains one or more baseline standards for different medical procedures, thus different educational content items correspond to different medical procedures. The attributes of the educational content item include an identifier for the type of medical procedure to indicate that the educational content item and its associated baseline standard correspond to the type of medical procedure. This allows the physician education module 235 to identify different baseline standards for different types of medical procedures.

[0077] In various implementations, one or more practitioners input baseline standards for a medical procedure into the physician education module 235. For example, a group of practitioners agree on values ​​for indicators, patterns of telemetry data, patterns of movement, or other information describing the execution of the medical procedure. The practitioners in this group input the agreed baseline standards into the physician education module 235 for storage associated with educational content items. This group of practitioners may be associated with a specific healthcare facility (e.g., a hospital, clinic) to provide facility-specific baseline standards. The physician education module 235 stores the healthcare facility's identifier as an attribute of the educational content item associated with the facility-specific baseline standards to indicate the baseline standards associated with a specific healthcare facility. Alternatively or additionally, the group of practitioners determining the baseline standards may not be associated with a specific healthcare facility but rather with a larger organization or standard institution, thus making the baseline standards for the medical procedure applicable to a variety of healthcare facilities. In various implementations, the physician education module 235 may store facility-specific baseline standards and more generally applicable baseline standards as different attributes of the educational content items. This allows the use of facility-specific baseline standards to enhance the more generally applicable baseline standards associated with the educational content items.

[0078] In various implementations, the physician education module 235 generates one or more baseline standards associated with educational content items by applying one or more trained machine learning models to metrics generated for multiple medical cases in which a type of medical procedure is performed by the analysis module 235. In various implementations, one or more trained machine learning models are also applied to telemetry data or video data captured by the telepresence module 225 during the medical cases in which this type of medical procedure is performed. For example, the machine learning model detects patterns in telemetry data captured during a particular type of medical procedure that occur in medical cases for which a specific value of a generated metric is generated, or for which the value of the generated metric is within a range of values. The specific value of the generated metric, or the range of values ​​of the generated metric, may correspond to one or more specific patient outcomes. For example, a specific value or a range of values ​​identifies a successful patient outcome for this type of medical procedure. In various implementations, one or more patterns of telemetry data detected at at least a threshold frequency during medical procedures occurring in medical cases are stored as baseline standards for educational content items associated with a particular type of medical procedure, for which the generated metric has a specific value or has a value within a specified range.

[0079] For example, applying a machine learning model to telemetry data identifies specific movement sequences of a medical device 160 detected at at least a threshold frequency during a specific type of completed medical procedure performed in a medical case. Indicators corresponding to positive outcomes are stored as baseline standards for educational content items corresponding to the movement of the medical device 160 during the specific type of medical procedure. The telemetry data describing the specific movement sequences of the medical device 160 can be stored in educational content items to specify the movement limits of the medical device 160 during the specific type of medical procedure, or to specify the limits of the forces exerted by the medical device 160 during the specific type of medical procedure. As another example, the captured telemetry data includes location data of a medical device 160 during the occurrence of that type of medical procedure in the medical case, where one or more indicators are associated with positive outcomes for the patient. The physician education module 235 stores educational content items associated with the type of medical procedure, which uses the location data from the captured telemetry data as a baseline standard. This allows the physician education module 235 to dynamically generate educational content items and associated baseline standards for a type of medical procedure based on telemetry data captured during the performance of that type of medical procedure over time, thereby simplifying the generation of educational content items for various medical procedures.

[0080] In other examples of generating educational content items from telemetry data, telemetry data from a medical device includes the dexterity of a practicing physician during a surgical procedure, wherein the dexterity information is stored as a baseline criterion in the educational content item in response to determining that the surgical procedure has a positive outcome. In various embodiments, the generated educational content item includes data for accessing a simulator of the medical device 160 used during the surgical procedure (e.g., an identifier of the simulator, one or more exercises or techniques to be performed on the identified simulator, etc.) to further refine the use of the medical device 160 in response to telemetry data from the practicing physician during the surgical procedure, which includes dexterity information deviating from the baseline criterion of the educational content item by at least a threshold amount. As another example, telemetry data from a medical device 160 includes tissue tension in a patient during a surgical procedure, wherein tissue tension is stored as a baseline criterion in the educational content item in response to the physician education module 235 determining that the surgical procedure has a positive outcome. The generated educational content items may include data for accessing a simulator of the medical device 160 used during a medical procedure (e.g., the simulator's identifier, one or more exercises or techniques to be performed on the identified simulator, etc.) to further refine the use of the medical device 160 in response to telemetry data from a practicing physician during the medical procedure, including hand dexterity information that deviates from the baseline standard of the educational content items by at least a threshold amount.

[0081] Additionally or alternatively, the physician education module 235 applies one or more machine learning models to video data captured during the performance of a specific type of medical procedure in previous medical cases to identify different pieces of medical equipment 160 used during the specific type of medical procedure, the movement of different pieces of medical equipment 160 during the specific type of medical procedure, the movement of the physician performing the specific type of medical procedure, or other information about the performance of the specific type of medical procedure. As further described above, applying machine learning models to video data of previous performances of a specific type of medical procedure detects movement patterns of the physician or a piece of medical equipment 160 during the performance of the specific type of medical procedure. Patterns or movements detected at at least a threshold frequency in the video data of completed medical procedures of a specific type performed in a medical case, with indicators corresponding to positive outcomes, are stored as baseline standards for one or more educational content items associated with the specific type of medical procedure. Such educational content items associated with the type of medical procedure and the baseline standards describing movement patterns include location data or other data describing the movement or positioning of the physician or a piece of medical equipment 160 during the specific type of medical procedure for later reference. Other information, such as depth-sensing data, the proximity of a medical device 160 to the patient's structure, the cross-sectional angle of the medical device 160 to the patient's structure, the path length of the medical device 160, tissue tension, the practitioner's dexterity, or other data, can be determined by the physician education module 235 based on video data and stored as baseline criteria in response to video data of medical procedures with threshold frequencies or in response to video data of medical procedures with indicators corresponding to positive outcomes. This allows the physician education module 235 to determine baseline criteria for a type of medical procedure based on video data of one or more medical procedures.

[0082] To select educational content items for practicing physicians, physician education module 235 compares data describing the execution of medical procedures performed by the practicing physician with baseline standards associated with various educational content items. In various implementations, after the practicing physician completes the medical procedure, physician education module 235 identifies educational content items associated with the type of medical procedure and compares the obtained information describing the medical procedure with one or more baseline standards associated with the identified educational content items. For example, physician education module 235 compares indicators generated by analysis module 230 from captured telemetry data, video data, or other data for the medical procedure with baseline standards associated with educational content items associated with the type of medical procedure. In response to an indicator differing from the baseline standard associated with an educational content item by at least a threshold amount, or otherwise failing to meet the baseline standard, physician education module 235 selects the educational content item associated with the baseline standard for demonstration to the practicing physician. For example, in response to determining that the time taken by a practicing physician to complete a medical procedure exceeds the average time taken to complete that type of medical procedure or exceeds the baseline time taken to complete that type of medical procedure, one or more educational content items are selected that are associated with that type of medical procedure and with a baseline standard specifying the time taken to complete that type of medical procedure. In some embodiments, the physician education module 235 selects one or more educational content items that are associated with the type of medical procedure and with a baseline standard, wherein the indicator determined for the medical procedure differs from the baseline standard by at least a threshold amount, thereby allowing the physician education module 235 to consider a specific amount of variance between the determined indicator and the baseline standard when selecting educational content items.

[0083] Alternatively or additionally, the physician education module 235 compares one or more patterns or data detected within telemetry data captured during the performance of a medical procedure with a baseline standard associated with the educational content item. The physician education module 235 selects educational content items associated with the type of medical procedure and associated with a baseline standard specifying a telemetry data pattern that differs from the captured telemetry data by at least a threshold amount. For example, the telemetry data includes depth-sensing data during the medical procedure, and the physician education module 235 selects educational content items associated with the type of medical procedure and associated with depth-sensing data that differs from the captured depth-sensing data by at least a threshold amount. In some embodiments, the selected educational content item includes information for accessing a simulator of the medical device 160 to be accessed by a practicing physician (e.g., the simulator's identifier, one or more exercises or techniques to be performed on the identified simulator, etc.). In another example, the telemetry data includes tissue tension data captured during the medical procedure, and the physician education module 235 selects educational content items associated with the type of medical procedure and associated with tissue tension data that differs from the captured tissue tension data by at least a threshold amount. In various implementations, the selected educational content items include information for accessing the simulator of the medical device 160 to be accessed by a practicing physician (e.g., the simulator's identifier, one or more exercises or techniques to be performed on the identified simulator, etc.).

[0084] Telemetry data from one or more sensors (e.g., sensors included in a medical device 160) can also describe the movement or positioning of the medical device 160 or medical instrument during a medical procedure, and the physician education module 235 selects educational content items that include a baseline standard from which the movement of a medical device or the positioning of a medical instrument in the telemetry data deviates from at least a threshold amount. For example, the telemetry data includes the path length of a medical device 160 during a medical procedure, and the physician education module 235 selects educational content items associated with the type of medical procedure and including a baseline standard specifying the path length of the medical device 160, where the path length in the captured telemetry data deviates from the baseline standard by at least a threshold amount. As another example, the telemetry data includes location data of a medical device 160 during a medical procedure, and the physician education module 235 selects educational content items associated with the type of medical procedure and including a baseline standard specifying the location data of the medical device 160, where the location data of the medical device 160 in the captured telemetry data deviates from the baseline standard by at least a threshold amount. In another example, the telemetry data includes the dexterity data of a practicing physician during a medical procedure, and the physician education module 235 selects educational content items associated with the type of medical procedure and including a baseline standard specifying the dexterity data of the medical device 160, wherein the dexterity data in the captured telemetry data deviates from the baseline standard by at least a threshold amount. In the foregoing example, the educational content items selected based on the telemetry data include information for (e.g., via the collaborative healthcare platform 140) accessing a simulator associated with the medical device 160 corresponding to the telemetry data, thereby providing the practicing physician with increased interaction with the medical device 160. Furthermore, educational content items selected based on the deviation of a medical device 160's positional data from the baseline standard may include one or more of the following: training videos associated with and describing the operation of the medical device 160, audio data describing the operation of the medical device 160, and information for accessing the simulator of the medical device 160 (e.g., simulator identifier, one or more exercises or techniques to be performed on the identified simulator, etc.). In some implementations, the captured telemetry data describes the usage pattern of a medical device 160 during a medical procedure. In response to determining that the usage pattern of the medical device 160 deviates from the baseline usage pattern of the medical device in the educational content item, the physician education module 235 selects an educational content item that may include baseline data describing the cost of the medical procedure based on the usage pattern and information on alternative usage patterns of the medical device 160 to reduce costs, or information describing recommended usage patterns of the medical device 160 or alternative usage of multiple medical devices 160 in this type of medical procedure.

[0085] In various implementations, the physician education module 235 compares one or more data points identified from video data captured during a medical procedure with baseline criteria associated with educational content items to select one or more educational content items for the practicing physician. The physician education module 235 selects educational content items associated with the type of medical procedure and with a baseline criterion specifying particular data that differs from the data identified from the captured video by at least a threshold amount. For example, the physician education module 235 obtains depth-sensing data from video data of a medical procedure during the procedure and selects educational content items associated with the type of medical procedure and with the depth-sensing data that differs from the depth-sensing data determined from the video data of the medical procedure by at least a threshold amount. In some implementations, the selected educational content items include information for accessing a simulator of the medical equipment 160 to be accessed by the practicing physician (e.g., simulator identifier, one or more exercises or techniques to be performed on the identified simulator, etc.). In another example, physician education module 235 determines tissue tension data during a medical procedure and selects educational content items associated with the type of medical procedure and the tissue tension data, which differ from tissue tension data determined from video data by at least a threshold amount. In various embodiments, the selected educational content items include information for accessing a simulator of the medical device 160 to be accessed by a practicing physician (e.g., the simulator's identifier, one or more exercises or techniques to be performed on the identified simulator, etc.).

[0086] In various implementations, the physician education module 235 determines the movement or location of medical device 160 or medical instrument during a medical procedure from video data of a medical procedure using one or more computer vision models or other models. Based on the movement or location information obtained from the video data, the physician education module 235 selects educational content items that include a baseline standard, whereby the movement of a piece of medical device or the location of a medical instrument in the telemetry data deviates from the baseline standard by at least a threshold amount. For example, the physician education module 235 determines the path length of a piece of medical device 160 during a medical procedure from the video data of the medical procedure, and the physician education module 235 selects educational content items that are associated with the type of medical procedure and include a baseline standard specifying the path length of the medical device 160, whereby the path length from the video data deviates from the path length of the medical device by at least a threshold amount. As another example, the physician education module 235 determines the location data of a piece of medical device 160 during a medical procedure from the video of the medical procedure, and the physician education module 235 selects educational content items that are associated with the type of medical procedure and include a baseline standard specifying the location data of the medical device 160, whereby the location data of the medical device 160 from the video data deviates from the baseline standard by at least a threshold amount. In another example, physician education module 235 determines the dexterity data of a practicing physician during a medical procedure from video data, and selects educational content items associated with the type of medical procedure and including a baseline standard for the dexterity data of the medical device 160, wherein the dexterity data determined from the video data deviates from the baseline standard by at least a threshold amount. In the aforementioned example, the educational content items selected based on telemetry data include information for accessing a simulator associated with the medical device 160 corresponding to the telemetry data (e.g., simulator identifier, one or more exercises or techniques to be performed on the identified simulator, etc.), thereby providing the practicing physician with increased interaction with the medical device. Furthermore, educational content items selected based on the deviation of the position data of a medical device 160 from the baseline standard may include one or more of the following: training videos associated with and describing the operation of the medical device 160, audio data describing the operation of the medical device 160, and information for accessing the simulator of the medical device 160 (e.g., simulator identifier, one or more exercises or techniques to be performed on the identified simulator, etc.).

[0087] In some implementations, physician education module 235 determines from video data of a medical procedure the angle at which a patient's structure (e.g., an organ) is transected by a medical device 160 during a medical procedure. Physician education module 235 selects educational content items associated with the type of medical procedure and including the angle used to transect the patient's structure, from which the angle determined from the video data of the medical procedure deviates by at least a threshold amount. Educational content items selected based on the deviation of the determined transect angle of the patient's structure from a baseline transect angle may include content describing the use of the medical device 160 (or medical device) to transect the patient's structure during the medical procedure, or content describing the correlation between the transect angle of the patient's structure and one or more outcomes of the medical procedure (e.g., information depicting the correlation between certain angles of transecting the patient's structure and positive outcomes of the medical procedure, or the correlation between the angles of transecting the patient's structure and negative outcomes of the medical procedure). As another example, physician education module 235 determines from video data of a medical procedure the proximity of a medical device 160 (or medical device) to one or more key structures of the patient (e.g., organs, bones, arteries) during the medical procedure. The physician education module 235 selects educational content items associated with the type of medical procedure and including baseline proximity of the medical device 160 (or medical instrument) from key structures of the patient, the proximity of the medical device 160 (or medical instrument) from video data deviating from the baseline proximity by at least a threshold amount. In various embodiments, the educational content items having baseline proximity to key structures of the patient include content describing the use of energy devices during the medical procedure, which may include interactive content (e.g., content with questions to be answered by a practicing physician), video or audio content describing the use of energy devices during the medical procedure, or other descriptive information about the use of energy devices during the medical procedure.

[0088] Furthermore, the physician education module 235 can determine the usage pattern of a medical device 160 during a medical procedure from video data of the procedure. In response to determining that the usage pattern of the medical device 160 deviates from the baseline usage pattern of that medical device in the educational content item, the physician education module 235 selects that educational content item. The selected educational content item may include baseline data describing the cost of the medical procedure based on the usage pattern and information on alternative usage patterns of the medical device 160 to reduce costs. As another example, the selected educational content item may include recommended usage patterns of the medical device 160 in this type of medical procedure or information on the use of multiple alternative medical devices 160.

[0089] In another example, the physician education module 235 compares one or more motion patterns detected within video data captured during a medical procedure (e.g., movement of a piece of medical equipment 160, movement of a portion of the physician's movements) with a baseline standard that includes motion patterns specific to the type of medical procedure, and selects an educational content item for the physician associated with the baseline standard for a specified motion pattern, the detected motion pattern differing from the motion pattern by at least a threshold amount. Therefore, the physician education module 235 can use data captured during the performance of a medical procedure (e.g., video data or telemetry data) to determine when to select an educational content item for the physician. Different detected patterns within the telemetry data or video data captured during the performance of a medical procedure can be compared with different educational content items, each associated with a different baseline standard. This allows the physician education module 235 to select educational content items for the physician based on deviations from the baseline criteria corresponding to the telemetry data or video data captured during the performance of the medical procedure, thereby allowing for customized selection of educational content items for specific portions of the medical procedure.

[0090] Physician Education Module 235 can apply one or more trained machine learning models to attributes of medical procedures performed by a licensed physician and educational content items (such as those associated with the type of medical procedure) to select one or more educational content items for presentation to the licensed physician. Example attributes of educational content items include: the type of medical procedure associated with the reference content item, one or more licensed physicians associated with the reference content item, the location where the medical procedure was performed (e.g., geographic location, identifier of a medical facility), the format of the reference content item (e.g., text data, audio data, video data, etc.), feedback from one or more licensed physicians regarding the reference content item (e.g., ratings, the amount of positive feedback received for the reference content item, etc.), or other descriptive information. In various implementations, Physician Education Module 235 trains one or more machine learning models to select one or more educational content items for the licensed physician based on the attributes of the educational content items and the characteristics of the licensed physicians. Example machine learning models include regression models, support vector machines, Naive Bayes, decision trees, k-nearest neighbors, random forests, augmentation algorithms, k-means, and hierarchical clustering. Machine learning models may also include neural networks, such as perceptrons, multilayer perceptrons, convolutional neural networks, recurrent neural networks, sequence-to-sequence models, generative adversarial networks, or transformers. In various implementations, other types of machine learning models may be additionally or alternatively trained or applied by the physician education module 235.

[0091] For example, in order to train a machine learning model to select one or more educational content items, the physician education module 235 generates a set of training examples, each of which includes data describing the execution of a medical procedure performed by a practicing physician and the attributes of the educational content item, and has labels indicating whether the practicing physician in the training example accessed the educational content item included in the training example (or indicating whether the practicing physician in the training example provided positive feedback to the educational content item included in the training example).

[0092] The machine learning model is applied to training examples to generate a predicted probability (or a predicted probability) that a practicing physician in the training example will access an educational content item in the training example. For each training example to which the machine learning model is applied by the physician education module 235, the physician education module 235 generates a score for the machine learning model, including an error term, based on the label applied to the training example and the predicted probability (or the predicted probability) that a practicing physician in the training example will access an educational content item in the training example. When the difference between the label applied to the training example and the predicted probability (or the predicted probability) that a practicing physician in the training example will access an educational content item in the training example, the error term and the corresponding score are large; conversely, when the difference between the label applied to the training example and the predicted probability (or the predicted probability) that a practicing physician in the training example will provide positive feedback to an educational content item in the training example, the error term and the corresponding score are small. In various implementations, the physician education module 235 uses a loss function to generate a score for the machine learning model applied to the training examples, based on the difference between the predicted probability of a practicing physician in the training examples accessing an educational content item in the training examples (or the predicted probability of a practicing physician in the training examples providing positive feedback to an educational content item included in the training examples). Example loss functions include mean squared error, mean absolute error, hinge loss, and cross-entropy loss.

[0093] The physician education module 235 backpropagates error terms to update a set of parameters including a machine learning model, and stops backpropagation in response to a score or loss function satisfying one or more criteria. For example, the physician education module 235 backpropagates the score of the machine learning model layer by layer to update the parameters of the machine learning model until the score is less than a threshold. For example, the physician education module 235 uses gradient descent to update the set of parameters including the machine learning model. The physician education module 235 stores trained machine learning models for application to data describing the execution of medical procedures performed by a practicing physician, as well as attributes of one or more educational content items. In some implementations, the physician education module 235 trains and maintains different machine learning models, each using a different combination of attributes of educational content items and data describing the execution of medical procedures performed by a practicing physician.

[0094] Alternatively or additionally, one or more machine learning models applied by the physician education module 235 to select educational content items are nearest neighbor models applied to embeddings corresponding to features of the educational content item and the practicing physician, including data describing the execution of medical procedures performed by the practicing physician. As further described above, the attributes of the educational content item include: the type of medical procedure associated with the reference content item, one or more practicing physicians associated with the reference content item, the location where the medical procedure was performed (e.g., geographic location, identifier of a medical facility), the format of the reference content item (e.g., text data, audio data, video data, etc.), feedback from one or more practicing physicians regarding the reference content item (e.g., ratings, the amount of positive feedback received for the reference content item, etc.), or other descriptive information. Example characteristics of a licensed physician include: the physician’s area of ​​expertise, the types of previous medical procedures performed by the physician, medical procedures planned to be performed by the physician, the location where the physician will perform the medical procedures (e.g., geographic location, identifier of a medical facility, etc.), collaborators connected to the physician via a connection diagram, or other descriptive information about the physician, and data describing the medical procedures performed by the physician as further described above.

[0095] In some implementations, the physician education module 235 applies a nearest neighbor model to the embeddings of a practicing physician, which determines the distance (or similarity measure) in the latent space between the physician's embeddings and the embeddings used for various educational content items. For example, the nearest neighbor model determines the Euclidean distance between the physician's embeddings and the embeddings used for educational content items. Based on this distance, the nearest neighbor model ranks the educational content items by the distance (or similarity measure) between the corresponding embeddings of the educational content items and the physician's embeddings, and selects one or more educational content items that have a threshold position in the ranking, thus having an embedding closest to the physician's embedding. Alternatively, the nearest neighbor model selects one or more educational content items that have a distance less than a threshold to the physician's embedding. Alternatively, the physician education module 235 generates embeddings of medical procedures performed by the practicing physician and selects one or more educational content items based on the distance between the embeddings of the medical procedures and the embeddings used for educational content items, as further described above.

[0096] Furthermore, in some implementations, the physician education module 235 generates embeddings for different practicing physicians based on the characteristics of the practicing physician, as further described above. The physician education module 235 determines the distance between the embedding for the practicing physician and the embeddings for additional practicing physicians. For example, the physician education module 235 determines the Euclidean distance between the embedding for the practicing physician and the embeddings for multiple additional practicing physicians. Based on this distance (or a measure of similarity), the physician education module 235 selects a set of additional practicing physicians. For example, the physician education module 235 selects additional practicing physicians with embeddings within a threshold distance of the practicing physician's embeddings. As another example, the physician education module 235 ranks the additional practicing physicians based on the distance between the embeddings of the additional practicing physicians and the embeddings of the practicing physician, and selects the additional practicing physicians who have at least a threshold position in the ranking. The physician education module 235 selects one or more educational content items to present to one or more of the selected additional practicing physicians for demonstration to the practicing physician. This implementation allows the physician education module 235 to utilize the similarities among various practicing physicians to select educational content items for demonstration to practicing physicians.

[0097] In various implementations, the physician education module 235 determines one or more baseline criteria for educational content items by applying one or more clustering models to one or more attributes of medical cases in which a specific type of medical procedure is performed. Attributes of the medical cases include one or more metrics generated for the medical case by the analysis module 230, telemetry data captured during the medical procedure in the medical case, video data captured during the medical procedure in the medical case, or other descriptive information about the medical case. Based on the attributes of the medical cases, the physician education module 235 generates embeddings for the medical cases. The physician education module 235 applies clustering models to the embeddings of different medical cases in which a specific type of medical procedure is performed to generate different clusters of cases in which a specific type of medical procedure is performed. Different clusters are represented by different centroids in a latent space including the embeddings for the medical cases, wherein the clusters include medical cases having embeddings within a threshold distance of the centroids of the clusters. In some implementations, the physician education module 235 applies a k-means clustering model to the embeddings of different medical cases in which a specific type of medical procedure is performed. K-means clustering is used such that medical cases in which a specific type of medical procedure is performed are included in clusters based on the distance between the embeddings of the medical cases and the centroids of different clusters. Medical cases in which a specific type of medical procedure is performed are included in clusters with centroids that have a minimum distance to the embeddings of the medical cases. The centroids of the clusters are iteratively updated based on the embeddings of the medical cases in which a specific type of medical procedure is performed and included in the various clusters until one or more criteria are met. This results in a specific number of clusters, each containing medical cases in which a specific type of medical procedure is performed with similar embeddings.

[0098] The physician education module 235 can identify baseline criteria based on medical cases included in one or more clusters. For example, a cluster of cases performing a specific type of medical procedure might correspond to a positive outcome for that specific type of medical procedure, while alternative clusters might correspond to a negative outcome. Based on telemetry or video data captured during the performance of a specific type of medical procedure in additional cases, the physician education module 235 generates embeddings for the additional cases and determines the clusters that include the additional cases based on the centroids of the clusters and the embeddings for the additional cases. In response to determining that the additional cases are included in alternative clusters corresponding to negative outcomes, the physician education module 235 selects one or more educational content items for demonstration to physicians performing the specific type of medical procedure during the additional cases. The physician education module 235 compares video or telemetry data captured during the performance of a specific type of medical procedure in an additional medical case with video or telemetry data associated with baseline criteria for educational content items related to the specific type of medical procedure, and selects one or more educational contents associated with the specific type of medical procedure and having baseline criteria for specified video or telemetry data that differ from the telemetry or video data captured during the performance of the specific type of medical procedure by at least a threshold amount.

[0099] Alternatively, the physician education module 235 selects educational content items for a medical case in response to determining that the embedding for the medical case is not included in a specific cluster. For example, the physician education module 235 selects educational content items for a medical case in response to determining that the embedding for the medical case is greater than a threshold distance from the centroid of a specific cluster of the medical case. This can indicate that the medical case has characteristics that deviate from those of other medical cases in which that type of medical procedure was performed and which have positive patient outcomes by at least a threshold amount. As further described above, the physician education module 235 can select educational content items that are associated with a specific type of medical procedure performed in the medical case and have baseline criteria, which include video data or telemetry data that differ from video data or telemetry data captured during the medical procedure performed in the medical case by at least a threshold amount.

[0100] When generating clusters of medical cases based on corresponding embeddings, the physician education module 235 can identify medical cases included in a specific cluster as reference cases for educational content items of the corresponding type of medical procedure. For example, in response to the physician education module 235 including a medical case in a specific cluster associated with a positive outcome, the physician education module 235 sends a prompt to the physician associated with the medical case to generate a reference case based on the medical case. In response to receiving authentication from the physician that a reference case has been generated from the medical case, the physician education module 235 pseudonyms the patient data in the medical case and stores the pseudonymed patent data, video data captured during the medical procedure, telemetry data captured during the medical procedure, and one or more indicators generated for the medical procedure as educational content items for the type of medical procedure. One or more patterns determined from the telemetry data or video data, or one or more generated indicators, are stored as baseline criteria associated with the educational content item. This simplifies the creation of educational content items for a type of medical procedure by utilizing data captured by the collaborative healthcare platform 140 during the medical procedure to generate educational content items for subsequent reference regarding the medical procedure.

[0101] In various implementations, educational content items selected by the physician based on the execution of the medical procedure are presented to the physician during the postoperative phase. Presenting educational content items to the physician during the postoperative phase allows for viewing of these items after the completion of the medical procedure. The physician education module 235 generates one or more interfaces that identify the selected educational content items to the physician. For example, the physician education module 235 includes a physician dashboard (such as those described below) that identifies the items presented to the physician. Figure 4 The physician education module 235 further describes the information of the selected educational content item in the physician dashboard. In various embodiments, the information identifying the selected educational content item includes a link that retrieves the selected educational content item for demonstration when selected by a practicing physician. Alternatively, the physician education module 235 presents the information identifying the educational content item in another interface or in another format. For example, the physician education module 235 sends a notification message to the practicing physician's client device 150, which includes a link that retrieves the selected educational content item for demonstration when selected by the practicing physician.

[0102] In various implementations, the physician education module 235 may include selected educational content items in one or more interfaces presented to practicing physicians when accessing the collaborative healthcare platform 140. For example, the physician education module 235 generates an interface including educational content and presents information describing the selected educational content items through this interface, allowing practicing physicians to access the selected educational content items by selecting the information describing the selected educational content items. As another example, the physician education module 235 includes information identifying the selected educational content items in a medical case page generated by the interface management module 215 for a medical procedure for which the educational content items have been selected. For example, the medical case page includes a section containing notes or feedback from the practicing physician regarding the medical case, wherein one or more educational content items selected by the physician education module 235 are included in this section. Furthermore, the interface management module 215 may generate one or more interfaces that include recommendations for practicing physicians based on indicators related to medical procedures, wherein the recommendation interface includes one or more educational content items selected by the physician education module for the practicing physician based on data describing the execution of one or more medical procedures.

[0103] In some implementations, the physician education module 235 includes selected educational content items in different interfaces, depending on the content of the selected educational content item. For example, an educational content item describing the use of a medical device or instrument is displayed in the recommended interface. As another example, educational content items including interactive materials or audio or video data for demonstration to practicing physicians are presented on a medical case page or in the educational interface. However, in other implementations, the physician education module 235 selects the interface for identifying the selected educational content item based on other characteristics of the educational content item.

[0104] Alternatively or additionally, the physician education module 235 presents selected educational content items to the physician during the intraoperative phase of a medical procedure. This presents the selected educational content items to the physician while the medical procedure is being performed. In various embodiments, the physician education module 235 sends a notification identifying the selected educational content item to a medical device 160 or a client device 150, which displays or audibly presents the notification to the physician. The notification may include specific content from the selected educational content item to facilitate access by the physician to relevant information from the selected educational content item. In various embodiments, the physician education module 235 sends a notification identifying the educational content item to a medical device 160 associated with the educational content item. For example, the educational content item includes recommended settings for the medical device 160 (e.g., force threshold, movement threshold), thus sending the notification to the medical device 160 simplifies the identification of the medical device 160 associated with the educational content item. A notification sent to a medical device 160 may include a link that, when selected by a practicing physician, causes the medical device 160 to execute one or more instructions to modify one or more settings based on educational content items. Similarly, information identifying educational content items associated with a medical device 160 presented by a client device 150 may include instructions that, when selected, send instructions to modify one or more settings of the medical device 160. This simplifies modifications to the settings of the medical device 160 based on selected educational content items by reducing the amount of interaction between the practicing physician and the medical device 160. Alternatively, the physician education module 235 includes information identifying the selected educational content item in the interface presented to the practicing physician via the client device 150.

[0105] In some implementations, the physician licensing education module 235 automatically modifies one or more settings of a medical device 160 based on educational content items selected for the physician. Such licensing may be specific to a particular medical procedure or limited to one or more specific medical devices 160 used during a particular medical procedure. When the physician licensing education module 235 automatically modifies one or more settings of the medical device 160, the physician education module 235 sends a notification, including one or more instructions corresponding to the selected educational content items, to the medical device 160 used in the medical procedure. The medical device 160 executes one or more instructions, thereby modifying one or more settings of the medical device 160 based on the selected educational content items. In various implementations, the medical device 160 displays a notification or otherwise informs the physician that one or more settings have been modified or specified based on the selected educational content items. The medical device 160 or client device 150 may present an instruction to a practicing physician to modify one or more settings based on selected educational content items, warning the physician that one or more settings of the medical device 160 are being automatically updated and providing the physician with the option to prevent modification of one or more settings. Alternatively, as further described above, unless the physician instructs the physician education module 235 not to automatically modify one or more settings of a medical device 160, the physician education module 235 may automatically modify one or more settings of a medical device 160 based on educational content items selected for the physician. This allows different implementations to allow the physician to opt in or out of the physician education module 235 automatically modifying one or more settings of a medical device 160.

[0106] In other implementations, presenting educational content items during the intraoperative phase of a medical case increases the number of interactions required to modify one or more settings of a medical device 160 used during the medical procedure. For example, presenting educational content items via a medical device 160 causes the device to request additional confirmation input from the physician after receiving input from the physician, to change a specific setting of the device to a value deviating from the corresponding value in the educational content item, or to specify a specific value of a specific setting of the device 160 outside the range corresponding to the educational content item. For example, presenting educational content items to the physician sends an instruction to a medical device 160 used during the medical procedure, which, when executed, causes the device to display one or more warnings, each warning requesting input from the physician when the device receives input from the physician to set the value of the device 160's settings to a value outside the range included in the educational content item. This increases the difficulty for practicing physicians to configure the medical device 160 in a way that is inconsistent with the selected educational content item, thereby increasing the likelihood that the settings of the medical device 160 are consistent with the selected educational item.

[0107] In various implementations, the physician education module 235 also generates one or more interfaces for practicing physicians to request evaluation of the practicing physician's performance of medical procedures. In various implementations, the evaluation physician has a connection to the practicing physician via a connection graph repository 255. For example, the evaluation physician collaborates with the practicing physician during the performance of a medical procedure. As an example, the evaluation physician supervises the practicing physician performing the medical procedure. Alternatively, the evaluation physician has a connection to the medical procedure in the connection graph repository 255.

[0108] Through an assessment interface generated by the physician education module 235, the practicing physician selects a medical procedure associated with the physician for assessment. For example, the physician education module 235 presents the assessment interface to the practicing physician in response to receiving an assessment request. In various embodiments, the physician education module 235 receives an identifier for a specific medical procedure from the practicing physician. Alternatively, the physician education module 235 receives a search query via the assessment interface and presents search results that include one or more medical procedures associated with the practicing physician and have attributes that at least partially match the received search query. The practicing physician then selects a presented medical procedure from the search results. The following is in conjunction with... Figure 11 Further description of the example evaluation interface.

[0109] For the selected medical procedure, the physician education model 235 receives a selection of frames for evaluating the physician's performance of the selected medical procedure. In some embodiments, the physician education module 235 presents a frame interface that identifies a set of frames for evaluating the performance of the medical procedure, and receives selections of frames from that set from the physician via the frame interface. The following is in conjunction with... Figure 12 Further description of the example framework interface. Different frameworks may correspond to different actions performed during a medical procedure or different techniques performed during a medical procedure. Frameworks may include guidelines or standards for performing different actions or techniques, or may include criteria for evaluating one or more actions or techniques. Each framework includes one or more categories, each corresponding to a different skill or technique used to perform a medical procedure. One or more frameworks may be specific to the medical facility in which the medical procedure is performed, while one or more other frameworks may be based on guidelines or criteria specified by a standardization body or by another entity associated with multiple medical facilities. Example techniques or actions evaluated by the frameworks include: the use of one or more pieces of medical equipment 160 (e.g., robotic medical equipment), techniques specific to a surgical procedure type, specific techniques used for one or more medical procedures, or other actions that affect the performance of a medical procedure.

[0110] When selecting a framework, a physician may also specify one or more assessment objectives. Assessment objectives specify one or more specific techniques to be assessed, identify specific information to be provided in the assessment, specify one or more types of educational content items to be received based on the assessment, or specify other information to be received based on the assessment. In various implementations, multiple assessment objectives may be specified. Specifying one or more assessment objectives allows a physician to request specific feedback from the assessment of a medical procedure.

[0111] Additionally, the Physician Education Module 235 generates an assessor selection interface for the selected medical procedure, an example of which is provided below. Figure 13Further description. The physician interacts with the assessor selection interface to identify the assessing physician for the medical procedure. The physician education module 235 receives, for example, an identifier of the assessing physician from the physician via the assessor selection interface. In various embodiments, the physician education module 235 retrieves identifiers of physicians connected to the physician or connected to the medical procedure via the connection graph repository 255 and presents the retrieved identifiers to the physician via the assessor selection interface. The physician selects an identifier of the physician or medical procedure connected via the assessor selection interface to select the assessing physician. Alternatively or additionally, the physician education module 235 receives one or more search terms via the assessor selection interface and generates search results that include physicians connected to the physician or medical procedure and have one or more characteristics that at least partially match the search query. The physician selects an identifier of the physician from the search results to select the assessing physician.

[0112] To evaluate the physician's performance of a medical procedure, the physician education module 235 retrieves stored data describing the physician's performance of the medical procedure and receives from the physician a selection of data segments describing the performance of the medical procedure. For example, the physician education module 235 retrieves video data and associated metadata (including telemetry data) captured during the medical procedure and stored in video library 250. In various embodiments, the data describing the performance of the medical procedure includes captured video data of the medical procedure, telemetry data captured by one or more sensors (e.g., sensors on one or more medical devices 160 used during the medical procedure), or both video data and telemetry data. The data segment describing the performance of the medical procedure selected by the physician includes a portion of data describing the physician's performance within a specific time interval. For example, the physician identifies a start time and an end time, where the data segment describing the performance of the medical procedure includes data describing the performance of the medical procedure that occurred between the start time and the end time. Therefore, in various embodiments, the segment includes video data of the physician between the start time and the end time or telemetry data of the physician captured between the start time and the end time. In some implementations, the physician education module 235 maintains one or more specific time intervals, and the practicing physician selects a start time, while the physician education module 235 determines an end time by incrementing the start time by the specific time interval to select a data segment describing the execution of a medical procedure. In some implementations, the physician education module 235 may maintain a single specific time interval. Alternatively, the physician education module 235 maintains multiple specific time intervals and receives the selection of specific time intervals by the practicing physician when selecting a data segment describing the execution of a medical procedure. In various implementations, the physician education module 235 generates the following combination Figure 14 The segment selection interface, described in further detail, allows licensed physicians to select one or more data segments describing the execution of a medical procedure.

[0113] Alternatively, the physician education module 235 can apply one or more models to data describing the execution of a medical procedure to automatically generate data segments describing the execution of the medical procedure, and store the generated segments in association with a practicing physician. Each segment includes a portion of the data describing the execution of the medical procedure between a start time and an end time. In various embodiments, the practicing physician selects a specific segment from the stored segments.

[0114] In some implementations, the physician may select multiple segments of data describing the performance of a medical procedure. Furthermore, in some implementations, the physician may select different frameworks to evaluate the different segments of data describing the performance of a medical procedure, thereby allowing for the evaluation of different skills or techniques within the different segments of data describing the performance of a medical procedure against different standards or criteria. In some implementations, the physician selects data segments describing the performance of a medical procedure and then selects a framework for evaluating those segments.

[0115] In some implementations, the practicing physician provides self-assessment data for one or more categories within a selected framework. For example, the self-assessment data includes self-assigned values ​​for each category within the selected framework, included in the physician education module 235 received from the practicing physician. In various implementations, self-assigned values ​​for categories are selected from a set of values. Providing self-assessment data for categories within a selected framework allows the practicing physician to compare their self-assessments of performance in different categories of the framework with assessment values ​​for the categories determined by the selected assessment physician. Such comparisons allow the practicing physician to more accurately assess their proficiency in various techniques or skills relative to the selected criteria.

[0116] The physician education module 235 generates an assessment message that includes: information about the selected medical procedure, information identifying the practicing physician, the selected framework, and one or more segments of data describing the execution of the medical procedure (and, in some embodiments, self-assessment data). In various embodiments, the physician education module 235 sends the assessment message to an assessing practicing physician who reviews the data describing the execution of the medical procedure in the selected segments and generates an assessment result for the physician by comparing the data describing the execution of the medical procedure with criteria in the selected framework. The assessment result includes an assessment value assigned by the assessing practicing physician to each category of the selected framework based on the review of the data describing the execution of the medical procedure in the selected segments.

[0117] Additionally, the evaluation results may include comments from the evaluating physician associated with one or more portions of the data describing the execution of a medical procedure within the selected segment. The evaluating physician may associate different comments with different portions of the data describing the execution of a medical procedure within the selected segment. For example, a comment may be associated with a specific time within a time interval that includes a data segment describing the execution of a medical procedure. As an example, a comment may be associated with a specific time within a video data segment describing the execution of a medical procedure. In another example, a comment may be associated with a specific time within a telemetry data segment describing the execution of a medical procedure. Including one or more comments associated with a specific time within a selected segment allows the evaluating physician to provide feedback to the practitioner regarding a specific time within the selected segment of data describing the execution of a medical procedure for further review.

[0118] In various implementation schemes, the physician education module 235 generates an assessment results interface in response to receiving assessment results from the assessing practicing physician. The following is in conjunction with... Figure 15 A sample assessment results interface is further described. The assessment results interface identifies the assessing physician, the physician, and the type of medical procedure performed by the physician. Additionally, the assessment results interface includes assessment values ​​for different categories within the selected framework. In various embodiments, the assessment results interface includes data describing the execution of the medical procedure corresponding to the selected segment, and may include comments associated with one or more specific times within the data segment describing the execution of the medical procedure received from the assessing physician. For example, the assessment results interface includes video data corresponding to the selected segment, and may include one or more comments associated with one or more specific times within the video data of the selected segment. In some embodiments, the physician education module 235 displays a specific portion of the data describing the execution of the medical procedure during the selected segment in response to receiving a selection of comments from the physician. In some embodiments, comments may be displayed near the corresponding specific portion of the data describing the execution of the medical procedure during the selected segment to simplify the review of the relevant portions of the data describing the execution of the medical procedure when reviewing comments.

[0119] In various implementations, the physician education module 235 selects one or more educational content items for demonstration to practicing physicians based on one or more assessment values ​​of one or more categories of the selected framework in the evaluation results. For example, in response to the physician education module 235 determining that the assessment value of a category of the selected framework is less than a threshold, the physician education module 235 selects educational content items associated with a type of medical procedure and having a baseline criterion that specifies a value of the category of the selected framework equal to or greater than the threshold. As another example, the physician education module 235 selects educational content items associated with a type of medical procedure and having a baseline criterion that specifies a specific value of a category of the selected framework that is at least a threshold amount greater than the assessment value of the category in the evaluation results. The physician education module 235 can select educational content items associated with each category of the selected framework that have an assessment value less than the value of the corresponding category of the selected framework, the value of which is specified as the baseline criterion in the educational content items associated with the type of medical procedure. Therefore, as further described above, the physician education module 235 can utilize the assessment results from evaluating practicing physicians to select one or more educational content items for demonstration to practicing physicians.

[0120] Additionally or alternatively, the physician education module 235 uses assessment results obtained for various medical procedures of specific types to train an assessment model to generate predicted assessment values ​​for one or more categories based on data describing the execution of the medical procedures. In various embodiments, the physician education module 235 generates training examples from the assessment results received for the medical procedures. Each training example includes a training segment of data describing the execution of a segment of a training medical procedure with a type and training framework; each training example also has one or more labels identifying the category of the training framework and an assessment score for the training category of the training framework. In some embodiments, the training example includes multiple labels, each label including a pair of identifiers for the category of the training framework and corresponding assessment scores. Alternatively, each training example includes a single label, which includes an identifier for the category of the framework and an assessment score for that category. In various embodiments, as further described above, the physician education module 235 obtains training examples based on assessment values ​​provided by one or more assessing physicians when evaluating segments of medical procedures performed by the physician.

[0121] The physician education module 235 trains an evaluation model by applying it to a set of training examples. Applying the evaluation model to the training examples generates predicted evaluation values ​​for one or more categories of the training framework included in the training examples based on training fragments of data describing the execution of training medical procedures in the training examples. For each training example to which the physician education module 235 applies the evaluation model, the physician education module 235 generates a score for the evaluation model, which includes an error term based on the labels applied to the training example and the predicted evaluation values ​​(or based on one or more error terms based on the labels corresponding to different categories of the training framework included in the training example and the predicted evaluation values ​​of the corresponding categories of the training framework included in the training example). The error term and the corresponding score are larger when the difference between the labels applied to the training example and the predicted evaluation values ​​of the training example are large, and smaller when the difference is small. In various embodiments, the physician education module 235 uses a loss function to generate the score of the evaluation model applied to the training examples, which is based on the difference between the labels applied to the training examples and the predicted evaluation values ​​of the training examples. Example loss functions include mean squared error, mean absolute error, hinge loss, and cross-entropy loss.

[0122] The physician education module 235 backpropagates error terms to update a set of parameters including the evaluation model, and stops backpropagation in response to a score or loss function satisfying one or more criteria. For example, the physician education module 235 backpropagates the model's score layer by layer to update the parameters including the machine learning model until the score is less than a threshold. For example, the physician education module 235 uses gradient descent to update the set of parameters including the evaluation model. The physician education module 235 stores trained evaluation models to be applied to data and frames describing the execution of segments of medical procedures performed by a practicing physician to generate predicted evaluation scores for one or more categories in the frame. In some implementations, the physician education module 235 trains and maintains different evaluation models for each category in the frame, where category-specific evaluation models generate predicted evaluation values ​​for different categories in the frame.

[0123] In some implementations, the physician education module 235 selects one or more educational content items for demonstration to a practicing physician based on predicted evaluation values ​​for categories of a framework. For example, in response to receiving an evaluation request and a framework selection from a practicing physician, the physician education module 235 applies one or more evaluation models trained as further described above to a data segment describing the performance of a medical procedure by the practicing physician and the selected framework, thereby generating predicted evaluation values ​​for one or more categories of the selected framework. In some implementations, the physician education module 235 applies different evaluation models to a data segment describing the performance of a medical procedure by the practicing physician and the selected framework to generate predicted evaluation values ​​for different categories of the selected framework. Similarly, the physician education module 235 may apply different evaluation models corresponding to different frameworks to different segments of data describing the performance of a medical procedure. This allows the use of different frameworks to evaluate different segments of data describing the performance of a medical procedure. Based on the difference between the predicted evaluation value of a category and the value of the category included as a baseline criterion in one or more educational content items, the physician education module 235 can select one or more educational content items for demonstration to a practicing physician, as further described above. Therefore, the physician education module 235 can utilize one or more trained assessment models to generate a predicted assessment score for the practicing physician based on the framework and data fragments describing the execution of medical procedures, and then use one or more additional machine learning models to select educational content items to be demonstrated to the practicing physician.

[0124] When a physician selects reference content items based on an assessment value or a predicted assessment value for a category of a frame, the physician education module 235 may select educational content items based on assessment values ​​or predicted assessment values ​​determined for each medical procedure performed by the physician, as further described above. Alternatively or additionally, the physician education module 235 determines a composite value for a category of a frame based on assessment values ​​or predicted assessment values ​​determined for multiple medical procedures of a common type performed by the physician. For example, the composite value for a category of a frame is the average of assessment values ​​or predicted assessment values ​​determined for multiple medical procedures of a common type performed by the physician. As another example, the composite value for a category of a frame is the median of assessment values ​​or predicted assessment values ​​determined for multiple medical procedures of a common type performed by the physician. As further described above, the physician education module 235 selects one or more educational content items for the physician by comparing the composite value of the category to a baseline standard of values ​​for the category in one or more designated educational content items. Alternatively or additionally, the physician education module 235 selects one or more educational content items for the practicing physician based on changes in the assessed or predicted assessed values ​​of a category within the framework for that physician over time. For example, in response to determining that during a specific duration, when performing a medical procedure of a particular type, the practicing physician's assessed or predicted assessed value in the category of the framework decreases by at least a threshold amount, the physician education module 235 selects educational content items associated with the specific type of medical procedure and category for demonstration to the practicing physician.

[0125] The demonstration module 240 utilizes stored information associated with completed medical procedures to facilitate the generation of demonstrations for educational, research, training, or other purposes. Demonstrations can be in the form of slides, posters, videos, animations, or other multimedia content. Demonstrations can incorporate various multimedia (e.g., videos, images, 3D models, and associated metadata), patient record data, medical equipment telemetry data, information from content feeds, analytics, or other information generated and / or stored by the collaborative healthcare platform 140.

[0126] In one implementation, the presentation module 240 may maintain one or more presentation templates for generating presentations. Templates may include pre-formatted content with various information fields that can be automatically populated from a set of records. For example, a physician wanting to prepare a presentation related to a set of recently performed surgeries may specify the set of surgeries to be included in the presentation, and the presentation module 240 may automatically populate the presentation pages based on data stored in association with those surgeries, where each page is associated with one or more types of data about the performed medical procedures. In some implementations, the presentation module 240 may apply one or more trained machine learning models to automatically generate and / or recommend presentation content that may be of interest to practicing physicians. In other implementations, the presentation module 240 may intelligently and automatically de-identify patient data included in the presentation.

[0127] The presentation module 240 may also include various editing tools for creating, viewing, and editing presentations. For example, editing tools can enable editing of text, video, images, animations, 3D models, or other content included in the presentation.

[0128] In one implementation, presentations can be presented directly via presentation module 240 without exporting the data associated with the presentation outside the collaborative healthcare platform 140. For example, presentation module 240 can enable live streaming of a presentation to a group of invited participants during a remote presentation session. Invited participants can be limited to users 155 of the collaborative healthcare platform 140, or can include external participants who can access the presentation via an external link. Sharing presentations in this way allows physicians to maintain data privacy and compliance, and avoids potential problems associated with exporting medical data externally.

[0129] Application integration module 245 manages the integration of applications with the collaborative healthcare platform 140. Applications can be used to add additional, optional functionality to the collaborative healthcare platform 140. For example, applications can integrate with specific EHR systems, scheduling systems, or other existing healthcare systems. Applications can also allow users to selectively add specific functionality beyond the core features of the collaborative healthcare platform 140. Application integration module 245 allows third parties to create applications that interface with the collaborative healthcare platform 140 and makes these applications available for addition.

[0130] Application integration module 245 can maintain a catalog of applications that can interface with collaborative healthcare platform 140 and can provide an interface that allows user 155 to selectively add applications for integration. In various implementations, applications identified by application integration module 245 have been authorized or approved for installation by the administrator of collaborative healthcare platform 140, thereby allowing the regulation of applications that can run on collaborative healthcare platform 140.

[0131] Additionally, the application integration module 245 may include one or more application programming interfaces (APIs) for applications installed through the application integration module 245. The application's API provides functionality for exchanging data between the application and one or more components of the collaborative healthcare platform 140, thereby simplifying data exchange between the application and other parts of the collaborative healthcare platform 140.

[0132] Video library 250 stores videos, training demonstrations, simulations, or other medical videos of various medical procedures, along with metadata associated with the videos. Examples of metadata associated with videos of medical procedures may include telemetry data from one or more medical devices received in conjunction with the video, comments or annotations received from one or more practitioners via a surgical interface during the medical procedure included in the video, segmented data dividing the video into time segments related to different steps of the procedure, profile information associated with the patient in the video (e.g., age, body mass index, sex, etc.), or other information supplementing the video. In various embodiments, various reference content items including video data may be stored in video library 250 for retrieval by physician education module 235.

[0133] Video library 250 can store videos in an indexed database that indexes videos based on various metadata. Video library 250 can then be browsed or searched via a video library interface to identify related videos. Metadata associated with a video can include permissions stored in a connection graph repository 255, which control which users 155 can access different videos. For example, a video in video library 250 can be accessed only by user 155 who has explicitly shared the video with that user or who otherwise has viewing rights to that video.

[0134] The connection graph storage 255 includes a database that stores information describing connections between entities or other objects (e.g., video or other multimedia) managed by the collaborative healthcare platform 140. For example, as described above, the connection graph storage 255 stores connections between users 155, connections between users 155 and surgery, connections between users 155 and multimedia content or other objects, or other connections between data entities of the collaborative healthcare platform 140.

[0135] User profile storage 260 stores profile data of users 155 of the collaborative healthcare platform 140. The user profile of a licensed physician includes descriptive information such as the physician's name, contact information, qualifications or certifications, resume, types of medical procedures the physician can perform, affiliated medical facilities, operating room preferences (such as patient positioning, equipment setup, preferred instruments, typical surgical procedure sequence, etc.), equipment configuration preferences (e.g., ergonomic settings of a robotic console), or other information describing the physician. Machine learning techniques can be used to infer aspects of the user profile. For example, preferred instruments or procedure sequences for physicians can be inferred from the application of a machine learning model trained to infer such preferences based on observed historical data. Additionally, the user profile of a licensed physician includes medical procedures performed or to be performed by the physician, and information describing the medical procedures. For example, a user profile identifies different types of medical procedures to be performed by or by a licensed physician, and may include characteristics of each medical procedure (e.g., the duration of the medical procedure, the number of times the licensed physician performs the type of medical procedure matching the medical procedure, etc.). Furthermore, as further described above, one or more metrics determined by the analysis module 230 for the licensed physician may be included in the licensed physician's user profile.

[0136] The patient data repository 265 includes a patient profile for each patient associated with a medical case. The patient profile includes characteristics of the corresponding patient, which can be obtained from the patient's electronic health record or provided via input from a practicing physician. Patient characteristics include demographic information about the patient, the patient's medical condition, previous medical procedures performed by the patient, the patient's allergies, the patient's contact information, current or previous prescriptions, or other medically relevant information about the patient. A patient identifier is associated with the patient profile to uniquely identify the patient profile.

[0137] All data stored on the collaborative healthcare platform 140 (or otherwise made available through the collaborative healthcare platform 140) may be stored, presented, and in some cases, the methods used to ensure compliance with various data privacy and protection regulations are limited.

[0138] Figures 3A to 3B Example Physician Dashboard 300 is shown. Figure 3A The upper part of the dashboard 300 is shown, while Figure 3BThe lower part of dashboard 300 (which may be continuously scrollable) is shown. The physician dashboard 300 can function as the main login page for practicing physicians after logging into the collaborative healthcare platform 140. The physician dashboard 300 may include various content sections, at least some of which may be specifically designed for physicians. Search bar 305 enables the input of text-based search queries to search for content available in the collaborative healthcare platform 140 (e.g., case pages, other user pages, videos, presentations, etc.). In response to the input of a search query, a list of results with links to content matching the search query can be displayed. Video promotion section 310 displays videos recently added by the physician, with user interface tools allowing the physician to promote the video by sharing it with other users, create highlight reels, or view various statistics about the video. Achievement section 315 presents achievements related to the use of the collaborative healthcare platform 140. In this example, achievement section 315 highlights that the user has recently reached 100 videos and provides links to view the user's videos and access the video library. Other examples of achievements in the Achievements section 315 may involve the number of cases managed, time spent using platform 140, the number of connections, the frequency of interactions, or other usage achievements. The Video Library 320 includes video thumbnails, video tags, or other links to enable browsing of videos selected as potentially relevant to the practitioner. For example, relevant videos may be selected based on the history of videos viewed by the practitioner, based on the practitioner's practice area or other profile information, or other factors related to past or upcoming surgeries associated with the practitioner. The Webinar Promotion section 325 includes promotional banners for upcoming webinars, which will be viewable within the Collaborative Healthcare Platform 140. Webinars may be identified as potentially of interest to practitioners based on factors such as the webinar's topic, the webinar's organizer, or other factors. The Shared Cases section 330 provides summary information and links to case pages already shared with practitioners. Examples of case pages are described in more detail below. The analysis summary 335 includes example analyses related to physicians' use of the collaborative healthcare platform 140, surgeries performed by physicians, or other analytical data derived from information stored in the collaborative healthcare platform 140. The analytical data may be presented in one or more visual presentations, such as graphs or charts. The feedback section 340 provides links to allow physicians to send feedback to the administrator of the collaborative healthcare platform 140.

[0139] Figures 3A to 3BThis is just one example of a physician dashboard 300. The types of content presented in the physician dashboard 300 can differ for different practicing physicians and / or change dynamically over time for the same practicing physician. Some sections can be fixed and always appear after accessing the dashboard 300 (e.g., search bar 305, video library 320, shared cases 330, analysis 335, and feedback section 340), while other sections (e.g., video promotion 310, achievements 315, webinars promotion 325) can be dynamically inserted only in certain contexts. For example, a webinars promotion 325 can only be presented when an upcoming webinars deemed of sufficient interest are about to take place. Similarly, an achievement 315 can only be displayed if a relevant achievement has recently been achieved. Furthermore, the dashboard 300 can be customized by the user to display desired sections in a configured order. When present, the individual sections 305, 310, 315, 320, 325, 330, 335, and 340 can also be presented in different orders in different contexts.

[0140] Figure 4 Another implementation of the physician dashboard 400 is shown. Figure 4 In the example shown, the physician dashboard 400 includes an education content item section 405 that includes information identifying education content items selected from a practicing physician based on previously performed medical procedures. The physician education module 235 selects the identified education content items based on stored baseline criteria for education content items associated with the type of previously performed medical procedure and captured data describing the execution of the previously performed medical procedure, as described above. Figure 2 Further description. In various implementations, the Educational Content Item section 405 identifies one or more reasons why the identified educational content item may be of potential interest to a practicing physician. Figure 4 In the example, the educational content item section 405 indicates that the identified educational content item includes recommended parameters or settings for a piece of medical equipment 160 (e.g., a robotic arm) used in a previously performed medical procedure. Figure 4 The example of educational content item 405 includes a link 410 that, when selected by a licensed physician, retrieves educational content items for presentation to the licensed physician via the licensed physician's client device 150.

[0141] For illustrative purposes, Figure 4An example physician dashboard 400 is shown, in which the educational content item section 405 is displayed near the search bar 305. For example, the educational content item section 405 is displayed below the search bar 305 of the physician dashboard 400, thus highlighting the educational content item section 405 in the physician dashboard 400 to increase the likelihood that a practicing physician will select the link 410 to the identified educational content item. However, in other embodiments, the physician dashboard 400 displays the educational content item section 405 in a different position relative to other sections. Similarly, although Figure 4 An example is shown in which the Achievements section 315 and the Video Library section 320 are displayed in combination with the Educational Content Items section 405. However, in other embodiments, the Physician Dashboard 400 displays different or additional sections in combination with the Recommended Reference Content Items section 405.

[0142] In various implementations, the educational content item section 405 is dynamically inserted into the physician dashboard 400 in some contexts and not included in the physician dashboard 400 in other contexts. For example, the physician dashboard 400 displays the educational content item section 405 after the physician has completed a medical procedure. In one example, the physician dashboard 400 displays the educational content item section 405 starting from a specific amount of time after the physician has completed the medical procedure, but does not display the educational content item section 405 until a specific amount of time has elapsed since the completion of the medical procedure. In various implementations, the physician dashboard 400 displays the educational content item section 405 within a specific time interval after the physician has completed the medical procedure.

[0143] Although Figure 4 An example is shown in which the Educational Content Items section 405 identifies a single reference content item; however, in other embodiments, the Educational Content Items section 405 displays multiple reference content items selected for the physician. For example, the Educational Content Items section 405 is a carousel of content items with multiple slides, each slide including information identifying a different selected educational content item and including links to the different selected educational content items. In response to a physician performing a specific interaction with the Educational Content Items section 405, the Educational Content Items section 405 is updated to display different slides including information identifying the different selected educational content items. For example, in response to a physician performing a swipe gesture along an axis perpendicular to the axis including the search bar 305, the Educational Content Items section 405, the Achievements section 315, and the Video Library section 320, the Educational Content Items section 405 displays an alternative slide including information identifying the different selected educational content items. This allows a single section of the Physician Dashboard 400 to identify multiple educational content items for the physician.

[0144] In some implementations, the collaborative healthcare platform 140 generates one or more educational interfaces, such as educational dashboards. These educational interfaces may additionally or alternatively display one or more recommended educational content items to physicians, thereby providing them with additional ways to access the recommended educational content items. The physician dashboard 400 may include interface elements that, when selected by a physician, cause the educational interface to be displayed. In some implementations, the educational interface may display information identifying multiple recommended educational content items, thereby allowing physicians easier access to a wider range of recommended educational content items.

[0145] Figure 5 An example embodiment of a case-sharing interface 500 for sharing cases with one or more contributors is shown. Adding a contributor to a case can create connections between the contributor and the case, and between the contributor and the case owner. The case-sharing interface 500 includes a selection element 505 for receiving identification information to identify a desired contributor. For example, the selection element 505 may receive an email address, name, username, or another identifier for a practicing physician or other requesting contributor. In some embodiments, after selecting the identification information of the desired contributor, the case-sharing interface 500 may display all or part of the profile data of the requested collaborator, allowing the requester to confirm whether the matching profile data is that of the intended collaborator. The case-sharing interface 500 then allows the requester to confirm or reject the selection of a collaborator and interact with a permission selection element 520 to set the desired permission level for the requested collaborator. Here, the permission level may restrict the invited collaborator's access to data about the case and / or may restrict actions that the collaborator is allowed to perform in relation to the case. In the example implementation, the permission level can be selected between "collaborator" level 525A and "representative" level 525B.

[0146] In response to receiving input (via permission selection element 520) selecting a requested collaborator and setting the desired permission level, the case-sharing interface 500 can send an invitation to the requested contributor (e.g., via email, text message, telephone call, portal message, or other communication mechanism) so that the requested collaborator can accept or decline the request. If the request is accepted, the case-sharing interface 500 can add the new collaborator's identifier or other information to the list of connected physicians 510 that lists the contributors added to the case. For example, the illustrated example shows a list of connected physicians 510 including the case owner 515 and three additional contributors who have been added to the case.

[0147] The case sharing interface 500 also allows the case owner to change the permission levels of existing contributors in the connected list of physicians 510. Furthermore, the case sharing interface 500 may include a removal element 530 associated with each contributor in the connected list of physicians 510, which enables the removal of a contributor from the case. Selecting the removal element 530 removes the stored connection between the physician and the case, thus depriving the physician of access to the case.

[0148] Figure 6 This is an example implementation of a case dashboard 600 for a practicing physician. The case dashboard 600 enables access to cases owned by the practicing physician, as well as cases shared with the practicing physician by other users 155, as indicated in the case summary 610. In this example, the case dashboard 600 is organized as a set of case cards 605, each case card graphically displaying a summary of the case. Selecting a case card 605 links to the case page 400 for that case. In an alternative implementation, the dashboard 600 may be presented in a list view or other view, instead of necessarily in a list view. Figure 6 The visual presentation shown is of case card 605.

[0149] Figure 7 An example of a telepresence interface 700 associated with a telepresence session that may occur during an actual surgery or a simulated surgery is shown. Alternatively, the telepresence session can be used for live planning purposes without necessarily performing or simulating surgery. In this example, the telepresence interface 700 displays a three-dimensional model of a target anatomical structure 705 associated with the surgery. The model may include comments with annotations that can be obtained during the telepresence session or added during the preoperative phase. Alternatively, the telepresence interface 700 may include views of live video or images associated with the ongoing surgery. In one embodiment, each contributor may be able to switch between different relevant views, such as live video or images, a three-dimensional model, preoperative images, or other relevant multimedia.

[0150] The telepresence interface 700 may also include a telepresence content feed 715 for sending and receiving real-time messages between contributors. For example, the telepresence content feed 715 allows users to post messages and / or view messages from other participants. Messages may include text, media content (e.g., images, videos, animations, etc.), or links to various media content or other resources (e.g., research articles). The telepresence interface 700 may also enable participants to provide annotations (presented as images, videos, or models) on target anatomical structures. For example, participants can pin comments to specific locations within the depicted anatomical structure, as indicated by an identifier 710.

[0151] Additionally, the telepresentation interface 700 can display statistics 720 or other analyses that may be relevant to the surgery. Statistics 720 may include estimated or modeled values ​​or measures related to anatomical structures based on various sensor data from the medical device 160. The telepresentation interface 700 can dynamically update statistics 720 over time during the surgery.

[0152] Figure 8 This is another example of a telepresentation interface 800 associated with a telepresentation session. In this example, the telepresentation interface 800 displays a live video of a surgery being performed, along with a set of annotation tools 810 that allow remote contributors to add annotations 805 overlays onto the video. The telepresentation interface 800 also includes a set of alternative views 815 that contributors can switch between during a telepresentation session. These alternative views 815 may include one or more different camera views (e.g., a view of the medical environment), one or more 3D models (e.g., such as...) Figure 7 (as shown), views of preoperative images, or other multimedia associated with the case. In various implementations, as combined with the above... Figure 2 Further described, if the licensed physician associated with the case generates a reference case based on the medical case, then a remote presentation interface (such as...) for the medical case is stored. Figure 7 or Figure 8 (as shown), and can then be presented to other practicing physicians.

[0153] exist Figure 8In the example, the telepresentation interface 800 also displays educational content item 820 to a practicing physician (such as one performing a medical procedure). In various embodiments, the telepresentation interface 800 dynamically selects educational content item 820 based on video or telemetry data captured during the medical procedure. The telepresentation interface 800 includes information describing or extracting from the educational content item 820, allowing the practicing physician to identify content from the educational content item 820 via the telepresentation interface 800. In various embodiments, the telepresentation interface 800 limits the presentation of educational content item 820 to certain time intervals. For example, the telepresentation interface 800 displays educational content item 820 in response to the collaborative healthcare platform 140 determining that video or telemetry data captured during the medical procedure deviates from a baseline standard associated with educational content item 820 by at least a threshold amount. When the captured telemetry or video data does not deviate from the corresponding baseline standard for educational content item 820 by at least a threshold amount, the telepresentation interface 800 does not present educational content item 820. For example, a client device 150 displaying a telepresence interface 800 receives a demonstration instruction for presenting educational content item 820 and the educational content item 820 from a collaborative healthcare platform 140, and subsequently receives an alternative instruction from the collaborative healthcare platform 140 to stop presenting the educational content item 820. The alternative instruction may be received in response to the collaborative healthcare platform 140 determining that video data or telemetry data received during a medical procedure meets a baseline standard associated with the educational content item 820, or in response to determining that the video data or telemetry data no longer identifies a pattern corresponding to the baseline standard associated with the educational content item 820.

[0154] To simplify the incorporation of information from educational content item 820 into medical procedures, telepresence interface 800 presents modification instructions 825 associated with educational content item 820. Modification instructions 825 include an identifier for a medical device 160 and values ​​for one or more settings of that medical device 160. In response to a physician selecting modification instructions 825 via telepresence interface 800, collaborative medical platform 140 receives a request identifying educational content item 820 and the medical device 160. Upon receiving this request, collaborative medical platform 140 sends instructions to the identified medical device 160 to modify the values ​​of one or more settings to values ​​retrieved from educational content item 820 and included in the instructions sent to the medical device 160. In various embodiments, collaborative medical platform 140 determines the identifier of the medical device 160 based on identifiers included in telemetry data received by collaborative medical platform 140 or identification information included in received video data of the medical procedure. This simplifies modifying one or more settings of the medical device based on the educational content item 820 via interaction with the remote presentation interface 800, rather than by manually entering the value of the setting identified by the educational content item into the medical device 160.

[0155] Figure 9 This is an example implementation of an analytics dashboard 900 for practicing physicians. In this example, the analytics dashboard 900 displays a summary of cases managed by the practicing physician, including, for example, the total number of cases, the number of cases in the current month, the number of cases in the current week, and the distribution of the types of cases performed by the physician. Figure 9 In the example, the analytics dashboard 900 also displays an educational content item section 905 to the physician. The educational content item section 905 includes information identifying the educational content items selected by the collaborative healthcare platform 140 for the physician based on data describing medical procedures performed by the physician, as described above. Figure 2 Further described. In various embodiments, the educational content item section 905 includes links that, when accessed, retrieve educational content items from the collaborative healthcare platform 140 or from a third-party server 170 for demonstration purposes. In some embodiments, the educational content item section 905 may identify educational content items selected based on medical procedures recently performed by a practicing physician. Alternatively, the analytics dashboard 900 includes multiple educational content item sections 905, each including educational content items selected for medical procedures previously performed by a practicing physician, thereby simplifying access to different educational content items associated with various medical procedures performed by the practicing physician.

[0156] Figure 10This is an example implementation of a case video interface dashboard 1000 for viewing case videos. Case videos can be captured during a telepresence session, or similarly during surgery without a live streaming telepresence session. The case video interface 1000 includes a video interface that displays one or more views of video associated with medical procedures. The video interface 1000 can include multiple captured views, which can originate from cameras in the medical environment, cameras inserted into anatomical structures (e.g., endoscopic cameras), or other cameras. The captured views can also include 3D models, preoperative images, surgical planning documents, or other visual information. The video can be segmented (manually or automatically using video processing and content recognition technologies) to divide the video into segments associated with different steps of the surgery. The video can include annotations provided by the practitioner during the telepresence session or in postoperative review. Content feeds 1010 can be presented in association with the video to enable users 155 to post comments, links, media, or other content associated with the presentation. Reference content items (such as reference cases) can be used in conjunction with... Figure 10 The described video interface 1000 displays videos and other information about medical procedures to a practicing physician (e.g., content feed 1010).

[0157] Figure 11 This is an example assessment interface 1100 used by a licensed physician to request an evaluation of at least a portion of a medical procedure performed by the licensed physician. In various implementations, the collaborative healthcare platform 140 displays the assessment interface 1100 in response to receiving a request from the licensed physician's client device 150 for the evaluation of the medical procedure by the evaluating physician. Figure 11 The example evaluation interface 1100 shown includes a search element 1105 configured to receive one or more search terms from a practicing physician. Based on the received search terms, the collaborative healthcare platform 140 identifies one or more medical procedures associated with a practicing physician and having at least partially satisfying one or more characteristics of the received search terms. The evaluation interface 1100 displays a medical procedure identifier 1110 for the one or more medical procedures identified in response to the received search terms. For example, the collaborative healthcare platform 140 retrieves medical procedures with connections to a practicing physician in a connection graph repository 255 and identifies one or more medical procedures among the retrieved medical procedures that have characteristics at least partially matching the search terms received via search element 1105 in the evaluation interface 1100. The practicing physician selects the medical procedure identifier 1110 corresponding to the medical procedure used for evaluation.

[0158] Figure 12This is an example framework interface 1200 for a licensed physician to select a framework for evaluating at least a portion of a medical procedure performed by the physician. In various embodiments, the collaborative healthcare platform 140 presents the framework interface 1200 to the licensed physician after receiving a selection of a medical procedure from the physician. The framework interface 1200 presents a set of frameworks 1205 for evaluating the medical procedure. Different frameworks in set 1205 may correspond to different types of actions performed during the medical procedure or different techniques performed during the medical procedure. Frameworks may include guidelines or standards for performing different types of actions or techniques, or may include criteria for evaluating one or more actions or techniques. Each framework includes categories corresponding to different skills or techniques applicable to performing the medical procedure. Different frameworks may specify different standards for evaluating categories, or different frameworks may include different categories. One or more frameworks may be specific to the medical facility in which the medical procedure is performed, while one or more other frameworks may be based on guidelines or criteria specified by a standardization body or other entities associated with multiple medical facilities. The licensed physician selects a framework from set 1205 via interaction with the framework interface 1200. For example, a practicing physician performs specific interactions with the information of the framework identified in group 1205 in order to select a framework for evaluating medical procedures.

[0159] In various implementations, the framework interface 1200 also includes an assessment target element 1210. The assessment target identifies one or more specific technologies to be assessed by the evaluating physician, identifies specific information to be provided in the assessment, identifies one or more types of educational content items to be received based on the assessment, or identifies other information to be received based on the assessment. In various implementations, multiple assessment targets may be specified. The assessment target element 1210 receives input from the physician describing or otherwise identifying the assessment target. For example, the assessment target element 1210 includes a text box configured to receive text input from the physician describing the assessment target. As another example, the assessment target element 1210 includes a set of assessment targets, and the physician selects one or more assessment targets from this set by interacting with the assessment target element 1210.

[0160] Figure 13This is an example evaluator selection interface 1300 for a licensed physician to select an evaluator for a medical procedure. In some embodiments, the collaborative healthcare platform 140 presents the evaluator selection interface 1300 after presenting the evaluation interface 1100 and the framework interface 1200 to the licensed physician. The evaluator selection interface 1300 includes an evaluator search element 1305 configured to receive a search query from the licensed physician. Based on the search query, the evaluator selection interface 1300 displays one or more licensed physician identifiers 1310. Each licensed physician identifier 1310 corresponds to a licensed physician having one or more characteristics that at least partially satisfy the search query. In some embodiments, the licensed physician identifier 1310 corresponds to a licensed physician who has a connection to a licensed physician or medical procedure in the connection graph repository 225 and has one or more characteristics that at least partially match the search query. The licensed physician corresponding to the licensed physician identifier 1310 selected by the licensed physician via the evaluation interface 1300 is the evaluator for the medical procedure performed by the licensed physician.

[0161] Figure 14 This is an example segment selection interface 1400 for a licensed physician to select a data segment describing the execution of a medical procedure for evaluation by an assessing physician. In some embodiments, segment selection interface 1400 is presented after the collaborative healthcare platform receives a selection from the assessing physician. However, in other embodiments, segment selection interface 1400 is displayed after assessment interface 1100 and before framing interface 1200, or at different times relative to framing selection and assessing physician selection. Figure 14 In the example, the segment selection interface displays video data 1405 of the medical procedure. In some embodiments, the segment selection interface 1400 displays telemetry data captured during the execution of the medical procedure, or a combination of the video data 1405 of the medical procedure and the telemetry data captured during the execution of the medical procedure. In various embodiments, the segment selection interface 1400 includes a media player for playing the video data 1405 of the medical procedure to a physician. The media player includes controls for changing the playback speed of the video data 1405 and for navigating the video data 1405 to facilitate the physician's viewing of the video data 1405.

[0162] Additionally, the segment selection interface 1400 includes a start time 1410 and an end time 1415 for the segment. In some embodiments, the physician manually enters the start time 1410 and the end time 1415 in the corresponding interface element of the segment selection interface 1400 to specify the time range of data (e.g., video data 1405, telemetry data, or a combination of video data and telemetry data) describing the execution of a medical procedure, including the segment. Alternatively or additionally, the physician performs a specific interaction with the video data 1405, and the time at which the physician performs the specific interaction includes the start time 1410 of the segment. Similarly, the physician performs an additional specific interaction with the video data at a later time, and the later time includes the end time 1415 of the segment. The segment selection interface 1400 displays the start time 1410 and the end time 1415 of the segment to allow the physician to review or modify the time interval including the segment.

[0163] In various implementations, the collaborative healthcare platform 140 determines the end time 1415 of a segment based on a start time 1410 in each implementation. For example, if a segment has a specific duration, the collaborative healthcare platform 140 increments the start time 1410 by a specific time interval to determine the end time 1415. In some implementations, the specific time interval is used for each segment. Alternatively, different specific time intervals can be selected for different segments, and the practitioner selects the specific time interval for the segment from a set presented by the segment selection interface 1400. Maintaining a set of specific time intervals allows the practitioner to change the duration of different segments.

[0164] In some implementations, such as Figure 14 In the example shown, the segment selection interface 1400 includes a video data timeline 1420 that can indicate specific times within video data 1405. For example, the video data timeline 1420 includes indications corresponding to each 15-minute interval in video data 1405. In response to a practitioner selecting a portion of the video data timeline 1420, the collaborative healthcare platform 140 updates the portion of video data 1405 presented by the segment selection interface 1400 to begin at the time corresponding to the selected portion of the video data timeline 1420. The video data timeline 1420 displays a segment identifier 1425 that visually indicates the start time 1410 and end time 1415 of the selected segment relative to the total duration of video data 1405, thereby allowing the practitioner to easily identify the location of the selected segment within video data 1405.

[0165] The segment specified by start time 1410 and end time 1415 includes video data occurring between start time 1410 and end time 1415. Furthermore, in various embodiments, the segment includes telemetry data captured during the execution of a medical procedure between start time 1410 and end time 1415, as well as video data 1405 captured between start time 1410 and end time 1415. The collaborative healthcare platform 140 can perform one or more preprocessing steps to synchronize the video data 1405 and telemetry data captured between start time 1410 and end time 1410. In some embodiments, the segment selection interface 1400 includes interface elements that allow practitioners to select whether a segment includes both video data and telemetry data, only video data, or only telemetry data.

[0166] Figure 15 This is an example assessment results interface 1500 that presents the assessment results for a licensed physician performing a medical procedure. The assessment results interface 1500 includes a physician identifier 1505 for the requesting physician. For example, the physician identifier 1505 may be the physician's name, image, username, or any other information that uniquely identifies the physician. In various implementations, such as Figure 15 In the example shown, the assessment results interface 1500 also presents physician information 1510 about the physician requesting the assessment. In some implementations, physician information 1510 is presented in response to interaction with physician identifier 1505. Example physician information 1510 includes: the physician's role in medical procedures, the physician's years of experience, the number of medical procedures performed by the physician, the number of medical procedures of the same type as the selected medical procedures performed by the physician for assessment, one or more previous assessment results for the physician, or other information about the physician.

[0167] The assessment results interface 1500 also presents an assessing physician identifier 1515 used to assess the physician. For example, the assessing physician identifier 1515 may be the assessing physician's name, an image of the assessing physician, or other information that uniquely identifies the assessing physician. Additionally, the assessment results interface 1500 identifies the type 1520 of the medical procedure selected for assessment. In various embodiments, performing one or more interactions with the type 1520 of the selected medical procedure results in the demonstration of surgical details 1525. Surgical details 1525 include background information about the selected medical procedure for assessment. Example surgical details 1525 include: the gender of the patient performing the procedure, the age of the patient performing the procedure, one or more physical characteristics of the patient performing the procedure, one or more other physicians associated with the procedure, or other information about the procedure.

[0168] To present the assessment results to the practicing physician, the assessment results interface 1500 includes a frame identifier 1530 that identifies the frame selected by the practicing physician for the assessment. The assessment results interface 1500 also identifies one or more categories within the frame and the corresponding assessment value for each category. Figure 15 In the example, the assessment results interface 1500 displays information identifying category 1535 (e.g., the name of category 1535) and displays the assessment value 1540 of category 1535 near the information identifying category 1535. The combination of categories and corresponding assessment values ​​in the framework includes the physician's assessment results for the medical procedure. In some embodiments, the assessment results also include an assessment summary 1545, which includes textual data or other information from the assessing physician describing the reasoning or underlying principles of one or more of the assessment values ​​1540.

[0169] The evaluation results interface 1500 also displays video data 1550 of the medical procedure selected for evaluation. In various implementations, the evaluation results interface 1500 includes a media player with one or more controls, allowing the practitioner to view or browse the video data 1550 of the medical procedure. (As described above...) Figure 2 Further described, the evaluating physician can provide one or more comments during the evaluation of a medical procedure. The comments may be associated with a specific time in video data 1550 (or in telemetry data captured during the medical procedure). The evaluation interface 1500 displays comments 1555, which in various embodiments include text data, and displays a specific time 1560 associated with comments 1555 near comments 1555. In various embodiments, the specific time 1560 identifies the start time of a portion of data (e.g., video data 1550) describing the execution of the medical procedure corresponding to comments 1555. The physician can select the specific time 1560 associated with comments 1550 (or select comments 1550 via the evaluation results interface 1500) to initiate a presentation of video data 1550 starting at the specific time 1560 via the evaluation results interface 1500, thereby simplifying the review of specific portions of the video data 1550 (or other data describing the execution of the medical procedure) corresponding to each comment. In some implementations, the evaluation results interface 1500 presents telemetry data or other data describing the execution of a medical procedure in conjunction with video data 1550 (or instead of video data 1550).

[0170] Figure 16This is an example implementation of a method for evaluating the performance of a medical procedure by a licensed physician via a collaborative medical platform 140. The collaborative medical platform 140 receives 1602 a request from the licensed physician for evaluation of the performance during the medical procedure, and 1604 a selection of medical procedures for evaluation. For example, in response to receiving request 1602, the collaborative medical platform 140 identifies medical procedures associated with the licensed physician and receives 1604 a selection of the identified medical procedures from the licensed physician. As another example, the request for evaluation of the performance during the medical procedure includes an identifier of the medical procedure to be evaluated.

[0171] Additionally, the collaborative healthcare platform 140 receives from practicing physicians 1606 selections of frameworks for evaluating chosen medical procedures. Each framework includes one or more categories corresponding to different skills or techniques applicable to performing the medical procedure. The framework also includes guidelines or standards for performing different types of actions or techniques, or includes criteria for evaluating one or more actions or techniques. In various implementations, the practicing physician selects a framework from a set of frameworks identified by the collaborative healthcare platform 140.

[0172] The collaborative healthcare platform 140 receives from a practicing physician 1608 a selection of data segments describing the execution of a chosen medical procedure. The data describing the execution of the chosen medical procedure includes video data, telemetry data, or a combination thereof captured during the execution of the chosen medical procedure. In various embodiments, the collaborative healthcare platform 140 retrieves stored data describing the execution of the chosen medical procedure, and the practicing physician selects 1608 the start and end times of the stored data describing the execution of the chosen medical procedure to identify segments. A segment includes the portion of the data describing the execution of the chosen medical procedure between the start and end times. In some embodiments, the practicing physician specifies the start time, and the collaborative healthcare platform 140 increments the start time by a specific time interval to determine the end time of the segment. In various embodiments, the specific time interval may be predefined by the collaborative healthcare platform 140 or may be specified by the practicing physician. The practicing physician may select 1608 multiple segments of data describing the execution of the medical procedure and select 1606 multiple frames, thereby allowing the use of different frames to evaluate different segments.

[0173] Based on the selected medical procedure, the selected framework, the selected assessing physician, and the selected data fragment describing the execution of the selected medical procedure, the collaborative healthcare platform 1612 generates an assessment message and sends it to the assessing physician. In response to receiving the assessment message, the assessing physician reviews the selected data fragment describing the execution of the selected medical procedure, taking into account the selected framework, and generates an assessment result including assessment values ​​for each category within the selected framework. (As described above...) Figure 2 Furthermore, the evaluation results may also include comments associated with a specific time within the selected data segment describing the execution of the medical procedure. The collaborative healthcare platform 140 provides the evaluation results through an interface (see above). Figure 15 (Further description) or other communication channels are used to present the 1614 assessment results to the practicing physician. In some implementations, the collaborative healthcare platform 140 selects one or more educational content items for the practicing physician based on the assessment results, as described above. Figure 2 Further described. For example, the collaborative healthcare platform 140 selects educational content items that are associated with the type of physician being selected and have a baseline standard that specifies a value for the category of the selected framework that is at least a threshold amount greater than the evaluation value of the category of the selected framework.

[0174] In an alternative implementation, the collaborative healthcare platform 140 applies one or more trained evaluation models to a selected data segment describing the execution of the selected medical procedure and a selected framework. (As described above...) Figure 2 Furthermore, the evaluation model generates a predicted evaluation value for the selected framework category based on selected data fragments describing the execution of the selected medical procedure. In some embodiments, the predicted evaluation value may be presented in conjunction with its corresponding category. Additionally, in various embodiments, the collaborative healthcare platform may select one or more educational content items for the practitioner based on one or more of the predicted evaluation values. Therefore, in some embodiments, the collaborative healthcare platform 140 automatically generates evaluation results using one or more trained models, rather than receiving evaluation results from the selected practitioner.

[0175] The described implementation incorporates various technological improvements that enhance the functionality of computer systems, machine learning techniques, data management systems (particularly those related to healthcare data management), computer-based user interfaces, robotic and / or other medical device systems, and other technological and technical fields. For example, the described implementation provides technological improvements in data availability and data privacy by automating the processing of sensitive and / or restricted data, such as operating room videos, patient health records, or other sensitive health data.

[0176] The described implementations also include improvements to machine learning methods, as they combine information from diverse data sources, including medical device telemetry data, video data, and mobile device data, to enhance predictive capabilities compared to traditional machine learning techniques. Furthermore, the described implementations provide technological improvements in the treatment of medical conditions by enabling the generation of various notifications, recommendations, or other content tailored to a specific practitioner, allowing that practitioner to improve their practice and thus achieve better patient outcomes.

[0177] Furthermore, the described implementation schemes include technological improvements in the field of robot-assisted surgery through the automated configuration of surgical robots based on accumulated and aggregated healthcare data associated with patients, facilities, and physicians. This results in improved robot performance, enhanced human-robot interaction, and improved patient outcomes.

[0178] For illustrative purposes, the foregoing description of the implementation scheme has been given; however, this description is not intended to be exhaustive or to limit the implementation scheme to the specific form disclosed. Those skilled in the art will understand that many modifications and variations are possible based on the foregoing disclosure.

[0179] Some parts of this description describe the implementation scheme based on the symbolic representation of algorithms and information operations. These operations, when described functionally, computationally, or logically, are understood to be implemented by computer programs or equivalent circuits, microcode, etc. Furthermore, arranging these operations as modules has proven convenient in some cases, without loss of generality. The described operations and their associated modules can be embodied in software, firmware, hardware, or any combination thereof.

[0180] Any of the steps, operations, or processes described herein may be performed or implemented, alone or in combination with other means, using one or more hardware or software modules. Implementations may also involve devices for performing the operations described herein. Such devices may be specifically configured for the desired purpose and / or may include general-purpose computing devices selectively started or reconfigured by computer programs stored in a computer. Such computer programs may be stored in a tangible, non-transitory, computer-readable storage medium or any type of medium suitable for storing electronic instructions and coupled to a computer system bus. Furthermore, any computing system mentioned in this specification may include a single processor or may be an architecture employing a multiprocessor design to enhance computing power.

[0181] As used herein, and unless explicitly stated otherwise, "or" refers to an inclusive "or" rather than an exclusive "or". For example, the condition "A or B" is satisfied by any of the following: A is true (or exists), and B is false (or does not exist); A is false (or does not exist), and B is true (or exists); and both A and B are true (or exist). Similarly, the condition "A, B, or C" is satisfied by any combination of A, B, and C being true (or existing). As a non-restrictive example, the condition "A, B, or C" is satisfied when A and B are true (or exist) and C is false (or does not exist). Similarly, as another non-restrictive example, the condition "A, B, or C" is satisfied when A is true (or exists) and B and C are false (or do not exist).

[0182] Finally, the language used in this specification has been chosen primarily for readability and guidance purposes and may not be intended to depict or limit the subject matter of the invention. Therefore, the scope of the invention is not limited by this detailed description, but rather by any of the claims published in the application based thereon. Thus, the disclosure of the embodiments is intended to illustrate rather than limit the scope of the invention, which is set forth in the appended claims.

Claims

1. A method for evaluating a licensed physician performing medical procedures through an online collaborative medical platform, the method comprising: The physician receives the assessment request at the collaborative medical platform. The physician receives a selection of the medical procedure for evaluation at the collaborative medical platform. The collaborative medical platform receives a selection of a framework for evaluating the practicing physician based on the medical procedure, the framework including categories corresponding to different skills or techniques suitable for performing the medical procedure; The physician receives the selection of the assessing physician at the collaborative medical platform. Receive selection of a data segment describing the execution of the medical procedure by the practicing physician, the segment including a portion of the data describing the execution of the medical procedure between the start time and the end time; The assessment results are obtained from the assessing physician, and the assessment results include assessment values ​​from the assessing physician for each category of the framework; as well as The assessment results are presented to the practicing physician.

2. The method according to claim 1, further comprising: Select an educational content item associated with at least one baseline criterion that is not met by the evaluation value for the category of the framework, the educational content item being associated with the type of medical procedure; as well as The selected educational content items are presented to the practicing physician.

3. The method of claim 2, wherein selecting the educational content item associated with at least one baseline criterion not met by the evaluation value for the category of the framework, wherein the educational content item associated with the type of medical procedure includes: Select an educational content item with a baseline standard, the baseline standard specifying a value for the category of the framework that is at least a threshold amount greater than the evaluation value for the category of the framework.

4. The method of claim 2, wherein selecting the educational content item associated with at least one baseline criterion not met by the evaluation value for the category of the framework, wherein the educational content item associated with the type of medical procedure includes: A composite value for the category of the framework is determined based on the assessment results for the category of the framework and one or more previously determined assessment results for the category of the framework for the practicing physician. as well as Select an educational content item associated with at least one baseline criterion that is not met by the comprehensive value for the category of the framework, the educational content item being associated with the type of medical procedure.

5. The method according to claim 1, further comprising: An evaluation model is applied to the fragments and frames of data describing the execution of the medical procedure, and the evaluation model generates a predicted evaluation value for a category of the frame based on the fragments of data describing the execution of the medical procedure; The evaluation model is scored using a loss function based on the difference between the predicted evaluation value for the category of the framework and the evaluation value for the category of the framework. as well as Based on the score, one or more parameters of the evaluation model are updated through backpropagation.

6. The method of claim 1, wherein the data describing the execution of the medical procedure includes video data captured during the execution of the medical procedure and telemetry data captured by one or more sensors during the execution of the medical procedure.

7. The method of claim 1, wherein the assessment result further includes one or more comments from the assessing physician, the comments being associated with a specific time in the segment of data describing the performance of the medical procedure.

8. The method of claim 1, wherein receiving the selection of the segment describing the data of the medical procedure performed by the practicing physician comprises: The practitioner receives the selection of the start time within the segment of data describing the execution of the medical procedure by the practitioner; as well as The collaborative medical platform determines the end time by adding a specific time interval to the start time.

9. A non-transitory computer-readable storage medium storing instructions for evaluating a practicing physician performing a medical procedure via an online collaborative medical platform, the instructions, when executed by one or more processors, causing the one or more processors to perform the following steps: The physician receives the assessment request at the collaborative medical platform. The physician receives a selection of the medical procedure for evaluation at the collaborative medical platform. The collaborative medical platform receives a selection of a framework for evaluating the practicing physician based on the medical procedure, the framework including categories corresponding to different skills or techniques suitable for performing the medical procedure; The physician receives the selection of the assessing physician at the collaborative medical platform. Receive selection of a data segment describing the execution of the medical procedure by the practicing physician, the segment including a portion of the data describing the execution of the medical procedure between the start time and the end time; The assessment results are obtained from the assessing physician, and the assessment results include assessment values ​​from the assessing physician for each category of the framework; and The assessment results are presented to the practicing physician.

10. The non-transitory computer-readable storage medium of claim 9, wherein the non-transitory computer-readable storage medium further stores instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: Select educational content items associated with at least one baseline criterion that is not met by the evaluation value for the category of the framework, wherein the educational content items are associated with the type of medical procedure; and The selected educational content items are presented to the practicing physician.

11. The non-transitory computer-readable storage medium of claim 10, wherein the educational content items associated with at least one baseline criterion not met by the evaluation value for the category of the framework, wherein the educational content items associated with the type of the medical procedure include: Select an educational content item with a baseline standard, the baseline standard specifying a value for the category of the framework that is at least a threshold amount greater than the evaluation value for the category of the framework.

12. The non-transitory computer-readable storage medium of claim 10, wherein the selection of the educational content items associated with at least one baseline criterion not met by the evaluation value for the category of the framework, wherein the educational content items associated with the type of the medical procedure include: A composite value for the category of the framework is determined based on the assessment results for the category of the framework and one or more previously determined assessment results for the category of the framework for the practicing physician. as well as Select an educational content item associated with at least one baseline criterion that is not met by the comprehensive value for the category of the framework, the educational content item being associated with the type of medical procedure.

13. The non-transitory computer-readable storage medium of claim 9, wherein the non-transitory computer-readable storage medium further stores instructions, which, when executed by one or more processors, cause the one or more processors to perform the following steps: An evaluation model is applied to the fragments and frames of data describing the execution of the medical procedure, and the evaluation model generates a predicted evaluation value for a category of the frame based on the fragments of data describing the execution of the medical procedure; The evaluation model is scored using a loss function based on the difference between the predicted evaluation value for the category of the framework and the evaluation value for the category of the framework. as well as Based on the score, one or more parameters of the evaluation model are updated through backpropagation.

14. The non-transitory computer-readable storage medium of claim 9, wherein the data describing the execution of the medical procedure includes video data captured during the execution of the medical procedure and telemetry data captured by one or more sensors during the execution of the medical procedure.

15. The non-transitory computer-readable storage medium of claim 9, wherein the evaluation result further includes one or more comments from the evaluating physician, the comments being associated with a specific time in the fragment of data describing the execution of the medical procedure.

16. The non-transitory computer-readable storage medium of claim 9, wherein receiving the selection of the segment of data describing the performance of the medical procedure by the practicing physician comprises: The practitioner receives the selection of the start time within the segment of data describing the execution of the medical procedure by the practitioner; as well as The collaborative medical platform determines the end time by adding a specific time interval to the start time.

17. A method for evaluating a practicing physician performing medical procedures through an online collaborative medical platform, the method comprising: The physician receives the assessment request at the collaborative medical platform. The physician receives a selection of the medical procedure for evaluation at the collaborative medical platform. The collaborative medical platform receives a selection of a framework for evaluating the practicing physician based on the medical procedure, the framework including categories corresponding to different skills or techniques suitable for performing the medical procedure; Receive selection of a data segment describing the execution of the medical procedure by the practicing physician, the segment including a portion of the data describing the execution of the medical procedure between the start time and the end time; The collaborative healthcare platform generates evaluation results by combining an evaluation model applied to the framework with fragments of data describing the execution of the medical procedure. The evaluation results include predicted evaluation values ​​for categories within the framework. The evaluation model is trained in the following manner: Multiple training examples are obtained, each training example including a training fragment and a training framework describing the data of training the execution of a medical procedure, and each training example has a label indicating an evaluation value for the category based on the training framework and the training fragment; The evaluation model is applied to each training example to generate a predicted evaluation value for the category based on the training fragment and the training framework; The evaluation model is scored using the loss function and the labels of the training examples; as well as The evaluation model is updated using backpropagation based on the scores until one or more criteria are met. as well as The assessment results are presented to the practicing physician.

18. The method according to claim 17, further comprising: Select an educational content item associated with at least one baseline criterion that is not met by the evaluation value for the category of the framework, the educational content item being associated with the type of medical procedure; as well as The selected educational content items are presented to the practicing physician.

19. The method of claim 17, wherein the data describing the execution of the medical procedure includes video data captured during the execution of the medical procedure and telemetry data captured by one or more sensors during the execution of the medical procedure.

20. The method of claim 17, wherein the assessment result further includes one or more comments from the assessing physician, the comments being associated with a specific time in the segment of data describing the performance of the medical procedure.

Images